system

Abstract

Provide a system. 【Solutions】 Means for acquiring visual data in real time, Means for recording and learning past movement data, Means for analyzing data to predict future movement situations, Means for operating a control device to optimize the flow of movement, Means for providing movement situation information to users, Means for proposing an optimal movement route and departure time based on the user's current location and destination, Means for guiding changes to the movement route during real time, A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In traffic volume surveys and urban planning, it is important to obtain and analyze accurate traffic data in real time. However, in conventional methods, manual tabulation is the mainstream, which is not only inefficient from the perspectives of labor costs and time, but also prone to human errors and difficult for long-term data collection. Due to these problems, there is an issue that efficient management and prediction of traffic cannot be sufficiently performed.

Means for Solving the Problems

[0005] This invention provides a system that analyzes traffic conditions in real time and predicts future traffic. Specifically, by using a device that acquires video data in real time and a device that records and learns from past traffic data, the system constantly reflects new information in the model to perform highly accurate analysis. Furthermore, by operating signal control devices based on this analysis and optimizing traffic flow, it achieves efficient traffic management and accident prevention. In addition, by providing users with real-time traffic conditions, it supports individual travel planning.

[0006] "Real-time" refers to the process of processing and analyzing data immediately upon acquisition.

[0007] "Video data" refers to digital data containing visual information acquired by a camera or similar device.

[0008] A "device" is a set of hardware or software components assembled to perform a specific function or process.

[0009] "Traffic data" refers to information that indicates traffic conditions, including data such as the number of vehicles passing through, their speed, routes, and congestion levels.

[0010] "Learning" is the process of algorithms used to analyze past data and make predictions and decisions based on that analysis.

[0011] "Analysis" is the process of examining complex data and finding useful information or patterns within it.

[0012] "Prediction" is the process of estimating future conditions based on past and present data.

[0013] A "traffic signal control system" is a system or device used to operate traffic signals and manage the flow of traffic.

[0014] "Optimization" is a process of adjustment to achieve the most effective or efficient results for a specific purpose.

[0015] "User" refers to a person or organization that uses a system or service and is the target of information provision.

Brief Description of Drawings

[0016] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10] It shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), etc.

[0020] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention is a system for understanding traffic conditions in real time and predicting future traffic conditions. This system operates through the following process to automate and streamline traffic management.

[0038] First, terminals (smart cameras installed on the streets) continuously acquire video data of the road and transmit this data to a server in real time. The server analyzes the received video data and automatically identifies the number, speed, and direction of movement of vehicles and pedestrians. Here, an AI-based object detection algorithm is used to achieve highly accurate analysis.

[0039] The analysis results are recorded in a digital database as current traffic conditions. The server uses this data to learn past traffic patterns. This makes it possible to make more accurate traffic predictions for specific dates and conditions.

[0040] Through an interface to the signal control system, the server sends instructions to the traffic lights based on real-time analysis results. As a result, traffic flow is optimized for smoother operation, and congestion is reduced as much as possible.

[0041] Users can obtain real-time traffic information and forecasts through a dedicated application or web portal. The interface for this information is designed to be intuitively understandable to users. Users can use this information to select the optimal travel route and departure time, resulting in a more comfortable and efficient journey.

[0042] As a concrete example, consider the weekday morning rush hour in a certain city. This system learns rush hour patterns from past data and predicts the traffic volume for the day. Based on this prediction, traffic signal timing is dynamically adjusted to manage traffic flow smoothly. This has the effect of shortening commute times and reducing stress caused by peak traffic.

[0043] The following describes the processing flow.

[0044] Step 1:

[0045] The device (smart camera) acquires video data from its installed location. The camera continuously captures images of the road and adds a timestamp to them.

[0046] Step 2:

[0047] The terminal compresses the acquired video data and efficiently transmits it to the server. An appropriate compression algorithm is used to optimize communication bandwidth.

[0048] Step 3:

[0049] The server receives the transmitted video data and inputs it into the data analysis pipeline. At this point, the video data is input into the real-time object detection model.

[0050] Step 4:

[0051] The server uses an AI-based object detection algorithm to identify vehicles and pedestrians. This allows for the analysis of traffic flow characteristics (number of vehicles, speed, direction of travel).

[0052] Step 5:

[0053] The server stores traffic information generated from raw data in a database. This information is used in subsequent learning processes and real-time monitoring.

[0054] Step 6:

[0055] The server analyzes recorded traffic data and uses a predictive model to estimate future traffic conditions. The prediction results are generated considering traffic patterns for specific days of the week and time periods.

[0056] Step 7:

[0057] The server uses the prediction results to send instructions to the traffic signal control system. The timing of the traffic lights is dynamically adjusted according to the traffic conditions at that time.

[0058] Step 8:

[0059] Users receive current and forecast traffic information through a dedicated application or web portal, enabling them to plan their travel efficiently.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] In modern metropolises, traffic congestion has a significant impact on economic activity and quality of life. However, current traffic management systems have limitations in real-time understanding and prediction of traffic conditions, preventing the implementation of effective congestion mitigation measures. To solve this problem, it is necessary to analyze traffic conditions in real time and realize efficient signal control and the provision of traffic information.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes means for acquiring video information in real time, means for analyzing the acquired video information to identify the state of vehicles and pedestrians, and means for recording and learning past traffic information. This makes it possible to grasp and analyze the situation in real time at each traffic point, and even to accurately predict future traffic conditions.

[0065] "Means for acquiring video information in real time" refers to a device or function that continuously records road conditions and collects that video information without delay.

[0066] "Means of analyzing acquired video information to identify the status of vehicles and pedestrians" refers to the process of analyzing collected video and using image recognition technology to identify the number, speed, and direction of travel of traffic participants.

[0067] "Methods for recording and learning from past traffic information" refers to methods that utilize machine learning techniques to save past traffic data and use that data to strengthen predictive models.

[0068] "Methods for predicting future traffic conditions based on analytical information" refer to technologies that estimate future traffic flow at specific times and locations by referring to current and past traffic patterns.

[0069] "Means of optimizing traffic flow by operating traffic control devices" refers to methods of adjusting traffic signals and other control devices to improve the efficiency of vehicle flow on roads.

[0070] "Means of providing traffic information to users" refers to methods of sharing current traffic conditions and forecast information in a format that is easily accessible to the general public.

[0071] To implement this invention, a system is constructed using the following hardware and software. First, the terminal functions as a smart camera installed along the roadside, and this camera continuously acquires high-resolution video information. The video information is continuously acquired in real time and transmitted quickly to the server using compression technology.

[0072] The server analyzes the received video information using AI-based object detection algorithms, such as YOLO and Faster R-CNN. This allows it to identify the number of vehicles in the video, the presence of pedestrians, and the speed and direction of each object. Simultaneously, it learns this information by referencing previously recorded traffic data and using machine learning techniques.

[0073] Based on the data analyzed by this server, it can issue instructions to traffic control systems to adjust signal timing. This dynamically adjusts traffic flow and reduces congestion in real time. Furthermore, users can obtain current and predicted traffic information in an intuitive and easy-to-understand format through a dedicated application or web portal. Based on this information, users can select the optimal route and departure time.

[0074] To give a concrete example, consider the morning rush hour in a major city. This system learns past rush hour patterns and predicts traffic volume for a particular day. By dynamically adjusting traffic light timing based on the prediction, traffic flow becomes smoother, reducing commuting time and stress.

[0075] An example of a prompt to input into a generative AI model is, "Analyze traffic patterns in a specific area, generate a predictive model, and propose ways to use it for traffic signal control." This allows the generative AI to propose a variety of approaches to traffic optimization.

[0076] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0077] Step 1:

[0078] The terminal uses smart cameras installed on the street to acquire real-time video footage of the road. The input is high-resolution video data captured by the cameras. The terminal converts this video data into a digital format and reduces the data size using a compression algorithm. The output is the compressed video data, which is sent to a server for subsequent analysis processing.

[0079] Step 2:

[0080] The server decompresses the compressed video data received from the terminal and detects vehicles and pedestrians within the video. The input is the decompressed video data. The server applies AI-based object detection algorithms (e.g., YOLO, Faster R-CNN) to identify the number, position, speed, and direction of travel of each object. The output is detailed traffic data based on the analysis results.

[0081] Step 3:

[0082] The server uses machine learning algorithms to predict future traffic conditions by referencing analysis results and historical traffic data. The input consists of current analysis data and stored historical traffic data. In this process, the server learns specific patterns and generates a model to predict future traffic flow. The output is data on predicted traffic patterns and congestion levels.

[0083] Step 4:

[0084] The server sends signal timing adjustment commands to the signal control system based on predictions. Here, the input is predicted traffic condition data, and the output is adjusted signal timing information. The server performs this in real time, and optimization is achieved through the traffic control system.

[0085] Step 5:

[0086] Users obtain the latest traffic and forecast information using a dedicated application or web portal. The input is the user's request (e.g., obtaining information for a specific route), and the server organizes and outputs the analysis results and forecast information. Based on this, users can optimize their travel routes and departure times.

[0087] (Application Example 1)

[0088] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0089] Urban traffic congestion is a major challenge, leading to increased travel times and environmental burdens. Furthermore, it is difficult for users to choose the optimal route and time, necessitating improvements in individual travel efficiency. Against this backdrop, there is a need to develop a system that utilizes real-time traffic information to provide effective route suggestions to users.

[0090] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0091] In this invention, the server includes means for acquiring visual data in real time, means for recording and learning past travel data, and means for analyzing the data to predict future travel conditions. As a result, users can receive suggestions for optimal travel routes and departure times, and real-time guidance on changes to their travel routes.

[0092] "Means of acquiring visual data in real time" refers to devices that use cameras and sensors installed on the streets to instantly collect data on current road conditions and traffic flow.

[0093] "Means of recording and learning from past movement data" refers to algorithms and systems that store previous traffic data, analyze it to identify specific patterns and trends, and predict future situations.

[0094] "A means of analyzing data to predict future travel patterns" refers to a function that uses AI technology to analyze traffic patterns based on collected data and show future traffic flows.

[0095] "Means of optimizing traffic flow by operating control devices" refers to technologies that smooth traffic flow and alleviate congestion by dynamically adjusting traffic signals and traffic control devices.

[0096] "Means of providing users with mobility information" refers to interfaces and applications that convey traffic conditions and forecast information to users via mobile devices or computers.

[0097] "A means of suggesting the optimal travel route and departure time based on the user's current location and destination" refers to a function that utilizes collected traffic data to calculate and present the most efficient route and appropriate departure time for the user.

[0098] A "means of providing real-time route change guidance" refers to a system that, in response to changes in traffic conditions, proposes new routes to users, supporting efficient travel to their destinations.

[0099] To implement this invention, terminals, servers, and users primarily play their respective roles. The terminal is a visual data acquisition device installed on the street, which acquires real-time traffic flow and conditions as video data. This visual data is immediately transmitted to the server, which is a data management device. The server uses AI technology to analyze the data and learn past traffic patterns. AI-based object detection algorithms such as TENSORFLOW® are used for this analysis and prediction process.

[0100] The server also optimizes travel routes and departure times for users. Specifically, it calculates and suggests the optimal route and departure time based on the user's current location and destination via a smartphone application. Furthermore, based on real-time traffic data, it notifies users of route changes if traffic conditions change during their journey. This allows users to avoid traffic congestion and arrive at their destination more efficiently.

[0101] The hardware primarily consists of a group of cameras for acquiring video data and smartphones. Database systems built on cloud services such as AWS® and Google® Cloud are used for data processing and analysis.

[0102] As a concrete example, consider the morning commute. When a user starts their journey from home to the office, they can use this app to check the optimal route and departure time, allowing them to avoid traffic and reach their destination quickly. Furthermore, real-time route suggestions that adapt to changing circumstances during the journey improve the user's travel efficiency.

[0103] An example of a prompt to a generative AI model is, "Check today's traffic conditions and tell me the fastest route to my destination." This prompt is used to generate information about the user's destination and the most efficient route possible.

[0104] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0105] Step 1:

[0106] The terminal uses cameras installed on the street to acquire real-time visual data of traffic conditions. The input is live video of the traffic scene, and this data is compressed and sent to the server as output. The data processing performed at this stage is the compression of the video stream.

[0107] Step 2:

[0108] The server receives visual data transmitted from the terminal and analyzes the data using an AI-based object detection algorithm. The input is compressed traffic video data, and the output is the analysis results, such as the number of vehicles, speed, and direction of travel. This analysis includes data computation using TensorFlow and other tools.

[0109] Step 3:

[0110] The server utilizes historical travel data to learn traffic patterns based on the analysis results. The input consists of current analysis results and historical traffic data. The output generates predictive information about future traffic conditions. This data processing includes pattern recognition and predictive model construction processes.

[0111] Step 4:

[0112] The user enters their current location and destination via a smartphone application. The application sends a request to a server, which retrieves the optimal travel route and departure time. The input consists of the user's current location, destination, and traffic forecast data, while the output is recommended route information.

[0113] Step 5:

[0114] The server sends information to the application to update the user's route as they travel, based on real-time traffic conditions. The input is the latest traffic data and the user's progress, and the output is a suggested updated route. This allows the user to be presented with the optimal route based on prompts generated using a generative AI model.

[0115] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0116] This invention combines a system that analyzes traffic conditions in real time and predicts future traffic with an emotion engine that recognizes user emotions. This system streamlines traffic management and enables the provision of information tailored to the user.

[0117] First, the terminal (smart camera) continuously acquires video data of the road, compresses it, and sends it to the server. The server uses AI to analyze the video data and records the current traffic situation as numerical data. Furthermore, it learns patterns from the accumulated data and predicts future traffic conditions.

[0118] This system includes an emotion engine to recognize user emotions. Users provide emotional data to the system through a dedicated device or application. The emotion engine analyzes this data to detect the user's stress level and other emotions. The detection results are used to adjust the traffic information provided; for example, users with high stress levels can be provided with more relaxed route information.

[0119] The emotion engine is also incorporated into the traffic signal control process. For example, if driver stress levels are generally high in a particular area, the server uses that information to adjust the signal patterns, making traffic flow smoother and thus reducing stress.

[0120] As a concrete example, consider a scenario where this system is operational during an event in a certain city. The system uses not only normal traffic data but also participant emotion data to guide traffic. As a result, it can adjust traffic light timing at points where congestion is predicted and provide relaxing voice guidance on some routes, creating an optimal traffic environment that takes user comfort into consideration.

[0121] The following describes the processing flow.

[0122] Step 1:

[0123] The terminal acquires road video data in real time. The camera adds a timestamp to each frame and sends the captured data to the server at regular intervals.

[0124] Step 2:

[0125] The video data acquired by the terminal is compressed and sent to the server in a format optimized for efficient data transfer.

[0126] Step 3:

[0127] The server analyzes the received video data. Using AI-based object detection algorithms, it identifies vehicles and pedestrians to understand traffic conditions.

[0128] Step 4:

[0129] The server stores the analysis results in a database. From this data, it learns past and present traffic trends and updates the model to predict future traffic conditions.

[0130] Step 5:

[0131] Users use a dedicated application to send emotional data from their device to a server. This emotional data includes information indicating the user's stress level and current emotional state.

[0132] Step 6:

[0133] The emotion engine on the server analyzes the emotion data sent by the user to identify stress levels and other emotional states.

[0134] Step 7:

[0135] The server adjusts the content of traffic information provided based on the user's emotional state. For users experiencing high stress levels, it suggests relaxing routes and information.

[0136] Step 8:

[0137] The server aggregates user sentiment data from across the region and uses this information to optimize traffic signal control parameters. For example, if stress levels are high in a specific area, the signal timing is adjusted to optimize traffic flow.

[0138] Step 9:

[0139] Users plan their travel based on the provided traffic information. They can use the application to select the most comfortable route and time.

[0140] (Example 2)

[0141] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0142] Current traffic management systems have limitations in real-time traffic information analysis and signal control, and in particular, they have not been able to provide optimal traffic information based on the emotional state of individual users. Therefore, there is a need to improve traffic flow while also reducing the psychological stress on users.

[0143] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0144] In this invention, the server includes means for acquiring video data in real time, means for compressing the acquired video data and transmitting it to the server, means for recording and learning past traffic data, means for predicting future traffic conditions based on the recorded data, means for collecting and analyzing user emotion data, means for providing traffic information according to the user's emotional state, and means for dynamically optimizing traffic signal control based on the user's emotions and traffic conditions. This makes it possible to manage traffic conditions more precisely and realize a comfortable traffic environment by providing information adapted to the user.

[0145] "Real-time" refers to a state where data acquisition and processing occur instantly, without any delay.

[0146] "Video data" refers to a collection of visual information acquired by a camera or other recording device.

[0147] "Compression" is a technique that reduces the amount of data, making it easier to transmit and store that data.

[0148] A "server" is a computer system that provides services to other devices over a network.

[0149] "Traffic data" refers to information about the volume, speed, and flow of vehicles on roads.

[0150] "Learning" is the process by which an algorithm discovers patterns and features from data and utilizes that information.

[0151] "Predicting future traffic conditions" means estimating future traffic flow and congestion based on past and present traffic data.

[0152] "User emotional data" refers to information that indicates a user's psychological state and stress level.

[0153] "Analysis" is the process of processing and examining data to derive meaning and trends from it.

[0154] "Providing traffic information" means informing users about road conditions and recommended routes.

[0155] "Traffic signal control" is the process of adjusting the flow of vehicles on a road by operating traffic signals.

[0156] "Dynamic optimization" means adjusting to the most suitable state in real time according to the situation.

[0157] This invention combines a system that analyzes traffic conditions in real time and predicts future traffic with an emotion engine that recognizes user emotions. This system streamlines traffic management and enables the provision of information tailored to the user.

[0158] First, a smart camera, acting as a terminal, continuously acquires video data of the road, compresses this data, and sends it to a server. The server analyzes the video data using AI technology. Specifically, it utilizes computer vision technology to quantify the number, speed, and direction of vehicles on the road. The analyzed data is stored in a database and recorded as historical traffic data. Then, a machine learning model is used to learn patterns from this data and predict future traffic conditions.

[0159] Furthermore, users provide emotional data to the system through a dedicated device or application. This data is analyzed by an emotion engine to detect the user's stress level and other emotional states. Based on these detection results, personalized traffic information is provided to the user. Specifically, for users with high stress levels, it is possible to suggest calmer routes or provide relaxation messages via voice.

[0160] Furthermore, the system can also incorporate emotional data into the traffic signal control process. If a high level of stress is detected overall in a particular area, the server will adjust the signal patterns based on that information to smooth the overall traffic flow. This helps reduce stress throughout the area.

[0161] For example, when a large-scale event is held in a city, this system analyzes both normal traffic data and the emotional data of event participants to provide optimal traffic guidance. It can adjust traffic light timing at predicted congestion points and enhance user comfort by providing relaxing messages through voice guidance on specific routes.

[0162] An example of a prompt for a generative AI model is, "Based on road congestion and participant sentiment data, please suggest the optimal traffic route and signal control method for the day of the event." This prompt allows the AI ​​to comprehensively analyze diverse information and support decision-making.

[0163] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0164] Step 1:

[0165] The terminal uses smart cameras installed on the road to acquire video data in real time. The input is video of the road, and the output is compressed video data. Compressing this video data reduces network load and enables rapid data transfer. Specifically, frames are captured continuously, and compression is applied at regular intervals.

[0166] Step 2:

[0167] The server receives compressed video data transmitted from the terminal and analyzes it using AI technology. The input is compressed video data, and the output is analyzed numerical data (number of vehicles, speed, direction, etc.). Specifically, computer vision algorithms are applied to quantify the traffic flow on the road.

[0168] Step 3:

[0169] The server stores the analyzed traffic data in a database and trains a machine learning model using historical data. The input is digitized traffic data, and the output is a predictive model. Specifically, it applies machine learning algorithms to learn traffic patterns and generates a model that predicts future traffic conditions.

[0170] Step 4:

[0171] Users input emotional data through a dedicated device or application and send it to a server. The input is emotional information obtained from the user's voice and face, and the output is analyzed emotional data. Specifically, emotion recognition software is used to process the data and determine the user's stress level.

[0172] Step 5:

[0173] The server analyzes the user's emotional data and compares it with current traffic conditions to provide personalized traffic information. The input is the analyzed emotional data and traffic data, and the output is personalized traffic guidance information. In actual processing, the system generates the optimal route and relaxation message for the user through the combination of data.

[0174] Step 6:

[0175] The server utilizes user sentiment data to dynamically optimize traffic signal timing. Inputs are traffic data and sentiment data, and output is an adjusted signal control pattern. Specifically, at a given intersection, it adjusts signal switching times considering the overall user state to improve traffic flow.

[0176] (Application Example 2)

[0177] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0178] Conventional traffic management systems have limitations in efficiently analyzing traffic conditions and do not provide information that takes into account the emotional state of users, resulting in problems with ensuring sufficient user comfort and safety. In particular, with autonomous vehicles, it was difficult to reduce stress because driving parameters could not be adjusted based on passenger emotions.

[0179] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0180] In this invention, the server includes means for acquiring and analyzing video data in real time, means for recording and learning from past traffic data, means for analyzing the user's emotional state and adjusting the content of traffic information provided, and means for automatically adjusting the vehicle's driving parameters based on the passenger's emotions. This improves the accuracy of traffic situation prediction and enables driving and information provision that takes into account the user's emotions.

[0181] A "device that acquires video data in real time" is a device that continuously acquires video data using sensors such as cameras in order to instantly grasp traffic conditions, and provides it in an analyzable format.

[0182] A "device for recording and learning from past traffic data" is a device that accumulates historical data on traffic flow and vehicle movement over a long period of time, generates identification patterns based on this data, and creates a traffic prediction model.

[0183] A "device that analyzes data to predict future traffic conditions" is a device that takes current and past traffic data as input and predicts future traffic conditions and flow rates through analysis of that data.

[0184] A "device that optimizes traffic flow by operating signal control devices" is a device that automatically adjusts the timing and sequence of traffic signals to efficiently manage traffic flow, thereby promoting the reduction of congestion and traffic accidents.

[0185] A "device that analyzes the user's emotional state and adjusts the content of traffic information provided" is a device that analyzes emotional data acquired from the user's device and, based on that analysis, provides traffic information that is adapted to the user's stress level and emotions.

[0186] A "device that automatically adjusts vehicle driving parameters based on passenger emotions" is a device that enhances passenger comfort and safety by dynamically changing the driving characteristics of an autonomous vehicle, such as speed and route, based on the passenger's emotional response.

[0187] In order to implement this invention, the interaction between the server, terminal, and user is of primary importance.

[0188] The server plays a central role in traffic situation analysis. First, it receives video data transmitted in real time from terminals and analyzes traffic conditions using AI models. Machine learning models built with TensorFlow and PyTorch are used for the analysis. Based on the results, it predicts future traffic conditions, issues appropriate instructions to traffic signal control devices, and optimizes traffic flow.

[0189] Simultaneously, the server analyzes the user's emotional data. Emotional data is collected from the user's device (smartphone or wearable device) and processed using an emotion recognition model. Based on the analysis, information and driving instructions tailored to the user's stress level are provided. For example, if high stress is detected, the vehicle's driving parameters are adjusted, and a stress reduction mode is activated. This provides a more relaxed driving experience.

[0190] As a concrete example, in complex urban traffic situations, the user's emotions are analyzed in real time, and a smooth and safe route is suggested. Based on this, the server can automatically adjust the vehicle's speed and route, and can even stream relaxing music.

[0191] As an example of a prompt, the AI ​​model can be instructed with a message such as, "Based on the current emotional state and traffic data, please suggest the safest and most efficient route," thereby achieving user-adaptive output.

[0192] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0193] Step 1:

[0194] The terminal acquires road video data in real time. The acquired video data is resized and compressed for efficient handling and sent to the server. This process allows the terminal to provide data in a format that is easy for the server to handle.

[0195] Step 2:

[0196] The server receives compressed video data transmitted from the terminal and decompresses it. An AI model is used to analyze this data and extract current traffic information. Specifically, image recognition technology is used to count the number of vehicles and pedestrians, generating traffic flow data. This data then becomes the input for the next prediction step.

[0197] Step 3:

[0198] The server receives previously accumulated traffic data and current traffic data as input and uses a generating AI model to predict future traffic conditions. The model performs time-series analysis and outputs predicted traffic congestion points and time periods. The resulting future prediction data is used to adjust traffic signal control.

[0199] Step 4:

[0200] Users input emotional data through a dedicated device or smartphone. This emotional data is analyzed in real time by AI, and the user's stress level and mental state are quantified. The emotional state obtained from the analysis is used to adjust the traffic information provided by the server.

[0201] Step 5:

[0202] The server integrates analyzed traffic and sentiment data to instruct signal control units on the optimal signal timing. It also provides users with optimized route suggestions and driving advice. As a result, users can experience safer and more comfortable travel.

[0203] Step 6:

[0204] To influence the user, the server streams relaxation-enhancing content (music and messages) within the autonomous vehicle. This content, tailored to the user's emotional state, directly enhances the driving experience.

[0205] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0206] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0207] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0208] [Second Embodiment]

[0209] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0210] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0211] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0212] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0213] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0214] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0215] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0216] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0217] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0218] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0219] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0220] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0221] This invention is a system for understanding traffic conditions in real time and predicting future traffic conditions. This system operates through the following process to automate and streamline traffic management.

[0222] First, terminals (smart cameras installed on the streets) continuously acquire video data of the road and transmit this data to a server in real time. The server analyzes the received video data and automatically identifies the number, speed, and direction of movement of vehicles and pedestrians. Here, an AI-based object detection algorithm is used to achieve highly accurate analysis.

[0223] The analysis results are recorded in a digital database as current traffic conditions. The server uses this data to learn past traffic patterns. This makes it possible to make more accurate traffic predictions for specific dates and conditions.

[0224] Through an interface to the signal control system, the server sends instructions to the traffic lights based on real-time analysis results. As a result, traffic flow is optimized for smoother operation, and congestion is reduced as much as possible.

[0225] Users can obtain real-time traffic information and forecasts through a dedicated application or web portal. The interface for this information is designed to be intuitively understandable to users. Users can use this information to select the optimal travel route and departure time, resulting in a more comfortable and efficient journey.

[0226] As a concrete example, consider the weekday morning rush hour in a certain city. This system learns rush hour patterns from past data and predicts the traffic volume for the day. Based on this prediction, traffic signal timing is dynamically adjusted to manage traffic flow smoothly. This has the effect of shortening commute times and reducing stress caused by peak traffic.

[0227] The following describes the processing flow.

[0228] Step 1:

[0229] The device (smart camera) acquires video data from its installed location. The camera continuously captures images of the road and adds a timestamp to them.

[0230] Step 2:

[0231] The terminal compresses the acquired video data and efficiently transmits it to the server. An appropriate compression algorithm is used to optimize communication bandwidth.

[0232] Step 3:

[0233] The server receives the transmitted video data and inputs it into the data analysis pipeline. At this point, the video data is input into the real-time object detection model.

[0234] Step 4:

[0235] The server uses an AI-based object detection algorithm to identify vehicles and pedestrians. This allows for the analysis of traffic flow characteristics (number of vehicles, speed, direction of travel).

[0236] Step 5:

[0237] The server stores traffic information generated from raw data in a database. This information is used in subsequent learning processes and real-time monitoring.

[0238] Step 6:

[0239] The server analyzes recorded traffic data and uses a predictive model to estimate future traffic conditions. The prediction results are generated considering traffic patterns for specific days of the week and time periods.

[0240] Step 7:

[0241] The server uses the prediction results to send instructions to the traffic signal control system. The timing of the traffic lights is dynamically adjusted according to the traffic conditions at that time.

[0242] Step 8:

[0243] Users receive current and forecast traffic information through a dedicated application or web portal, enabling them to plan their travel efficiently.

[0244] (Example 1)

[0245] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0246] In modern metropolises, traffic congestion has a significant impact on economic activity and quality of life. However, current traffic management systems have limitations in real-time understanding and prediction of traffic conditions, preventing the implementation of effective congestion mitigation measures. To solve this problem, it is necessary to analyze traffic conditions in real time and realize efficient signal control and the provision of traffic information.

[0247] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0248] In this invention, the server includes means for acquiring video information in real time, means for analyzing the acquired video information to identify the state of vehicles and pedestrians, and means for recording and learning past traffic information. This makes it possible to grasp and analyze the situation in real time at each traffic point, and even to accurately predict future traffic conditions.

[0249] "Means for acquiring video information in real time" refers to a device or function that continuously records road conditions and collects that video information without delay.

[0250] "Means of analyzing acquired video information to identify the status of vehicles and pedestrians" refers to the process of analyzing collected video and using image recognition technology to identify the number, speed, and direction of travel of traffic participants.

[0251] "Methods for recording and learning from past traffic information" refers to methods that utilize machine learning techniques to save past traffic data and use that data to strengthen predictive models.

[0252] "Methods for predicting future traffic conditions based on analytical information" refer to technologies that estimate future traffic flow at specific times and locations by referring to current and past traffic patterns.

[0253] "Means of optimizing traffic flow by operating traffic control devices" refers to methods of adjusting traffic signals and other control devices to improve the efficiency of vehicle flow on roads.

[0254] "Means of providing traffic information to users" refers to methods of sharing current traffic conditions and forecast information in a format that is easily accessible to the general public.

[0255] To implement this invention, a system is constructed using the following hardware and software. First, the terminal functions as a smart camera installed along the roadside, and this camera continuously acquires high-resolution video information. The video information is continuously acquired in real time and transmitted quickly to the server using compression technology.

[0256] The server analyzes the received video information using AI-based object detection algorithms, such as YOLO and Faster R-CNN. This allows it to identify the number of vehicles in the video, the presence of pedestrians, and the speed and direction of each object. Simultaneously, it learns this information by referencing previously recorded traffic data and using machine learning techniques.

[0257] Based on the data analyzed by this server, it can issue instructions to traffic control systems to adjust signal timing. This dynamically adjusts traffic flow and reduces congestion in real time. Furthermore, users can obtain current and predicted traffic information in an intuitive and easy-to-understand format through a dedicated application or web portal. Based on this information, users can select the optimal route and departure time.

[0258] To give a concrete example, consider the morning rush hour in a major city. This system learns past rush hour patterns and predicts traffic volume for a particular day. By dynamically adjusting traffic light timing based on the prediction, traffic flow becomes smoother, reducing commuting time and stress.

[0259] An example of a prompt to input into a generative AI model is, "Analyze traffic patterns in a specific area, generate a predictive model, and propose ways to use it for traffic signal control." This allows the generative AI to propose a variety of approaches to traffic optimization.

[0260] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0261] Step 1:

[0262] The terminal uses smart cameras installed on the street to acquire real-time video footage of the road. The input is high-resolution video data captured by the cameras. The terminal converts this video data into a digital format and reduces the data size using a compression algorithm. The output is the compressed video data, which is sent to a server for subsequent analysis processing.

[0263] Step 2:

[0264] The server decompresses the compressed video data received from the terminal and detects vehicles and pedestrians within the video. The input is the decompressed video data. The server applies AI-based object detection algorithms (e.g., YOLO, Faster R-CNN) to identify the number, position, speed, and direction of travel of each object. The output is detailed traffic data based on the analysis results.

[0265] Step 3:

[0266] The server uses machine learning algorithms to predict future traffic conditions by referencing analysis results and historical traffic data. The input consists of current analysis data and stored historical traffic data. In this process, the server learns specific patterns and generates a model to predict future traffic flow. The output is data on predicted traffic patterns and congestion levels.

[0267] Step 4:

[0268] The server sends signal timing adjustment commands to the signal control system based on predictions. Here, the input is predicted traffic condition data, and the output is adjusted signal timing information. The server performs this in real time, and optimization is achieved through the traffic control system.

[0269] Step 5:

[0270] Users obtain the latest traffic and forecast information using a dedicated application or web portal. The input is the user's request (e.g., obtaining information for a specific route), and the server organizes and outputs the analysis results and forecast information. Based on this, users can optimize their travel routes and departure times.

[0271] (Application Example 1)

[0272] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0273] Urban traffic congestion is a major challenge, leading to increased travel times and environmental burdens. Furthermore, it is difficult for users to choose the optimal route and time, necessitating improvements in individual travel efficiency. Against this backdrop, there is a need to develop a system that utilizes real-time traffic information to provide effective route suggestions to users.

[0274] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0275] In this invention, the server includes means for acquiring visual data in real time, means for recording and learning past travel data, and means for analyzing the data to predict future travel conditions. As a result, users can receive suggestions for optimal travel routes and departure times, and real-time guidance on changes to their travel routes.

[0276] "Means of acquiring visual data in real time" refers to devices that use cameras and sensors installed on the streets to instantly collect data on current road conditions and traffic flow.

[0277] "Means of recording and learning from past movement data" refers to algorithms and systems that store previous traffic data, analyze it to identify specific patterns and trends, and predict future situations.

[0278] "A means of analyzing data to predict future travel patterns" refers to a function that uses AI technology to analyze traffic patterns based on collected data and show future traffic flows.

[0279] "Means of optimizing traffic flow by operating control devices" refers to technologies that smooth traffic flow and alleviate congestion by dynamically adjusting traffic signals and traffic control devices.

[0280] "Means of providing users with mobility information" refers to interfaces and applications that convey traffic conditions and forecast information to users via mobile devices or computers.

[0281] "A means of suggesting the optimal travel route and departure time based on the user's current location and destination" refers to a function that utilizes collected traffic data to calculate and present the most efficient route and appropriate departure time for the user.

[0282] A "means of providing real-time route change guidance" refers to a system that, in response to changes in traffic conditions, proposes new routes to users, supporting efficient travel to their destinations.

[0283] To implement this invention, mainly the terminal, server, and user each play their respective roles. The terminal is a visual data acquisition device installed on the street, which acquires traffic flow and situations in real time as video data. These visual data are immediately transmitted to the server, which is a data management device. The server utilizes AI technology to analyze the data and learn past traffic patterns. AI-based object detection algorithms such as TensorFlow are used for this analysis and prediction processing.

[0284] The server also optimizes the travel route and departure time for the user. Specifically, it calculates and proposes the optimal route and departure time based on the user's current location and destination via a smartphone application. Also, based on real-time traffic data, if there are changes in the traffic situation during travel, it guides the user to change the route. This enables the user to avoid traffic congestion and arrive at the destination more efficiently.

[0285] As hardware, a group of cameras for acquiring video data and smartphones are mainly used. For data processing and analysis, a database system built on cloud services such as AWS or Google Cloud is utilized.

[0286] As a specific example, consider the morning commuting time. When the user starts moving from home to the office, by using this application to check the optimal route and departure time, they can avoid congestion and quickly reach the destination. Furthermore, with real-time route re-proposal according to the changing situation during travel, the user's transportation efficiency is improved.

[0287] An example of a prompt sentence for the generative AI model is "Investigate today's traffic conditions and tell me the route that will allow me to reach my destination the fastest." This prompt is utilized to generate the user's destination and maximally efficient route information.

[0288] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0289] Step 1:

[0290] The terminal uses cameras installed on the street to acquire real-time visual data of traffic conditions. The input is live video of the traffic scene, and this data is compressed and sent to the server as output. The data processing performed at this stage is the compression of the video stream.

[0291] Step 2:

[0292] The server receives visual data transmitted from the terminal and analyzes the data using an AI-based object detection algorithm. The input is compressed traffic video data, and the output is the analysis results, such as the number of vehicles, speed, and direction of travel. This analysis includes data computation using TensorFlow and other tools.

[0293] Step 3:

[0294] The server utilizes historical travel data to learn traffic patterns based on the analysis results. The input consists of current analysis results and historical traffic data. The output generates predictive information about future traffic conditions. This data processing includes pattern recognition and predictive model construction processes.

[0295] Step 4:

[0296] The user enters their current location and destination via a smartphone application. The application sends a request to a server, which retrieves the optimal travel route and departure time. The input consists of the user's current location, destination, and traffic forecast data, while the output is recommended route information.

[0297] Step 5:

[0298] The server sends information to the application to update the user's route as they travel, based on real-time traffic conditions. The input is the latest traffic data and the user's progress, and the output is a suggested updated route. This allows the user to be presented with the optimal route based on prompts generated using a generative AI model.

[0299] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0300] This invention combines a system that analyzes traffic conditions in real time and predicts future traffic with an emotion engine that recognizes user emotions. This system streamlines traffic management and enables the provision of information tailored to the user.

[0301] First, the terminal (smart camera) continuously acquires video data of the road, compresses it, and sends it to the server. The server uses AI to analyze the video data and records the current traffic situation as numerical data. Furthermore, it learns patterns from the accumulated data and predicts future traffic conditions.

[0302] This system includes an emotion engine to recognize user emotions. Users provide emotional data to the system through a dedicated device or application. The emotion engine analyzes this data to detect the user's stress level and other emotions. The detection results are used to adjust the traffic information provided; for example, users with high stress levels can be provided with more relaxed route information.

[0303] The emotion engine is also incorporated into the traffic signal control process. For example, if driver stress levels are generally high in a particular area, the server uses that information to adjust the signal patterns, making traffic flow smoother and thus reducing stress.

[0304] As a specific example, consider the case where this system operates during an event in a certain city. In addition to normal traffic data, the system conducts traffic guidance based on the emotional data of participants. As a result, it is possible to construct an optimal traffic environment considering user comfort, such as adjusting signal timings at points where traffic jams are predicted and providing messages with a relaxation effect through voice guidance on some routes.

[0305] The following describes the processing flow.

[0306] Step 1:

[0307] The terminal acquires real-time video data of the road. The camera attaches a timestamp to each frame and transmits the captured data to the server at regular intervals.

[0308] Step 2:

[0309] The terminal compresses the acquired video data and transmits it to the server in an optimized format for efficient data transfer.

[0310] Step 3:

[0311] The server analyzes the received video data. Using an AI-based object detection algorithm, it identifies vehicles and pedestrians to grasp the traffic situation.

[0312] Step 4:

[0313] The server accumulates the analysis results in the database. From this data, it learns past and current traffic trends and updates the model for predicting future traffic situations.

[0314] Step 5:

[0315] Users use a dedicated application to send emotional data from their device to a server. This emotional data includes information indicating the user's stress level and current emotional state.

[0316] Step 6:

[0317] The emotion engine on the server analyzes the emotion data sent by the user to identify stress levels and other emotional states.

[0318] Step 7:

[0319] The server adjusts the content of traffic information provided based on the user's emotional state. For users experiencing high stress levels, it suggests relaxing routes and information.

[0320] Step 8:

[0321] The server aggregates user sentiment data from across the region and uses this information to optimize traffic signal control parameters. For example, if stress levels are high in a specific area, the signal timing is adjusted to optimize traffic flow.

[0322] Step 9:

[0323] Users plan their travel based on the provided traffic information. They can use the application to select the most comfortable route and time.

[0324] (Example 2)

[0325] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0326] Current traffic management systems have limitations in real-time traffic information analysis and signal control, and in particular, they have not been able to provide optimal traffic information based on the emotional state of individual users. Therefore, there is a need to improve traffic flow while also reducing the psychological stress on users.

[0327] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0328] In this invention, the server includes means for acquiring video data in real time, means for compressing the acquired video data and transmitting it to the server, means for recording and learning past traffic data, means for predicting future traffic conditions based on the recorded data, means for collecting and analyzing user emotion data, means for providing traffic information according to the user's emotional state, and means for dynamically optimizing traffic signal control based on the user's emotions and traffic conditions. This makes it possible to manage traffic conditions more precisely and realize a comfortable traffic environment by providing information adapted to the user.

[0329] "Real-time" refers to a state where data acquisition and processing occur instantly, without any delay.

[0330] "Video data" refers to a collection of visual information acquired by a camera or other recording device.

[0331] "Compression" is a technique that reduces the amount of data, making it easier to transmit and store that data.

[0332] A "server" is a computer system that provides services to other devices over a network.

[0333] "Traffic data" refers to information about the volume, speed, and flow of vehicles on roads.

[0334] "Learning" is the process by which an algorithm discovers patterns and features from data and utilizes that information.

[0335] "Predicting future traffic conditions" means estimating future traffic flow and congestion based on past and present traffic data.

[0336] "User emotional data" refers to information that indicates a user's psychological state and stress level.

[0337] "Analysis" is the process of processing and examining data to derive meaning and trends from it.

[0338] "Providing traffic information" means informing users about road conditions and recommended routes.

[0339] "Traffic signal control" is the process of adjusting the flow of vehicles on a road by operating traffic signals.

[0340] "Dynamic optimization" means adjusting to the most suitable state in real time according to the situation.

[0341] This invention combines a system that analyzes traffic conditions in real time and predicts future traffic with an emotion engine that recognizes user emotions. This system streamlines traffic management and enables the provision of information tailored to the user.

[0342] First, a smart camera, acting as a terminal, continuously acquires video data of the road, compresses this data, and sends it to a server. The server analyzes the video data using AI technology. Specifically, it utilizes computer vision technology to quantify the number, speed, and direction of vehicles on the road. The analyzed data is stored in a database and recorded as historical traffic data. Then, a machine learning model is used to learn patterns from this data and predict future traffic conditions.

[0343] Furthermore, users provide emotional data to the system through a dedicated device or application. This data is analyzed by an emotion engine to detect the user's stress level and other emotional states. Based on these detection results, personalized traffic information is provided to the user. Specifically, for users with high stress levels, it is possible to suggest calmer routes or provide relaxation messages via voice.

[0344] Furthermore, the system can also incorporate emotional data into the traffic signal control process. If a high level of stress is detected overall in a particular area, the server will adjust the signal patterns based on that information to smooth the overall traffic flow. This helps reduce stress throughout the area.

[0345] For example, when a large-scale event is held in a city, this system analyzes both normal traffic data and the emotional data of event participants to provide optimal traffic guidance. It can adjust traffic light timing at predicted congestion points and enhance user comfort by providing relaxing messages through voice guidance on specific routes.

[0346] An example of a prompt for a generative AI model is, "Based on road congestion and participant sentiment data, please suggest the optimal traffic route and signal control method for the day of the event." This prompt allows the AI ​​to comprehensively analyze diverse information and support decision-making.

[0347] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0348] Step 1:

[0349] The terminal uses smart cameras installed on the road to acquire video data in real time. The input is video of the road, and the output is compressed video data. Compressing this video data reduces network load and enables rapid data transfer. Specifically, frames are captured continuously, and compression is applied at regular intervals.

[0350] Step 2:

[0351] The server receives compressed video data transmitted from the terminal and analyzes it using AI technology. The input is compressed video data, and the output is analyzed numerical data (number of vehicles, speed, direction, etc.). Specifically, computer vision algorithms are applied to quantify the traffic flow on the road.

[0352] Step 3:

[0353] The server stores the analyzed traffic data in a database and trains a machine learning model using historical data. The input is digitized traffic data, and the output is a predictive model. Specifically, it applies machine learning algorithms to learn traffic patterns and generates a model that predicts future traffic conditions.

[0354] Step 4:

[0355] Users input emotional data through a dedicated device or application and send it to a server. The input is emotional information obtained from the user's voice and face, and the output is analyzed emotional data. Specifically, emotion recognition software is used to process the data and determine the user's stress level.

[0356] Step 5:

[0357] The server analyzes the user's emotional data and compares it with current traffic conditions to provide personalized traffic information. The input is the analyzed emotional data and traffic data, and the output is personalized traffic guidance information. In actual processing, the system generates the optimal route and relaxation message for the user through the combination of data.

[0358] Step 6:

[0359] The server utilizes user sentiment data to dynamically optimize traffic signal timing. Inputs are traffic data and sentiment data, and output is an adjusted signal control pattern. Specifically, at a given intersection, it adjusts signal switching times considering the overall user state to improve traffic flow.

[0360] (Application Example 2)

[0361] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0362] Conventional traffic management systems have limitations in efficiently analyzing traffic conditions and do not provide information that takes into account the emotional state of users, resulting in problems with ensuring sufficient user comfort and safety. In particular, with autonomous vehicles, it was difficult to reduce stress because driving parameters could not be adjusted based on passenger emotions.

[0363] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0364] In this invention, the server includes means for acquiring and analyzing video data in real time, means for recording and learning from past traffic data, means for analyzing the user's emotional state and adjusting the content of traffic information provided, and means for automatically adjusting the vehicle's driving parameters based on the passenger's emotions. This improves the accuracy of traffic situation prediction and enables driving and information provision that takes into account the user's emotions.

[0365] A "device that acquires video data in real time" is a device that continuously acquires video data using sensors such as cameras in order to instantly grasp traffic conditions, and provides it in an analyzable format.

[0366] A "device for recording and learning from past traffic data" is a device that accumulates historical data on traffic flow and vehicle movement over a long period of time, generates identification patterns based on this data, and creates a traffic prediction model.

[0367] A "device that analyzes data to predict future traffic conditions" is a device that takes current and past traffic data as input and predicts future traffic conditions and flow rates through analysis of that data.

[0368] A "device that optimizes traffic flow by operating signal control devices" is a device that automatically adjusts the timing and sequence of traffic signals to efficiently manage traffic flow, thereby promoting the reduction of congestion and traffic accidents.

[0369] A "device that analyzes the user's emotional state and adjusts the content of traffic information provided" is a device that analyzes emotional data acquired from the user's device and, based on that analysis, provides traffic information that is adapted to the user's stress level and emotions.

[0370] A "device that automatically adjusts vehicle driving parameters based on passenger emotions" is a device that enhances passenger comfort and safety by dynamically changing the driving characteristics of an autonomous vehicle, such as speed and route, based on the passenger's emotional response.

[0371] In order to implement this invention, the interaction between the server, terminal, and user is of primary importance.

[0372] The server plays a central role in traffic situation analysis. First, it receives video data transmitted in real time from terminals and analyzes traffic conditions using AI models. Machine learning models built with TensorFlow and PyTorch are used for the analysis. Based on the results, it predicts future traffic conditions, issues appropriate instructions to traffic signal control devices, and optimizes traffic flow.

[0373] Simultaneously, the server analyzes the user's emotional data. Emotional data is collected from the user's device (smartphone or wearable device) and processed using an emotion recognition model. Based on the analysis, information and driving instructions tailored to the user's stress level are provided. For example, if high stress is detected, the vehicle's driving parameters are adjusted, and a stress reduction mode is activated. This provides a more relaxed driving experience.

[0374] As a concrete example, in complex urban traffic situations, the user's emotions are analyzed in real time, and a smooth and safe route is suggested. Based on this, the server can automatically adjust the vehicle's speed and route, and can even stream relaxing music.

[0375] As an example of a prompt, the AI ​​model can be instructed with a message such as, "Based on the current emotional state and traffic data, please suggest the safest and most efficient route," thereby achieving user-adaptive output.

[0376] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0377] Step 1:

[0378] The terminal acquires road video data in real time. The acquired video data is resized and compressed for efficient handling and sent to the server. This process allows the terminal to provide data in a format that is easy for the server to handle.

[0379] Step 2:

[0380] The server receives compressed video data transmitted from the terminal and decompresses it. An AI model is used to analyze this data and extract current traffic information. Specifically, image recognition technology is used to count the number of vehicles and pedestrians, generating traffic flow data. This data then becomes the input for the next prediction step.

[0381] Step 3:

[0382] The server receives previously accumulated traffic data and current traffic data as input and uses a generating AI model to predict future traffic conditions. The model performs time-series analysis and outputs predicted traffic congestion points and time periods. The resulting future prediction data is used to adjust traffic signal control.

[0383] Step 4:

[0384] Users input emotional data through a dedicated device or smartphone. This emotional data is analyzed in real time by AI, and the user's stress level and mental state are quantified. The emotional state obtained from the analysis is used to adjust the traffic information provided by the server.

[0385] Step 5:

[0386] The server integrates analyzed traffic and sentiment data to instruct signal control units on the optimal signal timing. It also provides users with optimized route suggestions and driving advice. As a result, users can experience safer and more comfortable travel.

[0387] Step 6:

[0388] To influence the user, the server streams relaxation-enhancing content (music and messages) within the autonomous vehicle. This content, tailored to the user's emotional state, directly enhances the driving experience.

[0389] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0390] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0391] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0392] [Third Embodiment]

[0393] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0394] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0395] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0396] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0397] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0398] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0399] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0400] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0401] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0402] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0403] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0404] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0405] This invention is a system for understanding traffic conditions in real time and predicting future traffic conditions. This system operates through the following process to automate and streamline traffic management.

[0406] First, terminals (smart cameras installed on the streets) continuously acquire video data of the road and transmit this data to a server in real time. The server analyzes the received video data and automatically identifies the number, speed, and direction of movement of vehicles and pedestrians. Here, an AI-based object detection algorithm is used to achieve highly accurate analysis.

[0407] The analysis results are recorded in a digital database as current traffic conditions. The server uses this data to learn past traffic patterns. This makes it possible to make more accurate traffic predictions for specific dates and conditions.

[0408] Through an interface to the signal control system, the server sends instructions to the traffic lights based on real-time analysis results. As a result, traffic flow is optimized for smoother operation, and congestion is reduced as much as possible.

[0409] Users can obtain real-time traffic information and forecasts through a dedicated application or web portal. The interface for this information is designed to be intuitively understandable to users. Users can use this information to select the optimal travel route and departure time, resulting in a more comfortable and efficient journey.

[0410] As a concrete example, consider the weekday morning rush hour in a certain city. This system learns rush hour patterns from past data and predicts the traffic volume for the day. Based on this prediction, traffic signal timing is dynamically adjusted to manage traffic flow smoothly. This has the effect of shortening commute times and reducing stress caused by peak traffic.

[0411] The following describes the processing flow.

[0412] Step 1:

[0413] The device (smart camera) acquires video data from its installed location. The camera continuously captures images of the road and adds a timestamp to them.

[0414] Step 2:

[0415] The terminal compresses the acquired video data and efficiently transmits it to the server. An appropriate compression algorithm is used to optimize communication bandwidth.

[0416] Step 3:

[0417] The server receives the transmitted video data and inputs it into the data analysis pipeline. At this point, the video data is input into the real-time object detection model.

[0418] Step 4:

[0419] The server uses an AI-based object detection algorithm to identify vehicles and pedestrians. This allows for the analysis of traffic flow characteristics (number of vehicles, speed, direction of travel).

[0420] Step 5:

[0421] The server stores traffic information generated from raw data in a database. This information is used in subsequent learning processes and real-time monitoring.

[0422] Step 6:

[0423] The server analyzes recorded traffic data and uses a predictive model to estimate future traffic conditions. The prediction results are generated considering traffic patterns for specific days of the week and time periods.

[0424] Step 7:

[0425] The server uses the prediction results to send instructions to the traffic signal control system. The timing of the traffic lights is dynamically adjusted according to the traffic conditions at that time.

[0426] Step 8:

[0427] Users receive current and forecast traffic information through a dedicated application or web portal, enabling them to plan their travel efficiently.

[0428] (Example 1)

[0429] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0430] In modern metropolises, traffic congestion has a significant impact on economic activity and quality of life. However, current traffic management systems have limitations in real-time understanding and prediction of traffic conditions, preventing the implementation of effective congestion mitigation measures. To solve this problem, it is necessary to analyze traffic conditions in real time and realize efficient signal control and the provision of traffic information.

[0431] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0432] In this invention, the server includes means for acquiring video information in real time, means for analyzing the acquired video information to identify the state of vehicles and pedestrians, and means for recording and learning past traffic information. This makes it possible to grasp and analyze the situation in real time at each traffic point, and even to accurately predict future traffic conditions.

[0433] "Means for acquiring video information in real time" refers to a device or function that continuously records road conditions and collects that video information without delay.

[0434] "Means of analyzing acquired video information to identify the status of vehicles and pedestrians" refers to the process of analyzing collected video and using image recognition technology to identify the number, speed, and direction of travel of traffic participants.

[0435] "Methods for recording and learning from past traffic information" refers to methods that utilize machine learning techniques to save past traffic data and use that data to strengthen predictive models.

[0436] "Methods for predicting future traffic conditions based on analytical information" refer to technologies that estimate future traffic flow at specific times and locations by referring to current and past traffic patterns.

[0437] "Means of optimizing traffic flow by operating traffic control devices" refers to methods of adjusting traffic signals and other control devices to improve the efficiency of vehicle flow on roads.

[0438] "Means of providing traffic information to users" refers to methods of sharing current traffic conditions and forecast information in a format that is easily accessible to the general public.

[0439] To implement this invention, a system is constructed using the following hardware and software. First, the terminal functions as a smart camera installed along the roadside, and this camera continuously acquires high-resolution video information. The video information is continuously acquired in real time and transmitted quickly to the server using compression technology.

[0440] The server analyzes the received video information using AI-based object detection algorithms, such as YOLO and Faster R-CNN. This allows it to identify the number of vehicles in the video, the presence of pedestrians, and the speed and direction of each object. Simultaneously, it learns this information by referencing previously recorded traffic data and using machine learning techniques.

[0441] Based on the data analyzed by this server, it can issue instructions to traffic control systems to adjust signal timing. This dynamically adjusts traffic flow and reduces congestion in real time. Furthermore, users can obtain current and predicted traffic information in an intuitive and easy-to-understand format through a dedicated application or web portal. Based on this information, users can select the optimal route and departure time.

[0442] To give a concrete example, consider the morning rush hour in a major city. This system learns past rush hour patterns and predicts traffic volume for a particular day. By dynamically adjusting traffic light timing based on the prediction, traffic flow becomes smoother, reducing commuting time and stress.

[0443] An example of a prompt to input into a generative AI model is, "Analyze traffic patterns in a specific area, generate a predictive model, and propose ways to use it for traffic signal control." This allows the generative AI to propose a variety of approaches to traffic optimization.

[0444] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0445] Step 1:

[0446] The terminal uses smart cameras installed on the street to acquire real-time video footage of the road. The input is high-resolution video data captured by the cameras. The terminal converts this video data into a digital format and reduces the data size using a compression algorithm. The output is the compressed video data, which is sent to a server for subsequent analysis processing.

[0447] Step 2:

[0448] The server decompresses the compressed video data received from the terminal and detects vehicles and pedestrians within the video. The input is the decompressed video data. The server applies AI-based object detection algorithms (e.g., YOLO, Faster R-CNN) to identify the number, position, speed, and direction of travel of each object. The output is detailed traffic data based on the analysis results.

[0449] Step 3:

[0450] The server uses machine learning algorithms to predict future traffic conditions by referencing analysis results and historical traffic data. The input consists of current analysis data and stored historical traffic data. In this process, the server learns specific patterns and generates a model to predict future traffic flow. The output is data on predicted traffic patterns and congestion levels.

[0451] Step 4:

[0452] The server sends signal timing adjustment commands to the signal control system based on predictions. Here, the input is predicted traffic condition data, and the output is adjusted signal timing information. The server performs this in real time, and optimization is achieved through the traffic control system.

[0453] Step 5:

[0454] Users obtain the latest traffic and forecast information using a dedicated application or web portal. The input is the user's request (e.g., obtaining information for a specific route), and the server organizes and outputs the analysis results and forecast information. Based on this, users can optimize their travel routes and departure times.

[0455] (Application Example 1)

[0456] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0457] Urban traffic congestion is a major challenge, leading to increased travel times and environmental burdens. Furthermore, it is difficult for users to choose the optimal route and time, necessitating improvements in individual travel efficiency. Against this backdrop, there is a need to develop a system that utilizes real-time traffic information to provide effective route suggestions to users.

[0458] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0459] In this invention, the server includes means for acquiring visual data in real time, means for recording and learning past travel data, and means for analyzing the data to predict future travel conditions. As a result, users can receive suggestions for optimal travel routes and departure times, and real-time guidance on changes to their travel routes.

[0460] "Means of acquiring visual data in real time" refers to devices that use cameras and sensors installed on the streets to instantly collect data on current road conditions and traffic flow.

[0461] "Means of recording and learning from past movement data" refers to algorithms and systems that store previous traffic data, analyze it to identify specific patterns and trends, and predict future situations.

[0462] "A means of analyzing data to predict future travel patterns" refers to a function that uses AI technology to analyze traffic patterns based on collected data and show future traffic flows.

[0463] "Means of optimizing traffic flow by operating control devices" refers to technologies that smooth traffic flow and alleviate congestion by dynamically adjusting traffic signals and traffic control devices.

[0464] "Means of providing users with mobility information" refers to interfaces and applications that convey traffic conditions and forecast information to users via mobile devices or computers.

[0465] "A means of suggesting the optimal travel route and departure time based on the user's current location and destination" refers to a function that utilizes collected traffic data to calculate and present the most efficient route and appropriate departure time for the user.

[0466] A "means of providing real-time route change guidance" refers to a system that, in response to changes in traffic conditions, proposes new routes to users, supporting efficient travel to their destinations.

[0467] To implement this invention, terminals, servers, and users primarily play their respective roles. The terminal is a visual data acquisition device installed on the street, which acquires real-time traffic flow and conditions as video data. This visual data is immediately transmitted to the server, which is a data management device. The server uses AI technology to analyze the data and learn past traffic patterns. AI-based object detection algorithms such as TensorFlow are used for this analysis and prediction process.

[0468] The server also optimizes travel routes and departure times for users. Specifically, it calculates and suggests the optimal route and departure time based on the user's current location and destination via a smartphone application. Furthermore, based on real-time traffic data, it notifies users of route changes if traffic conditions change during their journey. This allows users to avoid traffic congestion and arrive at their destination more efficiently.

[0469] The hardware primarily consists of a group of cameras to acquire video data and smartphones. Database systems built on cloud services such as AWS and Google Cloud are used for data processing and analysis.

[0470] As a concrete example, consider the morning commute. When a user starts their journey from home to the office, they can use this app to check the optimal route and departure time, allowing them to avoid traffic and reach their destination quickly. Furthermore, real-time route suggestions that adapt to changing circumstances during the journey improve the user's travel efficiency.

[0471] An example of a prompt to a generative AI model is, "Check today's traffic conditions and tell me the fastest route to my destination." This prompt is used to generate information about the user's destination and the most efficient route possible.

[0472] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0473] Step 1:

[0474] The terminal uses cameras installed on the street to acquire real-time visual data of traffic conditions. The input is live video of the traffic scene, and this data is compressed and sent to the server as output. The data processing performed at this stage is the compression of the video stream.

[0475] Step 2:

[0476] The server receives visual data transmitted from the terminal and analyzes the data using an AI-based object detection algorithm. The input is compressed traffic video data, and the output is the analysis results, such as the number of vehicles, speed, and direction of travel. This analysis includes data computation using TensorFlow and other tools.

[0477] Step 3:

[0478] The server utilizes historical travel data to learn traffic patterns based on the analysis results. The input consists of current analysis results and historical traffic data. The output generates predictive information about future traffic conditions. This data processing includes pattern recognition and predictive model construction processes.

[0479] Step 4:

[0480] The user enters their current location and destination via a smartphone application. The application sends a request to a server, which retrieves the optimal travel route and departure time. The input consists of the user's current location, destination, and traffic forecast data, while the output is recommended route information.

[0481] Step 5:

[0482] The server sends information to the application to update the user's route as they travel, based on real-time traffic conditions. The input is the latest traffic data and the user's progress, and the output is a suggested updated route. This allows the user to be presented with the optimal route based on prompts generated using a generative AI model.

[0483] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0484] This invention combines a system that analyzes traffic conditions in real time and predicts future traffic with an emotion engine that recognizes user emotions. This system streamlines traffic management and enables the provision of information tailored to the user.

[0485] First, the terminal (smart camera) continuously acquires video data of the road, compresses it, and sends it to the server. The server uses AI to analyze the video data and records the current traffic situation as numerical data. Furthermore, it learns patterns from the accumulated data and predicts future traffic conditions.

[0486] This system includes an emotion engine to recognize user emotions. Users provide emotional data to the system through a dedicated device or application. The emotion engine analyzes this data to detect the user's stress level and other emotions. The detection results are used to adjust the traffic information provided; for example, users with high stress levels can be provided with more relaxed route information.

[0487] The emotion engine is also incorporated into the traffic signal control process. For example, if driver stress levels are generally high in a particular area, the server uses that information to adjust the signal patterns, making traffic flow smoother and thus reducing stress.

[0488] As a concrete example, consider a scenario where this system is operational during an event in a certain city. The system uses not only normal traffic data but also participant emotion data to guide traffic. As a result, it can adjust traffic light timing at points where congestion is predicted and provide relaxing voice guidance on some routes, creating an optimal traffic environment that takes user comfort into consideration.

[0489] The following describes the processing flow.

[0490] Step 1:

[0491] The terminal acquires road video data in real time. The camera adds a timestamp to each frame and sends the captured data to the server at regular intervals.

[0492] Step 2:

[0493] The video data acquired by the terminal is compressed and sent to the server in a format optimized for efficient data transfer.

[0494] Step 3:

[0495] The server analyzes the received video data. Using AI-based object detection algorithms, it identifies vehicles and pedestrians to understand traffic conditions.

[0496] Step 4:

[0497] The server stores the analysis results in a database. From this data, it learns past and present traffic trends and updates the model to predict future traffic conditions.

[0498] Step 5:

[0499] Users use a dedicated application to send emotional data from their device to a server. This emotional data includes information indicating the user's stress level and current emotional state.

[0500] Step 6:

[0501] The emotion engine on the server analyzes the emotion data sent by the user to identify stress levels and other emotional states.

[0502] Step 7:

[0503] The server adjusts the content of traffic information provided based on the user's emotional state. For users experiencing high stress levels, it suggests relaxing routes and information.

[0504] Step 8:

[0505] The server aggregates user sentiment data from across the region and uses this information to optimize traffic signal control parameters. For example, if stress levels are high in a specific area, the signal timing is adjusted to optimize traffic flow.

[0506] Step 9:

[0507] Users plan their travel based on the provided traffic information. They can use the application to select the most comfortable route and time.

[0508] (Example 2)

[0509] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0510] Current traffic management systems have limitations in real-time traffic information analysis and signal control, and in particular, they have not been able to provide optimal traffic information based on the emotional state of individual users. Therefore, there is a need to improve traffic flow while also reducing the psychological stress on users.

[0511] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0512] In this invention, the server includes means for acquiring video data in real time, means for compressing the acquired video data and transmitting it to the server, means for recording and learning past traffic data, means for predicting future traffic conditions based on the recorded data, means for collecting and analyzing user emotion data, means for providing traffic information according to the user's emotional state, and means for dynamically optimizing traffic signal control based on the user's emotions and traffic conditions. This makes it possible to manage traffic conditions more precisely and realize a comfortable traffic environment by providing information adapted to the user.

[0513] "Real-time" refers to a state where data acquisition and processing occur instantly, without any delay.

[0514] "Video data" refers to a collection of visual information acquired by a camera or other recording device.

[0515] "Compression" is a technique that reduces the amount of data, making it easier to transmit and store that data.

[0516] A "server" is a computer system that provides services to other devices over a network.

[0517] "Traffic data" refers to information about the volume, speed, and flow of vehicles on roads.

[0518] "Learning" is the process by which an algorithm discovers patterns and features from data and utilizes that information.

[0519] "Predicting future traffic conditions" means estimating future traffic flow and congestion based on past and present traffic data.

[0520] "User emotional data" refers to information that indicates a user's psychological state and stress level.

[0521] "Analysis" is the process of processing and examining data to derive meaning and trends from it.

[0522] "Providing traffic information" means informing users about road conditions and recommended routes.

[0523] "Traffic signal control" is the process of adjusting the flow of vehicles on a road by operating traffic signals.

[0524] "Dynamic optimization" means adjusting to the most suitable state in real time according to the situation.

[0525] This invention combines a system that analyzes traffic conditions in real time and predicts future traffic with an emotion engine that recognizes user emotions. This system streamlines traffic management and enables the provision of information tailored to the user.

[0526] First, a smart camera, acting as a terminal, continuously acquires video data of the road, compresses this data, and sends it to a server. The server analyzes the video data using AI technology. Specifically, it utilizes computer vision technology to quantify the number, speed, and direction of vehicles on the road. The analyzed data is stored in a database and recorded as historical traffic data. Then, a machine learning model is used to learn patterns from this data and predict future traffic conditions.

[0527] Furthermore, users provide emotional data to the system through a dedicated device or application. This data is analyzed by an emotion engine to detect the user's stress level and other emotional states. Based on these detection results, personalized traffic information is provided to the user. Specifically, for users with high stress levels, it is possible to suggest calmer routes or provide relaxation messages via voice.

[0528] Furthermore, the system can also incorporate emotional data into the traffic signal control process. If a high level of stress is detected overall in a particular area, the server will adjust the signal patterns based on that information to smooth the overall traffic flow. This helps reduce stress throughout the area.

[0529] For example, when a large-scale event is held in a city, this system analyzes both normal traffic data and the emotional data of event participants to provide optimal traffic guidance. It can adjust traffic light timing at predicted congestion points and enhance user comfort by providing relaxing messages through voice guidance on specific routes.

[0530] An example of a prompt for a generative AI model is, "Based on road congestion and participant sentiment data, please suggest the optimal traffic route and signal control method for the day of the event." This prompt allows the AI ​​to comprehensively analyze diverse information and support decision-making.

[0531] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0532] Step 1:

[0533] The terminal uses smart cameras installed on the road to acquire video data in real time. The input is video of the road, and the output is compressed video data. Compressing this video data reduces network load and enables rapid data transfer. Specifically, frames are captured continuously, and compression is applied at regular intervals.

[0534] Step 2:

[0535] The server receives compressed video data transmitted from the terminal and analyzes it using AI technology. The input is compressed video data, and the output is analyzed numerical data (number of vehicles, speed, direction, etc.). Specifically, computer vision algorithms are applied to quantify the traffic flow on the road.

[0536] Step 3:

[0537] The server stores the analyzed traffic data in a database and trains a machine learning model using historical data. The input is digitized traffic data, and the output is a predictive model. Specifically, it applies machine learning algorithms to learn traffic patterns and generates a model that predicts future traffic conditions.

[0538] Step 4:

[0539] Users input emotional data through a dedicated device or application and send it to a server. The input is emotional information obtained from the user's voice and face, and the output is analyzed emotional data. Specifically, emotion recognition software is used to process the data and determine the user's stress level.

[0540] Step 5:

[0541] The server analyzes the user's emotional data and compares it with current traffic conditions to provide personalized traffic information. The input is the analyzed emotional data and traffic data, and the output is personalized traffic guidance information. In actual processing, the system generates the optimal route and relaxation message for the user through the combination of data.

[0542] Step 6:

[0543] The server utilizes user sentiment data to dynamically optimize traffic signal timing. Inputs are traffic data and sentiment data, and output is an adjusted signal control pattern. Specifically, at a given intersection, it adjusts signal switching times considering the overall user state to improve traffic flow.

[0544] (Application Example 2)

[0545] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0546] Conventional traffic management systems have limitations in efficiently analyzing traffic conditions and do not provide information that takes into account the emotional state of users, resulting in problems with ensuring sufficient user comfort and safety. In particular, with autonomous vehicles, it was difficult to reduce stress because driving parameters could not be adjusted based on passenger emotions.

[0547] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0548] In this invention, the server includes means for acquiring and analyzing video data in real time, means for recording and learning from past traffic data, means for analyzing the user's emotional state and adjusting the content of traffic information provided, and means for automatically adjusting the vehicle's driving parameters based on the passenger's emotions. This improves the accuracy of traffic situation prediction and enables driving and information provision that takes into account the user's emotions.

[0549] A "device that acquires video data in real time" is a device that continuously acquires video data using sensors such as cameras in order to instantly grasp traffic conditions, and provides it in an analyzable format.

[0550] A "device for recording and learning from past traffic data" is a device that accumulates historical data on traffic flow and vehicle movement over a long period of time, generates identification patterns based on this data, and creates a traffic prediction model.

[0551] A "device that analyzes data to predict future traffic conditions" is a device that takes current and past traffic data as input and predicts future traffic conditions and flow rates through analysis of that data.

[0552] A "device that optimizes traffic flow by operating signal control devices" is a device that automatically adjusts the timing and sequence of traffic signals to efficiently manage traffic flow, thereby promoting the reduction of congestion and traffic accidents.

[0553] A "device that analyzes the user's emotional state and adjusts the content of traffic information provided" is a device that analyzes emotional data acquired from the user's device and, based on that analysis, provides traffic information that is adapted to the user's stress level and emotions.

[0554] A "device that automatically adjusts vehicle driving parameters based on passenger emotions" is a device that enhances passenger comfort and safety by dynamically changing the driving characteristics of an autonomous vehicle, such as speed and route, based on the passenger's emotional response.

[0555] In order to implement this invention, the interaction between the server, terminal, and user is of primary importance.

[0556] The server plays a central role in traffic situation analysis. First, it receives video data transmitted in real time from terminals and analyzes traffic conditions using AI models. Machine learning models built with TensorFlow and PyTorch are used for the analysis. Based on the results, it predicts future traffic conditions, issues appropriate instructions to traffic signal control devices, and optimizes traffic flow.

[0557] Simultaneously, the server analyzes the user's emotional data. Emotional data is collected from the user's device (smartphone or wearable device) and processed using an emotion recognition model. Based on the analysis, information and driving instructions tailored to the user's stress level are provided. For example, if high stress is detected, the vehicle's driving parameters are adjusted, and a stress reduction mode is activated. This provides a more relaxed driving experience.

[0558] As a concrete example, in complex urban traffic situations, the user's emotions are analyzed in real time, and a smooth and safe route is suggested. Based on this, the server can automatically adjust the vehicle's speed and route, and can even stream relaxing music.

[0559] As an example of a prompt, the AI ​​model can be instructed to "Suggest the safest and most efficient route based on the current emotional state and traffic data," thereby achieving user-adaptive output.

[0560] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0561] Step 1:

[0562] The terminal acquires road video data in real time. The acquired video data is resized and compressed for efficient handling and sent to the server. This process allows the terminal to provide data in a format that is easy for the server to handle.

[0563] Step 2:

[0564] The server receives compressed video data transmitted from the terminal and decompresses it. An AI model is used to analyze this data and extract current traffic information. Specifically, image recognition technology is used to count the number of vehicles and pedestrians, generating traffic flow data. This data then becomes the input for the next prediction step.

[0565] Step 3:

[0566] The server receives previously accumulated traffic data and current traffic data as input and uses a generating AI model to predict future traffic conditions. The model performs time-series analysis and outputs predicted traffic congestion points and time periods. The resulting future prediction data is used to adjust traffic signal control.

[0567] Step 4:

[0568] Users input emotional data through a dedicated device or smartphone. This emotional data is analyzed in real time by AI, and the user's stress level and mental state are quantified. The emotional state obtained from the analysis is used to adjust the traffic information provided by the server.

[0569] Step 5:

[0570] The server integrates analyzed traffic and sentiment data to instruct signal control units on the optimal signal timing. It also provides users with optimized route suggestions and driving advice. As a result, users can experience safer and more comfortable travel.

[0571] Step 6:

[0572] To influence the user, the server streams relaxation-enhancing content (music and messages) within the autonomous vehicle. This content, tailored to the user's emotional state, directly enhances the driving experience.

[0573] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0574] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0575] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0576] [Fourth Embodiment]

[0577] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0578] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0579] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0580] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0581] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0582] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0583] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0584] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0585] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0586] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0587] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0588] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0589] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0590] This invention is a system for understanding traffic conditions in real time and predicting future traffic conditions. This system operates through the following process to automate and streamline traffic management.

[0591] First, terminals (smart cameras installed on the streets) continuously acquire video data of the road and transmit this data to a server in real time. The server analyzes the received video data and automatically identifies the number, speed, and direction of movement of vehicles and pedestrians. Here, an AI-based object detection algorithm is used to achieve highly accurate analysis.

[0592] The analysis results are recorded in a digital database as current traffic conditions. The server uses this data to learn past traffic patterns. This makes it possible to make more accurate traffic predictions for specific dates and conditions.

[0593] Through an interface to the signal control system, the server sends instructions to the traffic lights based on real-time analysis results. As a result, traffic flow is optimized for smoother operation, and congestion is reduced as much as possible.

[0594] Users can obtain real-time traffic information and forecasts through a dedicated application or web portal. The interface for this information is designed to be intuitively understandable to users. Users can use this information to select the optimal travel route and departure time, resulting in a more comfortable and efficient journey.

[0595] As a concrete example, consider the weekday morning rush hour in a certain city. This system learns rush hour patterns from past data and predicts the traffic volume for the day. Based on this prediction, traffic signal timing is dynamically adjusted to manage traffic flow smoothly. This has the effect of shortening commute times and reducing stress caused by peak traffic.

[0596] The following describes the processing flow.

[0597] Step 1:

[0598] The device (smart camera) acquires video data from its installed location. The camera continuously captures images of the road and adds a timestamp to them.

[0599] Step 2:

[0600] The terminal compresses the acquired video data and efficiently transmits it to the server. An appropriate compression algorithm is used to optimize communication bandwidth.

[0601] Step 3:

[0602] The server receives the transmitted video data and inputs it into the data analysis pipeline. At this point, the video data is input into the real-time object detection model.

[0603] Step 4:

[0604] The server uses an AI-based object detection algorithm to identify vehicles and pedestrians. This allows for the analysis of traffic flow characteristics (number of vehicles, speed, direction of travel).

[0605] Step 5:

[0606] The server stores traffic information generated from raw data in a database. This information is used in subsequent learning processes and real-time monitoring.

[0607] Step 6:

[0608] The server analyzes recorded traffic data and uses a predictive model to estimate future traffic conditions. The prediction results are generated considering traffic patterns for specific days of the week and time periods.

[0609] Step 7:

[0610] The server uses the prediction results to send instructions to the traffic signal control system. The timing of the traffic lights is dynamically adjusted according to the traffic conditions at that time.

[0611] Step 8:

[0612] Users receive current and forecast traffic information through a dedicated application or web portal, enabling them to plan their travel efficiently.

[0613] (Example 1)

[0614] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0615] In modern metropolises, traffic congestion has a significant impact on economic activity and quality of life. However, current traffic management systems have limitations in real-time understanding and prediction of traffic conditions, preventing the implementation of effective congestion mitigation measures. To solve this problem, it is necessary to analyze traffic conditions in real time and realize efficient signal control and the provision of traffic information.

[0616] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0617] In this invention, the server includes means for acquiring video information in real time, means for analyzing the acquired video information to identify the state of vehicles and pedestrians, and means for recording and learning past traffic information. This makes it possible to grasp and analyze the situation in real time at each traffic point, and even to accurately predict future traffic conditions.

[0618] "Means for acquiring video information in real time" refers to a device or function that continuously records road conditions and collects that video information without delay.

[0619] "Means of analyzing acquired video information to identify the status of vehicles and pedestrians" refers to the process of analyzing collected video and using image recognition technology to identify the number, speed, and direction of travel of traffic participants.

[0620] "Methods for recording and learning from past traffic information" refers to methods that utilize machine learning techniques to save past traffic data and use that data to strengthen predictive models.

[0621] "Methods for predicting future traffic conditions based on analytical information" refer to technologies that estimate future traffic flow at specific times and locations by referring to current and past traffic patterns.

[0622] "Means of optimizing traffic flow by operating traffic control devices" refers to methods of adjusting traffic signals and other control devices to improve the efficiency of vehicle flow on roads.

[0623] "Means of providing traffic information to users" refers to methods of sharing current traffic conditions and forecast information in a format that is easily accessible to the general public.

[0624] To implement this invention, a system is constructed using the following hardware and software. First, the terminal functions as a smart camera installed along the roadside, and this camera continuously acquires high-resolution video information. The video information is continuously acquired in real time and transmitted quickly to the server using compression technology.

[0625] The server analyzes the received video information using AI-based object detection algorithms, such as YOLO and Faster R-CNN. This allows it to identify the number of vehicles in the video, the presence of pedestrians, and the speed and direction of each object. Simultaneously, it learns this information by referencing previously recorded traffic data and using machine learning techniques.

[0626] Based on the data analyzed by this server, it can issue instructions to traffic control systems to adjust signal timing. This dynamically adjusts traffic flow and reduces congestion in real time. Furthermore, users can obtain current and predicted traffic information in an intuitive and easy-to-understand format through a dedicated application or web portal. Based on this information, users can select the optimal route and departure time.

[0627] To give a concrete example, consider the morning rush hour in a major city. This system learns past rush hour patterns and predicts traffic volume for a particular day. By dynamically adjusting traffic light timing based on the prediction, traffic flow becomes smoother, reducing commuting time and stress.

[0628] An example of a prompt to input into a generative AI model is, "Analyze traffic patterns in a specific area, generate a predictive model, and propose ways to use it for traffic signal control." This allows the generative AI to propose a variety of approaches to traffic optimization.

[0629] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0630] Step 1:

[0631] The terminal uses smart cameras installed on the street to acquire real-time video footage of the road. The input is high-resolution video data captured by the cameras. The terminal converts this video data into a digital format and reduces the data size using a compression algorithm. The output is the compressed video data, which is sent to a server for subsequent analysis processing.

[0632] Step 2:

[0633] The server decompresses the compressed video data received from the terminal and detects vehicles and pedestrians within the video. The input is the decompressed video data. The server applies AI-based object detection algorithms (e.g., YOLO, Faster R-CNN) to identify the number, position, speed, and direction of travel of each object. The output is detailed traffic data based on the analysis results.

[0634] Step 3:

[0635] The server uses machine learning algorithms to predict future traffic conditions by referencing analysis results and historical traffic data. The input consists of current analysis data and stored historical traffic data. In this process, the server learns specific patterns and generates a model to predict future traffic flow. The output is data on predicted traffic patterns and congestion levels.

[0636] Step 4:

[0637] The server sends signal timing adjustment commands to the signal control system based on predictions. Here, the input is predicted traffic condition data, and the output is adjusted signal timing information. The server performs this in real time, and optimization is achieved through the traffic control system.

[0638] Step 5:

[0639] Users obtain the latest traffic and forecast information using a dedicated application or web portal. The input is the user's request (e.g., obtaining information for a specific route), and the server organizes and outputs the analysis results and forecast information. Based on this, users can optimize their travel routes and departure times.

[0640] (Application Example 1)

[0641] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0642] Urban traffic congestion is a major challenge, leading to increased travel times and environmental burdens. Furthermore, it is difficult for users to choose the optimal route and time, necessitating improvements in individual travel efficiency. Against this backdrop, there is a need to develop a system that utilizes real-time traffic information to provide effective route suggestions to users.

[0643] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0644] In this invention, the server includes means for acquiring visual data in real time, means for recording and learning past travel data, and means for analyzing the data to predict future travel conditions. As a result, users can receive suggestions for optimal travel routes and departure times, and real-time guidance on changes to their travel routes.

[0645] "Means of acquiring visual data in real time" refers to devices that use cameras and sensors installed on the streets to instantly collect data on current road conditions and traffic flow.

[0646] "Means of recording and learning from past movement data" refers to algorithms and systems that store previous traffic data, analyze it to identify specific patterns and trends, and predict future situations.

[0647] "A means of analyzing data to predict future travel patterns" refers to a function that uses AI technology to analyze traffic patterns based on collected data and show future traffic flows.

[0648] "Means of optimizing traffic flow by operating control devices" refers to technologies that smooth traffic flow and alleviate congestion by dynamically adjusting traffic signals and traffic control devices.

[0649] "Means of providing users with mobility information" refers to interfaces and applications that convey traffic conditions and forecast information to users via mobile devices or computers.

[0650] "A means of suggesting the optimal travel route and departure time based on the user's current location and destination" refers to a function that utilizes collected traffic data to calculate and present the most efficient route and appropriate departure time for the user.

[0651] A "means of providing real-time route change guidance" refers to a system that, in response to changes in traffic conditions, proposes new routes to users, supporting efficient travel to their destinations.

[0652] To implement this invention, terminals, servers, and users primarily play their respective roles. The terminal is a visual data acquisition device installed on the street, which acquires real-time traffic flow and conditions as video data. This visual data is immediately transmitted to the server, which is a data management device. The server uses AI technology to analyze the data and learn past traffic patterns. AI-based object detection algorithms such as TensorFlow are used for this analysis and prediction process.

[0653] The server also optimizes travel routes and departure times for users. Specifically, it calculates and suggests the optimal route and departure time based on the user's current location and destination via a smartphone application. Furthermore, based on real-time traffic data, it notifies users of route changes if traffic conditions change during their journey. This allows users to avoid traffic congestion and arrive at their destination more efficiently.

[0654] The hardware primarily consists of a group of cameras to acquire video data and smartphones. Database systems built on cloud services such as AWS and Google Cloud are used for data processing and analysis.

[0655] As a concrete example, consider the morning commute. When a user starts their journey from home to the office, they can use this app to check the optimal route and departure time, allowing them to avoid traffic and reach their destination quickly. Furthermore, real-time route suggestions that adapt to changing circumstances during the journey improve the user's travel efficiency.

[0656] An example of a prompt to a generative AI model is, "Check today's traffic conditions and tell me the fastest route to my destination." This prompt is used to generate information about the user's destination and the most efficient route possible.

[0657] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0658] Step 1:

[0659] The terminal uses cameras installed on the street to acquire real-time visual data of traffic conditions. The input is live video of the traffic scene, and this data is compressed and sent to the server as output. The data processing performed at this stage is the compression of the video stream.

[0660] Step 2:

[0661] The server receives visual data transmitted from the terminal and analyzes the data using an AI-based object detection algorithm. The input is compressed traffic video data, and the output is the analysis results, such as the number of vehicles, speed, and direction of travel. This analysis includes data computation using TensorFlow and other tools.

[0662] Step 3:

[0663] The server utilizes historical travel data to learn traffic patterns based on the analysis results. The input consists of current analysis results and historical traffic data. The output generates predictive information about future traffic conditions. This data processing includes pattern recognition and predictive model construction processes.

[0664] Step 4:

[0665] The user enters their current location and destination via a smartphone application. The application sends a request to a server, which retrieves the optimal travel route and departure time. The input consists of the user's current location, destination, and traffic forecast data, while the output is recommended route information.

[0666] Step 5:

[0667] The server sends information to the application to update the user's route as they travel, based on real-time traffic conditions. The input is the latest traffic data and the user's progress, and the output is a suggested updated route. This allows the user to be presented with the optimal route based on prompts generated using a generative AI model.

[0668] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0669] This invention combines a system that analyzes traffic conditions in real time and predicts future traffic with an emotion engine that recognizes user emotions. This system streamlines traffic management and enables the provision of information tailored to the user.

[0670] First, the terminal (smart camera) continuously acquires video data of the road, compresses it, and sends it to the server. The server uses AI to analyze the video data and records the current traffic situation as numerical data. Furthermore, it learns patterns from the accumulated data and predicts future traffic conditions.

[0671] This system includes an emotion engine to recognize user emotions. Users provide emotional data to the system through a dedicated device or application. The emotion engine analyzes this data to detect the user's stress level and other emotions. The detection results are used to adjust the traffic information provided; for example, users with high stress levels can be provided with more relaxed route information.

[0672] The emotion engine is also incorporated into the traffic signal control process. For example, if driver stress levels are generally high in a particular area, the server uses that information to adjust the signal patterns, making traffic flow smoother and thus reducing stress.

[0673] As a concrete example, consider a scenario where this system is operational during an event in a certain city. The system uses not only normal traffic data but also participant emotion data to guide traffic. As a result, it can adjust traffic light timing at points where congestion is predicted and provide relaxing voice guidance on some routes, creating an optimal traffic environment that takes user comfort into consideration.

[0674] The following describes the processing flow.

[0675] Step 1:

[0676] The terminal acquires road video data in real time. The camera adds a timestamp to each frame and sends the captured data to the server at regular intervals.

[0677] Step 2:

[0678] The video data acquired by the terminal is compressed and sent to the server in a format optimized for efficient data transfer.

[0679] Step 3:

[0680] The server analyzes the received video data. Using AI-based object detection algorithms, it identifies vehicles and pedestrians to understand traffic conditions.

[0681] Step 4:

[0682] The server stores the analysis results in a database. From this data, it learns past and present traffic trends and updates the model to predict future traffic conditions.

[0683] Step 5:

[0684] Users use a dedicated application to send emotional data from their device to a server. This emotional data includes information indicating the user's stress level and current emotional state.

[0685] Step 6:

[0686] The emotion engine on the server analyzes the emotion data sent by the user to identify stress levels and other emotional states.

[0687] Step 7:

[0688] The server adjusts the content of traffic information provided based on the user's emotional state. For users experiencing high stress levels, it suggests relaxing routes and information.

[0689] Step 8:

[0690] The server aggregates user sentiment data from across the region and uses this information to optimize traffic signal control parameters. For example, if stress levels are high in a specific area, the signal timing is adjusted to optimize traffic flow.

[0691] Step 9:

[0692] Users plan their travel based on the provided traffic information. They can use the application to select the most comfortable route and time.

[0693] (Example 2)

[0694] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0695] Current traffic management systems have limitations in real-time traffic information analysis and signal control, and in particular, they have not been able to provide optimal traffic information based on the emotional state of individual users. Therefore, there is a need to improve traffic flow while also reducing the psychological stress on users.

[0696] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0697] In this invention, the server includes means for acquiring video data in real time, means for compressing the acquired video data and transmitting it to the server, means for recording and learning past traffic data, means for predicting future traffic conditions based on the recorded data, means for collecting and analyzing user emotion data, means for providing traffic information according to the user's emotional state, and means for dynamically optimizing traffic signal control based on the user's emotions and traffic conditions. This makes it possible to manage traffic conditions more precisely and realize a comfortable traffic environment by providing information adapted to the user.

[0698] "Real-time" refers to a state where data acquisition and processing occur instantly, without any delay.

[0699] "Video data" refers to a collection of visual information acquired by a camera or other recording device.

[0700] "Compression" is a technique that reduces the amount of data, making it easier to transmit and store that data.

[0701] A "server" is a computer system that provides services to other devices over a network.

[0702] "Traffic data" refers to information about the volume, speed, and flow of vehicles on roads.

[0703] "Learning" is the process by which an algorithm discovers patterns and features from data and utilizes that information.

[0704] "Predicting future traffic conditions" means estimating future traffic flow and congestion based on past and present traffic data.

[0705] "User emotional data" refers to information that indicates a user's psychological state and stress level.

[0706] "Analysis" is the process of processing and examining data to derive meaning and trends from it.

[0707] "Providing traffic information" means informing users about road conditions and recommended routes.

[0708] "Traffic signal control" is the process of adjusting the flow of vehicles on a road by operating traffic signals.

[0709] "Dynamic optimization" means adjusting to the most suitable state in real time according to the situation.

[0710] This invention combines a system that analyzes traffic conditions in real time and predicts future traffic with an emotion engine that recognizes user emotions. This system streamlines traffic management and enables the provision of information tailored to the user.

[0711] First, a smart camera, acting as a terminal, continuously acquires video data of the road, compresses this data, and sends it to a server. The server analyzes the video data using AI technology. Specifically, it utilizes computer vision technology to quantify the number, speed, and direction of vehicles on the road. The analyzed data is stored in a database and recorded as historical traffic data. Then, a machine learning model is used to learn patterns from this data and predict future traffic conditions.

[0712] Furthermore, users provide emotional data to the system through a dedicated device or application. This data is analyzed by an emotion engine to detect the user's stress level and other emotional states. Based on these detection results, personalized traffic information is provided to the user. Specifically, for users with high stress levels, it is possible to suggest calmer routes or provide relaxation messages via voice.

[0713] Furthermore, the system can also incorporate emotional data into the traffic signal control process. If a high level of stress is detected overall in a particular area, the server will adjust the signal patterns based on that information to smooth the overall traffic flow. This helps reduce stress throughout the area.

[0714] For example, when a large-scale event is held in a city, this system analyzes both normal traffic data and the emotional data of event participants to provide optimal traffic guidance. It can adjust traffic light timing at predicted congestion points and enhance user comfort by providing relaxing messages through voice guidance on specific routes.

[0715] An example of a prompt for a generative AI model is, "Based on road congestion and participant sentiment data, please suggest the optimal traffic route and signal control method for the day of the event." This prompt allows the AI ​​to comprehensively analyze diverse information and support decision-making.

[0716] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0717] Step 1:

[0718] The terminal uses smart cameras installed on the road to acquire video data in real time. The input is video of the road, and the output is compressed video data. Compressing this video data reduces network load and enables rapid data transfer. Specifically, frames are captured continuously, and compression is applied at regular intervals.

[0719] Step 2:

[0720] The server receives compressed video data transmitted from the terminal and analyzes it using AI technology. The input is compressed video data, and the output is analyzed numerical data (number of vehicles, speed, direction, etc.). Specifically, computer vision algorithms are applied to quantify the traffic flow on the road.

[0721] Step 3:

[0722] The server stores the analyzed traffic data in a database and trains a machine learning model using historical data. The input is digitized traffic data, and the output is a predictive model. Specifically, it applies machine learning algorithms to learn traffic patterns and generates a model that predicts future traffic conditions.

[0723] Step 4:

[0724] Users input emotional data through a dedicated device or application and send it to a server. The input is emotional information obtained from the user's voice and face, and the output is analyzed emotional data. Specifically, emotion recognition software is used to process the data and determine the user's stress level.

[0725] Step 5:

[0726] The server analyzes the user's emotional data and compares it with current traffic conditions to provide personalized traffic information. The input is the analyzed emotional data and traffic data, and the output is personalized traffic guidance information. In actual processing, the system generates the optimal route and relaxation message for the user through the combination of data.

[0727] Step 6:

[0728] The server utilizes user sentiment data to dynamically optimize traffic signal timing. Inputs are traffic data and sentiment data, and output is an adjusted signal control pattern. Specifically, at a given intersection, it adjusts signal switching times considering the overall user state to improve traffic flow.

[0729] (Application Example 2)

[0730] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0731] Conventional traffic management systems have limitations in efficiently analyzing traffic conditions and do not provide information that takes into account the emotional state of users, resulting in problems with ensuring sufficient user comfort and safety. In particular, with autonomous vehicles, it was difficult to reduce stress because driving parameters could not be adjusted based on passenger emotions.

[0732] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0733] In this invention, the server includes means for acquiring and analyzing video data in real time, means for recording and learning from past traffic data, means for analyzing the user's emotional state and adjusting the content of traffic information provided, and means for automatically adjusting the vehicle's driving parameters based on the passenger's emotions. This improves the accuracy of traffic situation prediction and enables driving and information provision that takes into account the user's emotions.

[0734] A "device that acquires video data in real time" is a device that continuously acquires video data using sensors such as cameras in order to instantly grasp traffic conditions, and provides it in an analyzable format.

[0735] A "device for recording and learning from past traffic data" is a device that accumulates historical data on traffic flow and vehicle movement over a long period of time, generates identification patterns based on this data, and creates a traffic prediction model.

[0736] A "device that analyzes data to predict future traffic conditions" is a device that takes current and past traffic data as input and predicts future traffic conditions and flow rates through analysis of that data.

[0737] A "device that optimizes traffic flow by operating signal control devices" is a device that automatically adjusts the timing and sequence of traffic signals to efficiently manage traffic flow, thereby promoting the reduction of congestion and traffic accidents.

[0738] A "device that analyzes the user's emotional state and adjusts the content of traffic information provided" is a device that analyzes emotional data acquired from the user's device and, based on that analysis, provides traffic information that is adapted to the user's stress level and emotions.

[0739] A "device that automatically adjusts vehicle driving parameters based on passenger emotions" is a device that enhances passenger comfort and safety by dynamically changing the driving characteristics of an autonomous vehicle, such as speed and route, based on the passenger's emotional response.

[0740] In order to implement this invention, the interaction between the server, terminal, and user is of primary importance.

[0741] The server plays a central role in traffic situation analysis. First, it receives video data transmitted in real time from terminals and analyzes traffic conditions using AI models. Machine learning models built with TensorFlow and PyTorch are used for the analysis. Based on the results, it predicts future traffic conditions, issues appropriate instructions to traffic signal control devices, and optimizes traffic flow.

[0742] Simultaneously, the server analyzes the user's emotional data. Emotional data is collected from the user's device (smartphone or wearable device) and processed using an emotion recognition model. Based on the analysis, information and driving instructions tailored to the user's stress level are provided. For example, if high stress is detected, the vehicle's driving parameters are adjusted, and a stress reduction mode is activated. This provides a more relaxed driving experience.

[0743] As a concrete example, in complex urban traffic situations, the user's emotions are analyzed in real time, and a smooth and safe route is suggested. Based on this, the server can automatically adjust the vehicle's speed and route, and can even stream relaxing music.

[0744] As an example of a prompt, the AI ​​model can be instructed to "Suggest the safest and most efficient route based on the current emotional state and traffic data," thereby achieving user-adaptive output.

[0745] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0746] Step 1:

[0747] The terminal acquires road video data in real time. The acquired video data is resized and compressed for efficient handling and sent to the server. This process allows the terminal to provide data in a format that is easy for the server to handle.

[0748] Step 2:

[0749] The server receives compressed video data transmitted from the terminal and decompresses it. An AI model is used to analyze this data and extract current traffic information. Specifically, image recognition technology is used to count the number of vehicles and pedestrians, generating traffic flow data. This data then becomes the input for the next prediction step.

[0750] Step 3:

[0751] The server receives previously accumulated traffic data and current traffic data as input and uses a generating AI model to predict future traffic conditions. The model performs time-series analysis and outputs predicted traffic congestion points and time periods. The resulting future prediction data is used to adjust traffic signal control.

[0752] Step 4:

[0753] Users input emotional data through a dedicated device or smartphone. This emotional data is analyzed in real time by AI, and the user's stress level and mental state are quantified. The emotional state obtained from the analysis is used to adjust the traffic information provided by the server.

[0754] Step 5:

[0755] The server integrates analyzed traffic and sentiment data to instruct signal control units on the optimal signal timing. It also provides users with optimized route suggestions and driving advice. As a result, users can experience safer and more comfortable travel.

[0756] Step 6:

[0757] To influence the user, the server streams relaxation-enhancing content (music and messages) within the autonomous vehicle. This content, tailored to the user's emotional state, directly enhances the driving experience.

[0758] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0759] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0760] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0761] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0762] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0763] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0764] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0765] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0766] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0767] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0768] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0769] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0770] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0771] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0772] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0773] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0774] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0775] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0776] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0777] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0778] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0779] The following is further disclosed regarding the embodiments described above.

[0780] (Claim 1)

[0781] A device that acquires video data in real time,

[0782] A device that records and learns from past traffic data,

[0783] A device that analyzes data to predict future traffic conditions,

[0784] A device that optimizes traffic flow by operating signal control devices,

[0785] A device that provides traffic condition information to users,

[0786] A system that includes this.

[0787] (Claim 2)

[0788] The system according to claim 1, comprising means for compressing real-time video data and transmitting it to a server.

[0789] (Claim 3)

[0790] The system according to claim 1, comprising means for dynamically adjusting the timing of signal control based on traffic conditions.

[0791] "Example 1"

[0792] (Claim 1)

[0793] A means of acquiring video information in real time,

[0794] A means of analyzing acquired video information to identify the status of vehicles and pedestrians,

[0795] A means of recording and learning from past traffic information,

[0796] A means of predicting future traffic conditions based on analytical information,

[0797] A means of optimizing traffic flow by operating a traffic control device,

[0798] A means of providing users with traffic condition information,

[0799] A system that includes this.

[0800] (Claim 2)

[0801] The system according to claim 1, comprising means for compressing real-time video information and transmitting it to an integrator.

[0802] (Claim 3)

[0803] The system according to claim 1, comprising means for dynamically adjusting the timing of traffic control based on traffic conditions.

[0804] "Application Example 1"

[0805] (Claim 1)

[0806] A means of acquiring visual data in real time,

[0807] A means of recording and learning from past movement data,

[0808] A means of analyzing data to predict future movement patterns,

[0809] A means for optimizing the flow of movement by operating a control device,

[0810] A means of providing users with information on their movement status,

[0811] A means of suggesting the optimal travel route and departure time based on the user's current location and destination,

[0812] A means of providing real-time guidance on changes to travel routes,

[0813] A system that includes this.

[0814] (Claim 2)

[0815] The system according to claim 1, further comprising means for compressing real-time visual data and transmitting it to a data management device.

[0816] (Claim 3)

[0817] The system according to claim 1, further comprising means for dynamically adjusting the timing of a control device based on traffic conditions.

[0818] "Example 2 of combining an emotion engine"

[0819] (Claim 1)

[0820] A means of acquiring video data in real time,

[0821] A means of compressing the acquired video data and sending it to the server,

[0822] A means of recording and learning from past traffic data,

[0823] A means of predicting future traffic conditions based on recorded data,

[0824] A means of collecting and analyzing user sentiment data,

[0825] A means of providing traffic information according to the user's emotional state,

[0826] A means for dynamically optimizing traffic signal control based on user emotions and traffic conditions,

[0827] A system that includes this.

[0828] (Claim 2)

[0829] The system according to claim 1, comprising means for compressing real-time video data and transmitting it to a server.

[0830] (Claim 3)

[0831] The system according to claim 1, further comprising means for dynamically adjusting the timing of signal control based on traffic conditions and user sentiment data.

[0832] "Application example 2 when combining with an emotional engine"

[0833] (Claim 1)

[0834] A device that acquires video data in real time,

[0835] A device that records and learns from past traffic data,

[0836] A device that analyzes data to predict future traffic conditions,

[0837] A device that optimizes traffic flow by operating signal control devices,

[0838] A device that analyzes the user's emotional state and adjusts the content of traffic information provided,

[0839] A device that automatically adjusts the vehicle's driving parameters based on the passenger's emotions,

[0840] A system that includes this.

[0841] (Claim 2)

[0842] The system according to claim 1, comprising means for compressing real-time video data and transmitting it to a server.

[0843] (Claim 3)

[0844] The system according to claim 1, further comprising means for dynamically adjusting the timing of signal control based on traffic conditions and the emotional state of the user. [Explanation of Symbols]

[0845] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of acquiring visual data in real time, A means of recording and learning from past movement data, A means of analyzing data to predict future movement patterns, A means for optimizing the flow of movement by operating a control device, A means of providing users with information on their movement status, A means of suggesting the optimal travel route and departure time based on the user's current location and destination, A means of providing real-time guidance on changes to travel routes, A system that includes this.

2. The system according to claim 1, further comprising means for compressing real-time visual data and transmitting it to a data management device.

3. The system according to claim 1, further comprising means for dynamically adjusting the timing of a control device based on traffic conditions.

Images