system

Abstract

Provide a system. 【Solution means】 Receive video information obtained from a data acquisition device, Analysis means for analyzing the video information using a machine learning model; Prediction means for predicting future traffic conditions based on past traffic data; Instruction generation means for generating a control instruction based on the analysis result and prediction information; Control means for transmitting the control instruction to a traffic control device; Recording means for recording the analysis result and prediction information and performing long-term analysis; Providing means for providing the analysis result of the recording means; A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern society, traffic congestion and accidents occur frequently, and the accompanying economic losses and environmental burdens are regarded as problems. Conventional traffic survey and management methods rely heavily on manual work, and it is difficult to collect and analyze data efficiently. Therefore, improvement of the traffic management system is required. For this reason, there is a need for a technology that can accurately grasp the traffic situation in real time and enable efficient analysis and prediction.

Means for Solving the Problems

[0005] This invention provides an analysis means that analyzes video information acquired from a data acquisition device using a machine learning model to identify the number, speed, and location of traffic participants in real time. Furthermore, it includes a prediction means that utilizes past traffic data to predict future traffic conditions, and an instruction generation means that generates instructions to adjust traffic control signals based on the analysis results and prediction information, thereby aiming to alleviate traffic congestion and reduce the risk of accidents. In addition, it proposes a system that improves user convenience by conducting long-term analysis and providing the results through the user's computing device.

[0006] A "data acquisition device" is a device used to collect traffic-related video information, and refers to sensor devices such as street cameras.

[0007] "Visual information" refers to visual data collected by data acquisition devices in order to understand traffic conditions.

[0008] A "machine learning model" refers to an algorithm used to extract and analyze specific patterns or features, and plays a crucial role in data analysis.

[0009] "Analysis means" refers to methods and techniques for processing video information received from a data acquisition device to identify the number, speed, and location of traffic participants.

[0010] "Predictive methods" refer to technologies and algorithms used to predict future traffic conditions and patterns based on past traffic data.

[0011] "Instruction generation means" refers to the process of creating control instructions, such as adjusting traffic signals, based on analysis and prediction.

[0012] "Control means" refers to the methods and techniques for actually operating the traffic control device based on the results of the instruction generation means.

[0013] "Recording means" refers to technologies and systems for storing analyzed and predicted data and making it available for long-term use.

[0014] "Means of provision" refers to methods for informing users of analysis results and predictive information, which are usually provided through the user's computing device.

[0015] "Traffic control devices" refer to traffic signals and other control equipment used to manage the flow of traffic. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12]It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the language used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention is a system for analyzing traffic conditions in real time and optimizing traffic flow. This system receives video information from data acquisition devices installed on the streets and uses machine learning models to analyze it, thereby identifying fluctuations in traffic volume and identifying traffic participants.

[0038] The server plays a central role in this system, receiving video information transmitted from data acquisition devices. The received data is input into a machine learning model, which analyzes information such as the number, speed, and location of vehicles in real time. Based on this analysis, past traffic data is used to predict future traffic conditions. Furthermore, the server uses the results of the analysis and prediction to generate control instructions, including changes to traffic signals, for traffic control devices such as traffic lights. These control instructions aim to smooth traffic flow and reduce the risk of congestion and traffic accidents.

[0039] The terminal's role is to execute control instructions received from the server. Specifically, it sends instructions to traffic control devices to adjust traffic signal timings and operate traffic guidance systems. This operation includes not only adjusting traffic signals but also changing the content of road signs and information boards as needed.

[0040] The user is the ultimate beneficiary of the information. The server provides analyzed traffic information and future predictions to the user's device. Based on this information, the user can choose the optimal route and reduce travel time. For example, it can help the user change their departure time to avoid peak hours. Real-time warnings such as accidents and congestion are also sent to the user, helping them to travel more safely.

[0041] This system automates a series of processes, from data collection and analysis to communication and instruction execution, thereby achieving efficient traffic management and contributing to the smooth operation of urban traffic. For example, if sudden congestion occurs at a particular intersection, the server immediately detects the change and adjusts the timing of traffic lights to improve traffic flow. This adjustment predicts an increase in vehicles using alternative routes and sends appropriate instructions to other intersections. As a result, larger-scale traffic route optimization is achieved.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The server receives video information from data acquisition devices such as street cameras. This video information includes road conditions and the movements of traffic participants.

[0045] Step 2:

[0046] The server inputs the received video information into a machine learning model and performs analysis in real time. This analysis identifies the number, speed, and location of vehicles, and helps to understand traffic flow.

[0047] Step 3:

[0048] The server references historical traffic data and predicts future traffic conditions based on the analysis results. This prediction takes into account patterns specific to certain times of day and days of the week.

[0049] Step 4:

[0050] The server generates control instructions for traffic lights and traffic signs based on the analysis and prediction results. This includes adjusting the color and timing of the signals.

[0051] Step 5:

[0052] The terminal receives control instructions from the server and executes those instructions on the target traffic control device. This adjusts the timing of traffic lights and smooths the flow of traffic.

[0053] Step 6:

[0054] The server records analysis results and predictive data in a database for long-term analysis. This data can be used for future urban planning and traffic management.

[0055] Step 7:

[0056] Users receive real-time traffic information from a server via their devices. Based on this information, users can select the optimal route and use it to plan their travel.

[0057] (Example 1)

[0058] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0059] In urban areas, optimizing traffic flow and reducing congestion is a critical challenge. Traditional methods often fail to fully utilize real-time traffic information, resulting in inefficient traffic control and consequently increased travel times and accident risks. Therefore, a system is needed that can analyze traffic conditions more accurately and quickly to achieve optimal traffic control.

[0060] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0061] In this invention, the server includes an analysis means that receives visual information acquired from a data collection device and processes it using machine learning techniques, a prediction means that predicts future movement conditions based on past movement data, and a command generation means that creates commands based on the analysis results and prediction information. This enables the optimization of traffic flow within the city by responding to changes in traffic conditions in real time and efficiently adjusting traffic signals and guidance displays, thereby mitigating congestion and improving safety.

[0062] A "data collection device" is a device installed at various locations in urban areas to acquire visual information and plays a role in monitoring traffic conditions.

[0063] "Visual information" refers to video and image data acquired through cameras and sensors, and represents the real-time situation of traffic.

[0064] "Machine learning techniques" are technologies that use algorithms to automatically extract features from large amounts of data and perform analysis and predictions.

[0065] "Analysis means" refers to a mechanism that uses machine learning techniques to process visual information and identify the quantity, speed, and location of traffic participants.

[0066] A "predictive means" is a mechanism for predicting future traffic conditions based on past travel data, and provides predictions to improve traffic flow.

[0067] "Command generation means" refers to a mechanism for creating commands for traffic control devices based on analysis results and predictive information.

[0068] "Mobility control devices" refer to devices and systems necessary for managing and controlling the flow of traffic, such as traffic signals and information displays.

[0069] "Recording means" refers to a mechanism for recording analysis results and predictive information and conducting long-term analysis.

[0070] The "supply means" refers to a mechanism for providing recorded analysis results to users, enabling real-time sharing of traffic information.

[0071] This invention is a system for analyzing and optimizing traffic flow in real time, and utilizes information from data collection devices to achieve more efficient traffic control.

[0072] The server plays a central role in the system. It receives visual information from data acquisition devices located on-site and uses this information to perform analysis using machine learning techniques. Specifically, it analyzes the acquired video data using software such as Python and TENSORFLOW®. This allows for the extraction of information such as the number, speed, and location of vehicles in real time, enabling a detailed understanding of traffic conditions.

[0073] The server then uses a generative AI model, referencing historical travel data, to predict future traffic trends. During this process, prompts may be used with the generative AI model. For example, providing specific instructions such as "Show the expected traffic volume fluctuation pattern for the next 30 minutes" can improve prediction accuracy.

[0074] Based on the analysis and prediction results, the server generates commands for traffic control systems. These commands include adjusting traffic light timing and updating road signs. This allows the server to optimize urban traffic flow and provide a safer traffic environment aimed at reducing congestion and preventing accidents.

[0075] The terminal receives commands from the server and puts them into action. Specifically, it adjusts the timing and updates the display content of traffic lights and information boards, which are the targets of its control, according to the commands from the server. In this way, the terminal controls and supports the optimization of traffic flow in real time.

[0076] Users can utilize traffic information analyzed and predicted by the server. Through their devices, users can receive real-time traffic information and use it to select the optimal travel route. For example, users can receive congestion information and change their planned route to reduce travel time. Furthermore, they can receive priority warnings in the event of accidents or other emergencies, supporting safer travel.

[0077] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0078] Step 1:

[0079] The server receives visual information in real time from data acquisition devices. The input consists of video data from cameras placed at multiple intersections and major roads. Upon receiving this video data, the server converts it to a specific format, stores it temporarily, and prepares it for analysis.

[0080] Step 2:

[0081] The server analyzes the received video data using machine learning techniques. The input is the video data temporarily stored in Step 1. The server uses software such as Python and TensorFlow to analyze this data and extract the number, speed, and location of traffic participants. The output is the extracted real-time traffic information.

[0082] Step 3:

[0083] The server uses a generative AI model to predict future traffic conditions based on the analyzed traffic information. This process takes historical travel data and current traffic information as input, and uses prompts to provide specific prediction instructions to the generative AI model. For example, it might use a prompt like, "Show the expected traffic light waiting time for the next hour." The output is predicted traffic fluctuation data.

[0084] Step 4:

[0085] The server generates commands for the traffic control system based on the analysis results and predictive information. The input is the data obtained from steps 2 and 3. The server generates specific commands, such as adjusting the sequence of traffic signals or updating information boards, and prepares to send them to the terminal. The output is the generated command data.

[0086] Step 5:

[0087] The terminal receives commands sent from the server and executes them. The input is the command data generated in step 4. The terminal plays a role in controlling the actual flow of traffic by making specific adjustments to various traffic control devices. The output is an update of the operating status of the control devices.

[0088] Step 6:

[0089] The user receives and utilizes traffic information analyzed and predicted by the server. The input is real-time traffic information and prediction data provided by the server. The user uses this information to select the optimal travel route. They can also adjust departure times as needed to reduce travel time. The output is the optimization of the user's travel plan.

[0090] (Application Example 1)

[0091] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0092] Urban traffic experiences significant fluctuations depending on the time of day, resulting in daily congestion and traffic accidents. This increases commuting and school travel times, and also hinders the rapid response of emergency vehicles to the scene. Furthermore, it is difficult to effectively predict future traffic conditions and provide users with useful information in real time. This invention aims to solve these problems, smooth traffic flow, and provide users with optimal travel routes and times.

[0093] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0094] In this invention, the server includes means for receiving video information acquired from a data acquisition device and analyzing it using a machine learning model, means for predicting future traffic conditions based on past traffic data, and means for generating control instructions based on the analysis results and predicted information and transmitting the information to the user terminal. This enables real-time analysis and prediction of traffic conditions, as well as personalized route suggestions and the presentation of optimal travel times.

[0095] A "data acquisition device" is a device installed on streets and at various transportation hubs to acquire video information for understanding traffic conditions.

[0096] A "machine learning model" is a learning system equipped with algorithms used to analyze traffic conditions and vehicle movements.

[0097] "Analysis means" refers to a method of analyzing video information received from a data acquisition device using a machine learning model to identify the number, speed, and location of traffic participants.

[0098] A "predictive method" is a process for predicting future traffic conditions based on past traffic data.

[0099] The "instruction generation means" is a function that generates control instructions for traffic control devices and users based on analysis results and predictive information.

[0100] A "traffic control system" is a device that controls the operation of traffic lights and road signs, and adjusts the flow of traffic.

[0101] "Recording methods" refer to methods for long-term storage and analysis of analysis results and predictive information to be used for future traffic improvements.

[0102] "Provisioning means" refers to the process of providing the analysis results from the recording means to the user's information processing device, enabling personalized route suggestions.

[0103] A "user terminal" is a computer device that receives analyzed traffic information and provides it to users in real time.

[0104] Regarding the embodiment for carrying out the invention, this system analyzes real-time traffic conditions and utilizes that information to perform traffic control and provide optimal travel suggestions to users. This system mainly consists of three components: a server, a terminal, and a user.

[0105] The server receives video information acquired from data acquisition devices installed on the streets. This video information is captured using Python's OpenCV. Next, the server uses a machine learning model employing TensorFlow to analyze the number, speed, and location of traffic participants in real time. The analyzed data is stored in the backend using the Django framework and used by a prediction algorithm to estimate future traffic conditions. Furthermore, the server generates instructions based on these analysis results and sends them to traffic control devices and users.

[0106] The terminal makes appropriate adjustments to traffic lights and signs based on control instructions sent from the server. Furthermore, analyzed traffic information is provided to the user's mobile device in real time through an application developed with React Native. This application suggests optimal routes and departure times to the user and provides recommendations to avoid congestion.

[0107] Users can utilize the information provided through their mobile devices to create optimized travel plans. For example, this could allow users to adjust their departure time based on traffic warnings provided by the app, thereby shortening their commute time.

[0108] By utilizing a generative AI model, it is also possible to suggest the most appropriate prompt message based on the current traffic conditions. An example of a prompt message would be: "Traffic volume is increasing at the current intersection. Please generate the best prompt to suggest a safe and smooth alternative route to the app user."

[0109] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0110] Step 1:

[0111] The server receives video information streamed from the data acquisition device. The received data is raw video to understand traffic flow in real time. Specifically, the OpenCV library in Python is used to capture the feed from the camera and prepare each frame for analysis.

[0112] Step 2:

[0113] The server uses TensorFlow to input video data into a machine learning model and analyze the number, speed, and location of vehicles. First, the received frames are preprocessed and converted into a format suitable for the model (e.g., resizing and normalization). Then, the data is fed into the model, and vehicle detection and tracking are performed to determine the number, location, and speed of vehicles.

[0114] Step 3:

[0115] The server uses the Django framework to store the analysis results in a database and combines them with historical traffic data to predict future traffic conditions. Specifically, it applies a time-series model using current traffic data to instantly calculate future traffic patterns and predict the occurrence and mitigation of congestion.

[0116] Step 4:

[0117] The server uses a generative AI model based on analysis results and predictive information to create prompt messages and send them to traffic control devices and user terminals. Instruction generation includes early warnings of predicted congestion points and suggestions for adjusting traffic light timings. This step leverages the generative AI model to create automated prompts and suggest optimal travel routes and times for users.

[0118] Step 5:

[0119] The terminal receives instructions from the server and performs specific traffic control actions. This includes, for example, shortening or extending the phase of traffic lights or dynamically updating traffic signs. Furthermore, the user's information processing device is provided with real-time suggestions for the optimal route and departure time through the application.

[0120] Step 6:

[0121] The user adjusts their travel plan using information provided by their device. For example, based on a provided prompt message such as, "Traffic is increasing at the current intersection. Generate the best prompt to suggest a safe and smooth alternative route to the app user," the system facilitates the selection of a smoother route and avoidance of traffic congestion.

[0122] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0123] This invention aims to provide optimal traffic information to users by incorporating an emotion engine that recognizes the emotional state of users into a system for improving traffic conditions in real time. In addition to basic traffic management functions such as receiving, analyzing, predicting, and controlling video information from data acquisition devices, this system also provides information tailored to individual users by analyzing the emotions of the users.

[0124] After receiving video information, the server performs analysis using a machine learning model to understand traffic flow. It then combines this with historical data to predict future traffic conditions and adjusts traffic signals based on the analysis results and predictions. Furthermore, the server activates an emotion engine to receive emotional information from the user's device. This emotion engine analyzes the user's facial expressions and voice data to identify emotional states such as stress, dissatisfaction, and satisfaction.

[0125] The terminal receives control instructions from the server and sends those instructions to the actual traffic control system. This process guides traffic flow appropriately and alleviates congestion. Furthermore, based on the analysis results of the emotion engine, the terminal provides traffic information and route suggestions that take the user's emotions into consideration. For example, for a user experiencing high stress levels, it can suggest routes that avoid congestion or quieter paths.

[0126] For users, real-time traffic information provided by the server can be received via devices such as smartphones. By receiving customized information tailored to the user's emotional state, a more comfortable and safer journey becomes possible. This system aims to enhance user satisfaction with their travel experiences.

[0127] As a concrete example, if congestion is predicted during rush hour, the server will grasp the situation early and adjust traffic signals accordingly. At the same time, it will detect increased user stress through an emotion engine. Based on this information, the terminal will suggest measures to reduce stress, such as recommending the use of public transportation or alternative departure times. In this way, the system optimizes traffic flow while providing information that takes individual emotions into consideration.

[0128] The following describes the processing flow.

[0129] Step 1:

[0130] The server receives video information transmitted from the data acquisition device. This includes camera data used to understand road conditions, among other things.

[0131] Step 2:

[0132] The server inputs the received video information into a machine learning model and performs analysis in real time. This analysis allows for the identification of the number, speed, and location of vehicles, and enables the understanding of traffic flow.

[0133] Step 3:

[0134] The server uses current analysis results based on past traffic data to predict future traffic conditions. These predictions are used to adjust traffic signals and avoid congestion.

[0135] Step 4:

[0136] The server receives emotional information from the user's device and uses an emotion engine to evaluate the user's emotional state. This is done to understand stress levels, satisfaction levels, and other factors through facial expression analysis and voice data analysis.

[0137] Step 5:

[0138] The server generates control instructions to send to the traffic control system based on analysis results, predictive information, and the user's emotional state. These instructions include changes to traffic signals and adjustments to route guidance.

[0139] Step 6:

[0140] The terminal transmits control instructions received from the server to the traffic control system, which then operates traffic lights and information boards. This optimizes the flow of traffic.

[0141] Step 7:

[0142] The server selects information based on the user's emotional state and sends it to the user's device in order to provide the user with suitable routes and traffic information in real time. This information makes the user's journey more comfortable.

[0143] Step 8:

[0144] Based on the customized traffic information they receive, users can adjust their travel plans to reach their destination more efficiently and safely.

[0145] (Example 2)

[0146] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0147] While conventional traffic management systems have the capability to analyze and control traffic flow and congestion in real time, they have limitations in providing information that takes into account the individual emotional state of users. As a result, they cannot adequately reduce user stress and dissatisfaction, making it difficult to improve satisfaction with travel.

[0148] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0149] In this invention, the server includes means for receiving image information acquired from data collection means and analyzing it using a machine learning method, means for predicting future traffic conditions based on past traffic data, and means for analyzing the user's emotional information and providing personalized traffic information. This makes it possible to provide real-time traffic information that takes into account the user's emotional state, thereby realizing comfortable and safe travel.

[0150] "Data collection means" is a general term for devices and sensors used to acquire image information and environmental data related to traffic conditions.

[0151] "Machine learning methods" refer to algorithms and models that allow computers to improve their task performance based on experience, and are particularly used in traffic analysis for recognizing image and audio data.

[0152] "Analysis means" refers to software or algorithms that process received data to identify current traffic flow and the emotional state of users.

[0153] "Predictive methods" refer to algorithms and processing methods used to predict future traffic conditions based on past data.

[0154] "User emotional information" refers to data on the user's psychological state obtained from their facial expressions and voice, and is used to quantify stress levels, satisfaction levels, and other factors.

[0155] "Personalized traffic information" refers to the provision of travel routes and traffic information that are customized based on the user's emotions and current traffic conditions.

[0156] "Means of providing" refers to methods and devices for transmitting specific information to users, particularly system configurations for providing personalized information in real time.

[0157] This invention is a system for optimizing traffic conditions and providing personalized information. The system aims to improve user satisfaction through the coordinated operation of the server, terminal, and user units.

[0158] The server receives image information in real time from traffic cameras and sensors through data collection methods. The received data is processed using image processing libraries such as OpenCV. Furthermore, TensorFlow is used as a machine learning method to analyze the data and understand traffic flow and congestion levels.

[0159] The server uses LSTM models and other tools to predict future traffic conditions based on past traffic data. This prediction enables the control of traffic signals and the optimization of routes.

[0160] In addition, the server activates an emotion engine to analyze facial and voice data obtained from the user's device. This allows the system to identify the user's stress and satisfaction levels, enabling personalized information delivery.

[0161] The terminal receives control instructions from the server and sends instructions to traffic control devices such as traffic lights. Furthermore, the terminal utilizes Google® Maps API and other services to deliver personalized traffic information to the user's device. As a result, users are offered routes that avoid congestion and comfortable travel paths.

[0162] Users receive real-time traffic information provided by their devices through their smart devices. This reduces the stress of travel and allows users to have a safe and comfortable experience.

[0163] For example, if congestion is predicted during rush hour, the server uses a prediction algorithm to adjust the timing of traffic signals to smooth traffic flow. Simultaneously, the emotion engine detects the user's stress level, and the terminal notifies the user with suggestions to encourage the use of public transportation. In this way, it becomes possible to provide information that takes into account both traffic conditions and the user's psychological state.

[0164] Examples of prompts generated using AI models include, "Please tell me how to provide traffic information that takes the user's emotions into consideration." This enables a variety of approaches to improving the user's travel experience.

[0165] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0166] Step 1:

[0167] The server acquires image information in real time from traffic cameras and sensors via data collection methods. The input image information is preprocessed using image processing libraries such as OpenCV. Specifically, this includes image normalization and vehicle detection. The output is image data prepared for analysis.

[0168] Step 2:

[0169] The server uses TensorFlow to analyze image data with a machine learning model to understand traffic flow and vehicle density. The input here is the output data from step 1, and the number and location of traffic participants are identified through the analysis. The output is numerical data indicating traffic conditions and information on traffic patterns.

[0170] Step 3:

[0171] The server uses an LSTM model to combine historical traffic data with current data from step 2 to predict future traffic conditions. The input consists of historical traffic data and current traffic data. Through data calculations, the prediction algorithm estimates future congestion and delays. The output is the predicted traffic condition for the following time period.

[0172] Step 4:

[0173] The server acquires user emotional information and analyzes it using an emotion engine. Inputs include audio and facial expression data from the user's device. The analysis results quantify the user's stress level and satisfaction level. The output is numerical information about the user's emotional state.

[0174] Step 5:

[0175] The server generates instructions for adjusting traffic signals and providing route guidance based on the analysis results and predictive information. It then executes a control algorithm using the outputs from steps 2, 3, and 4 as input. Specifically, it generates instructions to adjust the timing of traffic signals to alleviate congestion. The output consists of instructions related to traffic control.

[0176] Step 6:

[0177] The terminal receives control instructions from the server and transmits them to the actual traffic control device. The input is the output information from step 5. Specific actions include switching signals and updating traffic information. The output is the traffic environment resulting from the execution of the control instructions.

[0178] Step 7:

[0179] The device provides users with personalized traffic information. Inputs include traffic data and sentiment analysis results from a server. Using this information, and leveraging the Google Maps API, the device notifies users of optimal routes and travel advice tailored to their device. Output is customized information displayed on the user's device.

[0180] Step 8:

[0181] The user makes a move based on information received from the device. The input is the information provided in step 7. For example, a specific action would be to select a route that avoids congestion and move accordingly. The output is an improvement in the user's travel experience.

[0182] (Application Example 2)

[0183] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0184] While understanding and managing traffic conditions has been done with conventional technologies, providing information that takes into account the emotional state of users has been insufficient. In particular, with the spread of autonomous vehicles, there is a need for technologies that reduce user stress and anxiety and provide safer and more comfortable travel. This invention aims to solve these problems and realize personalized traffic information provision that responds to the emotions of users.

[0185] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0186] In this invention, the server includes means for receiving video information acquired from a data acquisition device, means for analyzing the emotional state, and means for providing emotionally-based routes to the user via an in-vehicle device. This makes it possible to travel in a way that takes the user's emotional state into consideration and reduces stress and anxiety.

[0187] A "data acquisition device" is a device that collects video information necessary to recognize traffic conditions.

[0188] A "machine learning model" uses statistical methods to analyze patterns in traffic conditions based on large amounts of data and predict future situations.

[0189] "Analysis means" refers to methods for identifying the characteristics and circumstances of traffic participants based on acquired video information.

[0190] A "predictive tool" is a means of predicting future traffic conditions based on past traffic data and current conditions.

[0191] "Instruction generation means" refers to means for generating instructions for traffic control based on predictive information and analysis results.

[0192] "Control means" refers to means that receive generated instructions and transmit appropriate commands to the traffic control device.

[0193] "Recording means" refers to means for storing data and making it available for long-term analysis.

[0194] "Means of provision" refers to the means of providing recorded information or analysis results to a user or some system.

[0195] An "emotion recognition tool" is a means of providing information tailored to individual users by analyzing their emotional state.

[0196] An "in-vehicle device" is a device installed inside a vehicle that functions as an interface with the user and displays traffic information and route suggestions.

[0197] To implement this invention, a system for optimizing traffic conditions is constructed. First, the server receives video information from a data acquisition device and analyzes this information using a machine learning model. Based on the analyzed data, it predicts traffic flow and the occurrence of congestion. Based on this, the server generates instructions for traffic lights and other traffic control devices and transmits them through the control means.

[0198] Furthermore, the server analyzes the user's emotional state using emotion recognition tools. This process involves acquiring facial and voice data using cameras and microphones, and utilizing an emotion analysis engine (for example, a common cloud-based emotion analysis API). The analyzed emotion information forms the basis for the server to provide personalized traffic information to each user.

[0199] The terminal displays information sent from the server to the user in real time via an in-vehicle device. This information includes route suggestions that take the user's emotions into consideration. The in-vehicle interface is designed to be operable via touch panel and voice recognition.

[0200] Users can receive information on their smartphones or in-vehicle devices and travel comfortably by utilizing emotionally sensitive options. For example, the system suggests routes that avoid congestion to users experiencing high stress levels. This is particularly useful for users commuting or traveling.

[0201] An example of a prompt to a generative AI model is, "The user is planning a holiday drive; please suggest a relaxing route with beautiful scenery." This prompt forms the basis for providing information that matches the user's desired experience.

[0202] In this way, the present invention can be implemented as a system in which the server, terminal, and user each play their own unique roles and work together to optimize traffic conditions and improve the user experience.

[0203] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0204] Step 1:

[0205] The server receives video information acquired from data acquisition devices. Video data from cameras installed on roads is used as input. The server collects this data and stores it in a database for analysis in a digital format. The output is data ready for the next analysis step.

[0206] Step 2:

[0207] The server analyzes video information using a machine learning model. The input is the video data organized in Step 1. Based on this data, the server recognizes the number, speed, and location of traffic participants. This analysis is performed to identify traffic flow and potential congestion information, and the output generates information on the current state of traffic.

[0208] Step 3:

[0209] The server predicts future traffic conditions based on historical traffic data. Input includes not only current traffic information but also historical data. The server analyzes this data using statistical algorithms. The output is a prediction of traffic conditions for the near future.

[0210] Step 4:

[0211] The server generates instructions based on the analysis results and predictive information. The traffic conditions and predictive information obtained up to step 3 are used as input. Based on this, the server formulates specific traffic control instructions, such as adjusting traffic light timings and suggesting alternative routes. Specific control commands are generated as output.

[0212] Step 5:

[0213] The server collects and analyzes data to recognize the user's emotional state. Data from the user's smartphone or cameras and microphones installed in the vehicle are used as input. An emotion analysis engine extracts emotional information from the user's facial expressions and voice. This process identifies the user's emotional state, and that information is provided as output.

[0214] Step 6:

[0215] The server provides customized traffic information based on emotional data. It considers emotional states and traffic data as input to generate optimal routes and traffic-related suggestions for the user. Specifically, it can suggest routes with relaxing scenery, for example. The output is customized traffic information displayed on the user's device.

[0216] Step 7:

[0217] The terminal displays control instructions and emotion-sensitive suggestions from the server on the in-vehicle device. Inputs are control instructions and customized traffic information sent from the server. The terminal notifies the user of these and helps them take appropriate action. Outputs include information to guide the user's actions, provided through the in-vehicle display and audio output.

[0218] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0219] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0220] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0221] [Second Embodiment]

[0222] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0223] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0224] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0225] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0226] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0227] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0228] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0229] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0230] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0231] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0232] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0233] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0234] This invention is a system for analyzing traffic conditions in real time and optimizing traffic flow. This system receives video information from data acquisition devices installed on the streets and uses machine learning models to analyze it, thereby identifying fluctuations in traffic volume and identifying traffic participants.

[0235] The server plays a central role in this system, receiving video information transmitted from data acquisition devices. The received data is input into a machine learning model, which analyzes information such as the number, speed, and location of vehicles in real time. Based on this analysis, past traffic data is used to predict future traffic conditions. Furthermore, the server uses the results of the analysis and prediction to generate control instructions, including changes to traffic signals, for traffic control devices such as traffic lights. These control instructions aim to smooth traffic flow and reduce the risk of congestion and traffic accidents.

[0236] The terminal's role is to execute control instructions received from the server. Specifically, it sends instructions to traffic control devices to adjust traffic signal timings and operate traffic guidance systems. This operation includes not only adjusting traffic signals but also changing the content of road signs and information boards as needed.

[0237] The user is the ultimate beneficiary of the information. The server provides analyzed traffic information and future predictions to the user's device. Based on this information, the user can choose the optimal route and reduce travel time. For example, it can help the user change their departure time to avoid peak hours. Real-time warnings such as accidents and congestion are also sent to the user, helping them to travel more safely.

[0238] This system automates a series of processes, from data collection and analysis to communication and instruction execution, thereby achieving efficient traffic management and contributing to the smooth operation of urban traffic. For example, if sudden congestion occurs at a particular intersection, the server immediately detects the change and adjusts the timing of traffic lights to improve traffic flow. This adjustment predicts an increase in vehicles using alternative routes and sends appropriate instructions to other intersections. As a result, larger-scale traffic route optimization is achieved.

[0239] The following describes the processing flow.

[0240] Step 1:

[0241] The server receives video information from data acquisition devices such as street cameras. This video information includes road conditions and the movements of traffic participants.

[0242] Step 2:

[0243] The server inputs the received video information into a machine learning model and performs analysis in real time. This analysis identifies the number, speed, and location of vehicles, and helps to understand traffic flow.

[0244] Step 3:

[0245] The server references historical traffic data and predicts future traffic conditions based on the analysis results. This prediction takes into account patterns specific to certain times of day and days of the week.

[0246] Step 4:

[0247] The server generates control instructions for traffic lights and traffic signs based on the analysis and prediction results. This includes adjusting the color and timing of the signals.

[0248] Step 5:

[0249] The terminal receives control instructions from the server and executes those instructions on the target traffic control device. This adjusts the timing of traffic lights and smooths the flow of traffic.

[0250] Step 6:

[0251] The server records analysis results and predictive data in a database for long-term analysis. This data can be used for future urban planning and traffic management.

[0252] Step 7:

[0253] Users receive real-time traffic information from a server via their devices. Based on this information, users can select the optimal route and use it to plan their travel.

[0254] (Example 1)

[0255] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0256] In urban areas, optimizing traffic flow and reducing congestion is a critical challenge. Traditional methods often fail to fully utilize real-time traffic information, resulting in inefficient traffic control and consequently increased travel times and accident risks. Therefore, a system is needed that can analyze traffic conditions more accurately and quickly to achieve optimal traffic control.

[0257] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0258] In this invention, the server includes an analysis means that receives visual information acquired from a data collection device and processes it using machine learning techniques, a prediction means that predicts future movement conditions based on past movement data, and a command generation means that creates commands based on the analysis results and prediction information. This enables the optimization of traffic flow within the city by responding to changes in traffic conditions in real time and efficiently adjusting traffic signals and guidance displays, thereby mitigating congestion and improving safety.

[0259] A "data collection device" is a device installed at various locations in urban areas to acquire visual information and plays a role in monitoring traffic conditions.

[0260] "Visual information" refers to video and image data acquired through cameras and sensors, and represents the real-time situation of traffic.

[0261] "Machine learning techniques" are technologies that use algorithms to automatically extract features from large amounts of data and perform analysis and predictions.

[0262] "Analysis means" refers to a mechanism that uses machine learning techniques to process visual information and identify the quantity, speed, and location of traffic participants.

[0263] A "predictive means" is a mechanism for predicting future traffic conditions based on past travel data, and provides predictions to improve traffic flow.

[0264] "Command generation means" refers to a mechanism for creating commands for traffic control devices based on analysis results and predictive information.

[0265] "Mobility control devices" refer to devices and systems necessary for managing and controlling the flow of traffic, such as traffic signals and information displays.

[0266] "Recording means" refers to a mechanism for recording analysis results and predictive information and conducting long-term analysis.

[0267] The "supply means" refers to a mechanism for providing recorded analysis results to users, enabling real-time sharing of traffic information.

[0268] This invention is a system for analyzing and optimizing traffic flow in real time, and utilizes information from data collection devices to achieve more efficient traffic control.

[0269] The server plays a central role in the system. It receives visual information from data collection devices located on-site and uses this information to perform analysis using machine learning techniques. Specifically, it uses software such as Python and TensorFlow to analyze the acquired video data. This allows for the extraction of information such as the number, speed, and location of vehicles in real time, enabling a detailed understanding of traffic conditions.

[0270] The server then uses a generative AI model, referencing historical travel data, to predict future traffic trends. During this process, prompts may be used with the generative AI model. For example, providing specific instructions such as "Show the expected traffic volume fluctuation pattern for the next 30 minutes" can improve prediction accuracy.

[0271] Based on the analysis and prediction results, the server generates commands for traffic control systems. These commands include adjusting traffic light timing and updating road signs. This allows the server to optimize urban traffic flow and provide a safer traffic environment aimed at reducing congestion and preventing accidents.

[0272] The terminal receives commands from the server and puts them into action. Specifically, it adjusts the timing and updates the display content of traffic lights and information boards, which are the targets of its control, according to the commands from the server. In this way, the terminal controls and supports the optimization of traffic flow in real time.

[0273] Users can utilize traffic information analyzed and predicted by the server. Through their devices, users can receive real-time traffic information and use it to select the optimal travel route. For example, users can receive congestion information and change their planned route to reduce travel time. Furthermore, they can receive priority warnings in the event of accidents or other emergencies, supporting safer travel.

[0274] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0275] Step 1:

[0276] The server receives visual information in real time from data acquisition devices. The input consists of video data from cameras placed at multiple intersections and major roads. Upon receiving this video data, the server converts it to a specific format, stores it temporarily, and prepares it for analysis.

[0277] Step 2:

[0278] The server analyzes the received video data using machine learning techniques. The input is the video data temporarily stored in Step 1. The server uses software such as Python and TensorFlow to perform analysis to extract the number, speed, and position of traffic participants from these data. The output is the extracted real-time traffic information.

[0279] Step 3:

[0280] Based on the analyzed traffic information, the server uses a generative AI model to predict future traffic situations. In this process, past movement data and current traffic information are used as inputs, and prompt sentences are utilized to give specific prediction instructions to the generative AI model. For example, a prompt sentence like "Please show the expected signal waiting time within the next hour" is used. The output is the predicted traffic variation data.

[0281] Step 4:

[0282] Based on the analysis results and prediction information, the server generates commands for the traffic control device. The input is the data obtained from Step 2 and Step 3. The server generates specific commands such as adjusting the signal sequence and updating the information guide board, and prepares to send these to the terminal. The output is the generated command data.

[0283] Step 5:

[0284] The terminal receives the commands sent from the server and proceeds to execute them. The input is the command data generated in Step 4. The terminal performs specific adjustments to various traffic control devices and plays a role in controlling the actual traffic flow. The output is the update of the operating state of the control device.

[0285] Step 6:

[0286] The user receives the traffic information analyzed and predicted by the server and utilizes it. The input is the real-time traffic information and prediction data provided by the server. The user makes use of these to select an optimal travel route. Also, if necessary, the departure time is changed to shorten the travel time. The output is the optimization of the user's travel plan.

[0287] (Application Example 1)

[0288] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0289] Traffic in urban areas varies greatly depending on the time of day, and traffic jams and traffic accidents occur daily. This increases commuting and school commuting times, and also causes problems such that emergency vehicles cannot reach the scene quickly. Furthermore, it is difficult to effectively predict future traffic conditions and provide useful information to users in real time. The present invention aims to solve these problems, smooth the traffic flow, and provide users with an optimal travel route and time.

[0290] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following respective means <​​​​​​​​​A "machine learning model" is a learning system equipped with algorithms used to analyze traffic conditions and vehicle movements.

[0294] "Analysis means" refers to a method of analyzing video information received from a data acquisition device using a machine learning model to identify the number, speed, and location of traffic participants.

[0295] A "predictive method" is a process for predicting future traffic conditions based on past traffic data.

[0296] The "instruction generation means" is a function that generates control instructions for traffic control devices and users based on analysis results and predictive information.

[0297] A "traffic control system" is a device that controls the operation of traffic lights and road signs, and adjusts the flow of traffic.

[0298] "Recording methods" refer to methods for long-term storage and analysis of analysis results and predictive information to be used for future traffic improvements.

[0299] "Provisioning means" refers to the process of providing the analysis results from the recording means to the user's information processing device, enabling personalized route suggestions.

[0300] A "user terminal" is a computer device that receives analyzed traffic information and provides it to users in real time.

[0301] Regarding the embodiment for carrying out the invention, this system analyzes real-time traffic conditions and utilizes that information to perform traffic control and provide optimal travel suggestions to users. This system mainly consists of three components: a server, a terminal, and a user.

[0302] The server receives video information acquired from data acquisition devices installed on the street. Such video information is captured using OpenCV in Python. Next, the server uses a machine learning model employing TensorFlow to analyze the number, speed, and position of traffic participants in real time. The analyzed data is saved to the backend using the Django framework and utilized for estimating future traffic conditions by a prediction algorithm. Furthermore, the server generates instructions based on these analysis results and transmits them to traffic control devices and users.

[0303] Based on the control instructions sent from the server, the terminal makes appropriate adjustments to traffic lights and traffic signs. Also, the analyzed traffic information is provided in real time to the user's mobile terminal through an application developed with React Native. This application proposes the optimal travel route and departure time for the user and provides recommendation information to avoid congestion.

[0304] The user can utilize the information provided through their mobile terminal to formulate an optimized travel plan. For example, a case is envisioned where the user adjusts their departure time based on the traffic warnings provided by the application and shortens their commuting time.

[0305] It is also possible to utilize a generative AI model to propose an optimal prompt sentence according to the current traffic situation. Examples of prompt sentences include "The traffic volume at the current intersection is increasing. Please generate an optimal prompt for proposing a safe and smooth alternative route to app users."

[0306] The flow of specific processing in Application Example 1 will be described using FIG. 12.

[0307] Step 1:

[0308] The server receives video information streamed from the data acquisition device. The received data is raw video to understand traffic flow in real time. Specifically, the OpenCV library in Python is used to capture the feed from the camera and prepare each frame for analysis.

[0309] Step 2:

[0310] The server uses TensorFlow to input video data into a machine learning model and analyze the number, speed, and location of vehicles. First, the received frames are preprocessed and converted into a format suitable for the model (e.g., resizing and normalization). Then, the data is fed into the model, and vehicle detection and tracking are performed to determine the number, location, and speed of vehicles.

[0311] Step 3:

[0312] The server uses the Django framework to store the analysis results in a database and combines them with historical traffic data to predict future traffic conditions. Specifically, it applies a time-series model using current traffic data to instantly calculate future traffic patterns and predict the occurrence and mitigation of congestion.

[0313] Step 4:

[0314] The server uses a generative AI model based on analysis results and predictive information to create prompt messages and send them to traffic control devices and user terminals. Instruction generation includes early warnings of predicted congestion points and suggestions for adjusting traffic light timings. This step leverages the generative AI model to create automated prompts and suggest optimal travel routes and times for users.

[0315] Step 5:

[0316] The terminal receives instructions from the server and performs specific traffic control actions. This includes, for example, shortening or extending the phase of traffic lights or dynamically updating traffic signs. Furthermore, the user's information processing device is provided with real-time suggestions for the optimal route and departure time through the application.

[0317] Step 6:

[0318] The user adjusts their travel plan using information provided by their device. For example, based on a provided prompt message such as, "Traffic is increasing at the current intersection. Generate the best prompt to suggest a safe and smooth alternative route to the app user," the system facilitates the selection of a smoother route and avoidance of traffic congestion.

[0319] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0320] This invention aims to provide optimal traffic information to users by incorporating an emotion engine that recognizes the emotional state of users into a system for improving traffic conditions in real time. In addition to basic traffic management functions such as receiving, analyzing, predicting, and controlling video information from data acquisition devices, this system also provides information tailored to individual users by analyzing the emotions of the users.

[0321] After receiving video information, the server performs analysis using a machine learning model to understand traffic flow. It then combines this with historical data to predict future traffic conditions and adjusts traffic signals based on the analysis results and predictions. Furthermore, the server activates an emotion engine to receive emotional information from the user's device. This emotion engine analyzes the user's facial expressions and voice data to identify emotional states such as stress, dissatisfaction, and satisfaction.

[0322] The terminal receives control instructions from the server and sends those instructions to the actual traffic control system. This process guides traffic flow appropriately and alleviates congestion. Furthermore, based on the analysis results of the emotion engine, the terminal provides traffic information and route suggestions that take the user's emotions into consideration. For example, for a user experiencing high stress levels, it can suggest routes that avoid congestion or quieter paths.

[0323] For users, real-time traffic information provided by the server can be received via devices such as smartphones. By receiving customized information tailored to the user's emotional state, a more comfortable and safer journey becomes possible. This system aims to enhance user satisfaction with their travel experiences.

[0324] As a concrete example, if congestion is predicted during rush hour, the server will grasp the situation early and adjust traffic signals accordingly. At the same time, it will detect increased user stress through an emotion engine. Based on this information, the terminal will suggest measures to reduce stress, such as recommending the use of public transportation or alternative departure times. In this way, the system optimizes traffic flow while providing information that takes individual emotions into consideration.

[0325] The following describes the processing flow.

[0326] Step 1:

[0327] The server receives video information transmitted from the data acquisition device. This includes camera data used to understand road conditions, among other things.

[0328] Step 2:

[0329] The server inputs the received video information into a machine learning model and performs analysis in real time. This analysis allows for the identification of the number, speed, and location of vehicles, and enables the understanding of traffic flow.

[0330] Step 3:

[0331] The server uses current analysis results based on past traffic data to predict future traffic conditions. These predictions are used to adjust traffic signals and avoid congestion.

[0332] Step 4:

[0333] The server receives emotional information from the user's device and uses an emotion engine to evaluate the user's emotional state. This is done to understand stress levels, satisfaction levels, and other factors through facial expression analysis and voice data analysis.

[0334] Step 5:

[0335] The server generates control instructions to send to the traffic control system based on analysis results, predictive information, and the user's emotional state. These instructions include changes to traffic signals and adjustments to route guidance.

[0336] Step 6:

[0337] The terminal transmits control instructions received from the server to the traffic control system, which then operates traffic lights and information boards. This optimizes the flow of traffic.

[0338] Step 7:

[0339] The server selects information based on the user's emotional state and sends it to the user's device in order to provide the user with suitable routes and traffic information in real time. This information makes the user's journey more comfortable.

[0340] Step 8:

[0341] Based on the customized traffic information they receive, users can adjust their travel plans to reach their destination more efficiently and safely.

[0342] (Example 2)

[0343] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0344] While conventional traffic management systems have the capability to analyze and control traffic flow and congestion in real time, they have limitations in providing information that takes into account the individual emotional state of users. As a result, they cannot adequately reduce user stress and dissatisfaction, making it difficult to improve satisfaction with travel.

[0345] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0346] In this invention, the server includes means for receiving image information acquired from data collection means and analyzing it using a machine learning method, means for predicting future traffic conditions based on past traffic data, and means for analyzing the user's emotional information and providing personalized traffic information. This makes it possible to provide real-time traffic information that takes into account the user's emotional state, thereby realizing comfortable and safe travel.

[0347] "Data collection means" is a general term for devices and sensors used to acquire image information and environmental data related to traffic conditions.

[0348] "Machine learning methods" refer to algorithms and models that allow computers to improve their task performance based on experience, and are particularly used in traffic analysis for recognizing image and audio data.

[0349] "Analysis means" refers to software or algorithms that process received data to identify current traffic flow and the emotional state of users.

[0350] "Predictive methods" refer to algorithms and processing methods used to predict future traffic conditions based on past data.

[0351] "User emotional information" refers to data on the user's psychological state obtained from their facial expressions and voice, and is used to quantify stress levels, satisfaction levels, and other factors.

[0352] "Personalized traffic information" refers to the provision of travel routes and traffic information that are customized based on the user's emotions and current traffic conditions.

[0353] "Means of providing" refers to methods and devices for transmitting specific information to users, particularly system configurations for providing personalized information in real time.

[0354] This invention is a system for optimizing traffic conditions and providing personalized information. The system aims to improve user satisfaction through the coordinated operation of the server, terminal, and user units.

[0355] The server receives image information in real time from traffic cameras and sensors through data collection methods. The received data is processed using image processing libraries such as OpenCV. Furthermore, TensorFlow is used as a machine learning method to analyze the data and understand traffic flow and congestion levels.

[0356] The server uses LSTM models and other tools to predict future traffic conditions based on past traffic data. This prediction enables the control of traffic signals and the optimization of routes.

[0357] In addition, the server activates an emotion engine to analyze facial and voice data obtained from the user's device. This allows the system to identify the user's stress and satisfaction levels, enabling personalized information delivery.

[0358] The terminal receives control instructions from the server and sends instructions to traffic control devices such as traffic lights. Furthermore, the terminal utilizes the Google Maps API and other tools to deliver personalized traffic information to the user's device. As a result, users are offered routes that avoid congestion and comfortable travel paths.

[0359] Users receive real-time traffic information provided by their devices through their smart devices. This reduces the stress of travel and allows users to have a safe and comfortable experience.

[0360] For example, if congestion is predicted during rush hour, the server uses a prediction algorithm to adjust the timing of traffic signals to smooth traffic flow. Simultaneously, the emotion engine detects the user's stress level, and the terminal notifies the user with suggestions to encourage the use of public transportation. In this way, it becomes possible to provide information that takes into account both traffic conditions and the user's psychological state.

[0361] Examples of prompts generated using AI models include, "Please tell me how to provide traffic information that takes the user's emotions into consideration." This enables a variety of approaches to improving the user's travel experience.

[0362] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0363] Step 1:

[0364] The server acquires image information in real time from traffic cameras and sensors via data collection methods. The input image information is preprocessed using image processing libraries such as OpenCV. Specifically, this includes image normalization and vehicle detection. The output is image data prepared for analysis.

[0365] Step 2:

[0366] The server uses TensorFlow to analyze image data with a machine learning model to understand traffic flow and vehicle density. The input here is the output data from step 1, and the number and location of traffic participants are identified through the analysis. The output is numerical data indicating traffic conditions and information on traffic patterns.

[0367] Step 3:

[0368] The server uses an LSTM model to combine historical traffic data with current data from step 2 to predict future traffic conditions. The input consists of historical traffic data and current traffic data. Through data calculations, the prediction algorithm estimates future congestion and delays. The output is the predicted traffic condition for the following time period.

[0369] Step 4:

[0370] The server acquires user emotional information and analyzes it using an emotion engine. Inputs include audio and facial expression data from the user's device. The analysis results quantify the user's stress level and satisfaction level. The output is numerical information about the user's emotional state.

[0371] Step 5:

[0372] The server generates instructions for adjusting traffic signals and providing route guidance based on the analysis results and predictive information. It then executes a control algorithm using the outputs from steps 2, 3, and 4 as input. Specifically, it generates instructions to adjust the timing of traffic signals to alleviate congestion. The output consists of instructions related to traffic control.

[0373] Step 6:

[0374] The terminal receives control instructions from the server and transmits them to the actual traffic control device. The input is the output information from step 5. Specific actions include switching signals and updating traffic information. The output is the traffic environment resulting from the execution of the control instructions.

[0375] Step 7:

[0376] The device provides users with personalized traffic information. Inputs include traffic data and sentiment analysis results from a server. Using this information, and leveraging the Google Maps API, the device notifies users of optimal routes and travel advice tailored to their device. Output is customized information displayed on the user's device.

[0377] Step 8:

[0378] The user makes a move based on information received from the device. The input is the information provided in step 7. For example, a specific action would be to select a route that avoids congestion and move accordingly. The output is an improvement in the user's travel experience.

[0379] (Application Example 2)

[0380] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0381] While understanding and managing traffic conditions has been done with conventional technologies, providing information that takes into account the emotional state of users has been insufficient. In particular, with the spread of autonomous vehicles, there is a need for technologies that reduce user stress and anxiety and provide safer and more comfortable travel. This invention aims to solve these problems and realize personalized traffic information provision that responds to the emotions of users.

[0382] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0383] In this invention, the server includes means for receiving video information acquired from a data acquisition device, means for analyzing the emotional state, and means for providing emotionally-based routes to the user via an in-vehicle device. This makes it possible to travel in a way that takes the user's emotional state into consideration and reduces stress and anxiety.

[0384] A "data acquisition device" is a device that collects video information necessary to recognize traffic conditions.

[0385] A "machine learning model" uses statistical methods to analyze patterns in traffic conditions based on large amounts of data and predict future situations.

[0386] "Analysis means" refers to methods for identifying the characteristics and circumstances of traffic participants based on acquired video information.

[0387] A "predictive tool" is a means of predicting future traffic conditions based on past traffic data and current conditions.

[0388] "Instruction generation means" refers to means for generating instructions for traffic control based on predictive information and analysis results.

[0389] "Control means" refers to means that receive generated instructions and transmit appropriate commands to the traffic control device.

[0390] "Recording means" refers to means for storing data and making it available for long-term analysis.

[0391] "Means of provision" refers to the means of providing recorded information or analysis results to a user or some system.

[0392] An "emotion recognition tool" is a means of providing information tailored to individual users by analyzing their emotional state.

[0393] An "in-vehicle device" is a device installed inside a vehicle that functions as an interface with the user and displays traffic information and route suggestions.

[0394] To implement this invention, a system for optimizing traffic conditions is constructed. First, the server receives video information from a data acquisition device and analyzes this information using a machine learning model. Based on the analyzed data, it predicts traffic flow and the occurrence of congestion. Based on this, the server generates instructions for traffic lights and other traffic control devices and transmits them through the control means.

[0395] Furthermore, the server analyzes the user's emotional state using emotion recognition tools. This process involves acquiring facial and voice data using cameras and microphones, and utilizing an emotion analysis engine (for example, a common cloud-based emotion analysis API). The analyzed emotion information forms the basis for the server to provide personalized traffic information to each user.

[0396] The terminal displays information sent from the server to the user in real time via an in-vehicle device. This information includes route suggestions that take the user's emotions into consideration. The in-vehicle interface is designed to be operable via touch panel and voice recognition.

[0397] Users can receive information on their smartphones or in-vehicle devices and travel comfortably by utilizing emotionally sensitive options. For example, the system suggests routes that avoid congestion to users experiencing high stress levels. This is particularly useful for users commuting or traveling.

[0398] An example of a prompt to a generative AI model is, "The user is planning a holiday drive; please suggest a relaxing route with beautiful scenery." This prompt forms the basis for providing information that matches the user's desired experience.

[0399] In this way, the present invention can be implemented as a system in which the server, terminal, and user each play their own unique roles and work together to optimize traffic conditions and improve the user experience.

[0400] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0401] Step 1:

[0402] The server receives video information acquired from data acquisition devices. Video data from cameras installed on roads is used as input. The server collects this data and stores it in a database for analysis in a digital format. The output is data ready for the next analysis step.

[0403] Step 2:

[0404] The server analyzes video information using a machine learning model. The input is the video data organized in Step 1. Based on this data, the server recognizes the number, speed, and location of traffic participants. This analysis is performed to identify traffic flow and potential congestion information, and the output generates information on the current state of traffic.

[0405] Step 3:

[0406] The server predicts future traffic conditions based on historical traffic data. Input includes not only current traffic information but also historical data. The server analyzes this data using statistical algorithms. The output is a prediction of traffic conditions for the near future.

[0407] Step 4:

[0408] The server generates instructions based on the analysis results and predictive information. The traffic conditions and predictive information obtained up to step 3 are used as input. Based on this, the server formulates specific traffic control instructions, such as adjusting traffic light timings and suggesting alternative routes. Specific control commands are generated as output.

[0409] Step 5:

[0410] The server collects and analyzes data to recognize the user's emotional state. Data from the user's smartphone or cameras and microphones installed in the vehicle are used as input. An emotion analysis engine extracts emotional information from the user's facial expressions and voice. This process identifies the user's emotional state, and that information is provided as output.

[0411] Step 6:

[0412] The server provides customized traffic information based on emotional data. It considers emotional states and traffic data as input to generate optimal routes and traffic-related suggestions for the user. Specifically, it can suggest routes with relaxing scenery, for example. The output is customized traffic information displayed on the user's device.

[0413] Step 7:

[0414] The terminal displays control instructions and emotion-sensitive suggestions from the server on the in-vehicle device. Inputs are control instructions and customized traffic information sent from the server. The terminal notifies the user of these and helps them take appropriate action. Outputs include information to guide the user's actions, provided through the in-vehicle display and audio output.

[0415] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0416] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0417] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0418] [Third Embodiment]

[0419] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0420] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0421] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0422] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0423] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0424] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0425] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0426] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0427] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0428] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0429] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0430] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0431] This invention is a system for analyzing traffic conditions in real time and optimizing traffic flow. This system receives video information from data acquisition devices installed on the streets and uses machine learning models to analyze it, thereby identifying fluctuations in traffic volume and identifying traffic participants.

[0432] The server plays a central role in this system, receiving video information transmitted from data acquisition devices. The received data is input into a machine learning model, which analyzes information such as the number, speed, and location of vehicles in real time. Based on this analysis, past traffic data is used to predict future traffic conditions. Furthermore, the server uses the results of the analysis and prediction to generate control instructions, including changes to traffic signals, for traffic control devices such as traffic lights. These control instructions aim to smooth traffic flow and reduce the risk of congestion and traffic accidents.

[0433] The terminal's role is to execute control instructions received from the server. Specifically, it sends instructions to traffic control devices to adjust traffic signal timings and operate traffic guidance systems. This operation includes not only adjusting traffic signals but also changing the content of road signs and information boards as needed.

[0434] The user is the ultimate beneficiary of the information. The server provides analyzed traffic information and future predictions to the user's device. Based on this information, the user can choose the optimal route and reduce travel time. For example, it can help the user change their departure time to avoid peak hours. Real-time warnings such as accidents and congestion are also sent to the user, helping them to travel more safely.

[0435] This system automates a series of processes, from data collection and analysis to communication and instruction execution, thereby achieving efficient traffic management and contributing to the smooth operation of urban traffic. For example, if sudden congestion occurs at a particular intersection, the server immediately detects the change and adjusts the timing of traffic lights to improve traffic flow. This adjustment predicts an increase in vehicles using alternative routes and sends appropriate instructions to other intersections. As a result, larger-scale traffic route optimization is achieved.

[0436] The following describes the processing flow.

[0437] Step 1:

[0438] The server receives video information from data acquisition devices such as street cameras. This video information includes road conditions and the movements of traffic participants.

[0439] Step 2:

[0440] The server inputs the received video information into a machine learning model and performs analysis in real time. This analysis identifies the number, speed, and location of vehicles, and helps to understand traffic flow.

[0441] Step 3:

[0442] The server references historical traffic data and predicts future traffic conditions based on the analysis results. This prediction takes into account patterns specific to certain times of day and days of the week.

[0443] Step 4:

[0444] The server generates control instructions for traffic lights and traffic signs based on the analysis and prediction results. This includes adjusting the color and timing of the signals.

[0445] Step 5:

[0446] The terminal receives control instructions from the server and executes those instructions on the target traffic control device. This adjusts the timing of traffic lights and smooths the flow of traffic.

[0447] Step 6:

[0448] The server records analysis results and predictive data in a database for long-term analysis. This data can be used for future urban planning and traffic management.

[0449] Step 7:

[0450] Users receive real-time traffic information from a server via their devices. Based on this information, users can select the optimal route and use it to plan their travel.

[0451] (Example 1)

[0452] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0453] In urban areas, optimizing traffic flow and reducing congestion is a critical challenge. Traditional methods often fail to fully utilize real-time traffic information, resulting in inefficient traffic control and consequently increased travel times and accident risks. Therefore, a system is needed that can analyze traffic conditions more accurately and quickly to achieve optimal traffic control.

[0454] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0455] In this invention, the server includes an analysis means that receives visual information acquired from a data collection device and processes it using machine learning techniques, a prediction means that predicts future movement conditions based on past movement data, and a command generation means that creates commands based on the analysis results and prediction information. This enables the optimization of traffic flow within the city by responding to changes in traffic conditions in real time and efficiently adjusting traffic signals and guidance displays, thereby mitigating congestion and improving safety.

[0456] A "data collection device" is a device installed at various locations in urban areas to acquire visual information and plays a role in monitoring traffic conditions.

[0457] "Visual information" refers to video and image data acquired through cameras and sensors, and represents the real-time situation of traffic.

[0458] "Machine learning techniques" are technologies that use algorithms to automatically extract features from large amounts of data and perform analysis and predictions.

[0459] "Analysis means" refers to a mechanism that uses machine learning techniques to process visual information and identify the quantity, speed, and location of traffic participants.

[0460] A "predictive means" is a mechanism for predicting future traffic conditions based on past travel data, and provides predictions to improve traffic flow.

[0461] "Command generation means" refers to a mechanism for creating commands for traffic control devices based on analysis results and predictive information.

[0462] "Mobility control devices" refer to devices and systems necessary for managing and controlling the flow of traffic, such as traffic signals and information displays.

[0463] "Recording means" refers to a mechanism for recording analysis results and predictive information and conducting long-term analysis.

[0464] The "supply means" refers to a mechanism for providing recorded analysis results to users, enabling real-time sharing of traffic information.

[0465] This invention is a system for analyzing and optimizing traffic flow in real time, and utilizes information from data collection devices to achieve more efficient traffic control.

[0466] The server plays a central role in the system. It receives visual information from data collection devices located on-site and uses this information to perform analysis using machine learning techniques. Specifically, it uses software such as Python and TensorFlow to analyze the acquired video data. This allows for the extraction of information such as the number, speed, and location of vehicles in real time, enabling a detailed understanding of traffic conditions.

[0467] The server then uses a generative AI model, referencing historical travel data, to predict future traffic trends. During this process, prompts may be used with the generative AI model. For example, providing specific instructions such as "Show the expected traffic volume fluctuation pattern for the next 30 minutes" can improve prediction accuracy.

[0468] Based on the analysis and prediction results, the server generates commands for traffic control systems. These commands include adjusting traffic light timing and updating road signs. This allows the server to optimize urban traffic flow and provide a safer traffic environment aimed at reducing congestion and preventing accidents.

[0469] The terminal receives commands from the server and puts them into action. Specifically, it adjusts the timing and updates the display content of traffic lights and information boards, which are the targets of its control, according to the commands from the server. In this way, the terminal controls and supports the optimization of traffic flow in real time.

[0470] Users can utilize traffic information analyzed and predicted by the server. Through their devices, users can receive real-time traffic information and use it to select the optimal travel route. For example, users can receive congestion information and change their planned route to reduce travel time. Furthermore, they can receive priority warnings in the event of accidents or other emergencies, supporting safer travel.

[0471] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0472] Step 1:

[0473] The server receives visual information in real time from data acquisition devices. The input consists of video data from cameras placed at multiple intersections and major roads. Upon receiving this video data, the server converts it to a specific format, stores it temporarily, and prepares it for analysis.

[0474] Step 2:

[0475] The server analyzes the received video data using machine learning techniques. The input is the video data temporarily stored in Step 1. The server uses software such as Python and TensorFlow to analyze this data and extract the number, speed, and location of traffic participants. The output is the extracted real-time traffic information.

[0476] Step 3:

[0477] The server uses a generative AI model to predict future traffic conditions based on the analyzed traffic information. This process takes historical travel data and current traffic information as input, and uses prompts to provide specific prediction instructions to the generative AI model. For example, it might use a prompt like, "Show the expected traffic light waiting time for the next hour." The output is predicted traffic fluctuation data.

[0478] Step 4:

[0479] The server generates commands for the traffic control system based on the analysis results and predictive information. The input is the data obtained from steps 2 and 3. The server generates specific commands, such as adjusting the sequence of traffic signals or updating information boards, and prepares to send them to the terminal. The output is the generated command data.

[0480] Step 5:

[0481] The terminal receives commands sent from the server and executes them. The input is the command data generated in step 4. The terminal plays a role in controlling the actual flow of traffic by making specific adjustments to various traffic control devices. The output is an update of the operating status of the control devices.

[0482] Step 6:

[0483] The user receives and utilizes traffic information analyzed and predicted by the server. The input is real-time traffic information and prediction data provided by the server. The user uses this information to select the optimal travel route. They can also adjust departure times as needed to reduce travel time. The output is the optimization of the user's travel plan.

[0484] (Application Example 1)

[0485] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0486] Urban traffic experiences significant fluctuations depending on the time of day, resulting in daily congestion and traffic accidents. This increases commuting and school travel times, and also hinders the rapid response of emergency vehicles to the scene. Furthermore, it is difficult to effectively predict future traffic conditions and provide users with useful information in real time. This invention aims to solve these problems, smooth traffic flow, and provide users with optimal travel routes and times.

[0487] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0488] In this invention, the server includes means for receiving video information acquired from a data acquisition device and analyzing it using a machine learning model, means for predicting future traffic conditions based on past traffic data, and means for generating control instructions based on the analysis results and predicted information and transmitting the information to the user terminal. This enables real-time analysis and prediction of traffic conditions, as well as personalized route suggestions and the presentation of optimal travel times.

[0489] A "data acquisition device" is a device installed on streets and at various transportation hubs to acquire video information for understanding traffic conditions.

[0490] A "machine learning model" is a learning system equipped with algorithms used to analyze traffic conditions and vehicle movements.

[0491] "Analysis means" refers to a method of analyzing video information received from a data acquisition device using a machine learning model to identify the number, speed, and location of traffic participants.

[0492] A "predictive method" is a process for predicting future traffic conditions based on past traffic data.

[0493] The "instruction generation means" is a function that generates control instructions for traffic control devices and users based on analysis results and predictive information.

[0494] A "traffic control system" is a device that controls the operation of traffic lights and road signs, and adjusts the flow of traffic.

[0495] "Recording methods" refer to methods for long-term storage and analysis of analysis results and predictive information to be used for future traffic improvements.

[0496] "Provisioning means" refers to the process of providing the analysis results from the recording means to the user's information processing device, enabling personalized route suggestions.

[0497] A "user terminal" is a computer device that receives analyzed traffic information and provides it to users in real time.

[0498] Regarding the embodiment for carrying out the invention, this system analyzes real-time traffic conditions and utilizes that information to perform traffic control and provide optimal travel suggestions to users. This system mainly consists of three components: a server, a terminal, and a user.

[0499] The server receives video information acquired from data acquisition devices installed on the streets. This video information is captured using Python's OpenCV. Next, the server uses a machine learning model employing TensorFlow to analyze the number, speed, and location of traffic participants in real time. The analyzed data is stored in the backend using the Django framework and used by a prediction algorithm to estimate future traffic conditions. Furthermore, the server generates instructions based on these analysis results and sends them to traffic control devices and users.

[0500] The terminal makes appropriate adjustments to traffic lights and signs based on control instructions sent from the server. Furthermore, analyzed traffic information is provided to the user's mobile device in real time through an application developed with React Native. This application suggests optimal routes and departure times to the user and provides recommendations to avoid congestion.

[0501] Users can utilize the information provided through their mobile devices to create optimized travel plans. For example, this could allow users to adjust their departure time based on traffic warnings provided by the app, thereby shortening their commute time.

[0502] By utilizing a generative AI model, it is also possible to suggest the most appropriate prompt message based on the current traffic conditions. An example of a prompt message would be: "Traffic volume is increasing at the current intersection. Please generate the best prompt to suggest a safe and smooth alternative route to the app user."

[0503] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0504] Step 1:

[0505] The server receives video information streamed from the data acquisition device. The received data is raw video to understand traffic flow in real time. Specifically, the OpenCV library in Python is used to capture the feed from the camera and prepare each frame for analysis.

[0506] Step 2:

[0507] The server uses TensorFlow to input video data into a machine learning model and analyze the number, speed, and location of vehicles. First, the received frames are preprocessed and converted into a format suitable for the model (e.g., resizing and normalization). Then, the data is fed into the model, and vehicle detection and tracking are performed to determine the number, location, and speed of vehicles.

[0508] Step 3:

[0509] The server uses the Django framework to store the analysis results in a database and combines them with historical traffic data to predict future traffic conditions. Specifically, it applies a time-series model using current traffic data to instantly calculate future traffic patterns and predict the occurrence and mitigation of congestion.

[0510] Step 4:

[0511] The server uses a generative AI model based on analysis results and predictive information to create prompt messages and send them to traffic control devices and user terminals. Instruction generation includes early warnings of predicted congestion points and suggestions for adjusting traffic light timings. This step leverages the generative AI model to create automated prompts and suggest optimal travel routes and times for users.

[0512] Step 5:

[0513] The terminal receives instructions from the server and performs specific traffic control actions. This includes, for example, shortening or extending the phase of traffic lights or dynamically updating traffic signs. Furthermore, the user's information processing device is provided with real-time suggestions for the optimal route and departure time through the application.

[0514] Step 6:

[0515] The user adjusts their travel plan using information provided by their device. For example, based on a provided prompt message such as, "Traffic is increasing at the current intersection. Generate the best prompt to suggest a safe and smooth alternative route to the app user," the system facilitates the selection of a smoother route and avoidance of traffic congestion.

[0516] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0517] This invention aims to provide optimal traffic information to users by incorporating an emotion engine that recognizes the emotional state of users into a system for improving traffic conditions in real time. In addition to basic traffic management functions such as receiving, analyzing, predicting, and controlling video information from data acquisition devices, this system also provides information tailored to individual users by analyzing the emotions of the users.

[0518] After receiving video information, the server performs analysis using a machine learning model to understand traffic flow. It then combines this with historical data to predict future traffic conditions and adjusts traffic signals based on the analysis results and predictions. Furthermore, the server activates an emotion engine to receive emotional information from the user's device. This emotion engine analyzes the user's facial expressions and voice data to identify emotional states such as stress, dissatisfaction, and satisfaction.

[0519] The terminal receives control instructions from the server and sends those instructions to the actual traffic control system. This process guides traffic flow appropriately and alleviates congestion. Furthermore, based on the analysis results of the emotion engine, the terminal provides traffic information and route suggestions that take the user's emotions into consideration. For example, for a user experiencing high stress levels, it can suggest routes that avoid congestion or quieter paths.

[0520] For users, real-time traffic information provided by the server can be received via devices such as smartphones. By receiving customized information tailored to the user's emotional state, a more comfortable and safer journey becomes possible. This system aims to enhance user satisfaction with their travel experiences.

[0521] As a concrete example, if congestion is predicted during rush hour, the server will grasp the situation early and adjust traffic signals accordingly. At the same time, it will detect increased user stress through an emotion engine. Based on this information, the terminal will suggest measures to reduce stress, such as recommending the use of public transportation or alternative departure times. In this way, the system optimizes traffic flow while providing information that takes individual emotions into consideration.

[0522] The following describes the processing flow.

[0523] Step 1:

[0524] The server receives video information transmitted from the data acquisition device. This includes camera data used to understand road conditions, among other things.

[0525] Step 2:

[0526] The server inputs the received video information into a machine learning model and performs analysis in real time. This analysis allows for the identification of the number, speed, and location of vehicles, and enables the understanding of traffic flow.

[0527] Step 3:

[0528] The server uses current analysis results based on past traffic data to predict future traffic conditions. These predictions are used to adjust traffic signals and avoid congestion.

[0529] Step 4:

[0530] The server receives emotional information from the user's device and uses an emotion engine to evaluate the user's emotional state. This is done to understand stress levels, satisfaction levels, and other factors through facial expression analysis and voice data analysis.

[0531] Step 5:

[0532] The server generates control instructions to send to the traffic control system based on analysis results, predictive information, and the user's emotional state. These instructions include changes to traffic signals and adjustments to route guidance.

[0533] Step 6:

[0534] The terminal transmits control instructions received from the server to the traffic control system, which then operates traffic lights and information boards. This optimizes the flow of traffic.

[0535] Step 7:

[0536] The server selects information based on the user's emotional state and sends it to the user's device in order to provide the user with suitable routes and traffic information in real time. This information makes the user's journey more comfortable.

[0537] Step 8:

[0538] Based on the customized traffic information they receive, users can adjust their travel plans to reach their destination more efficiently and safely.

[0539] (Example 2)

[0540] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0541] While conventional traffic management systems have the capability to analyze and control traffic flow and congestion in real time, they have limitations in providing information that takes into account the individual emotional state of users. As a result, they cannot adequately reduce user stress and dissatisfaction, making it difficult to improve satisfaction with travel.

[0542] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0543] In this invention, the server includes means for receiving image information acquired from data collection means and analyzing it using a machine learning method, means for predicting future traffic conditions based on past traffic data, and means for analyzing the user's emotional information and providing personalized traffic information. This makes it possible to provide real-time traffic information that takes into account the user's emotional state, thereby realizing comfortable and safe travel.

[0544] "Data collection means" is a general term for devices and sensors used to acquire image information and environmental data related to traffic conditions.

[0545] "Machine learning methods" refer to algorithms and models that allow computers to improve their task performance based on experience, and are particularly used in traffic analysis for recognizing image and audio data.

[0546] "Analysis means" refers to software or algorithms that process received data to identify current traffic flow and the emotional state of users.

[0547] "Predictive methods" refer to algorithms and processing methods used to predict future traffic conditions based on past data.

[0548] "User emotional information" refers to data on the user's psychological state obtained from their facial expressions and voice, and is used to quantify stress levels, satisfaction levels, and other factors.

[0549] "Personalized traffic information" refers to the provision of travel routes and traffic information that are customized based on the user's emotions and current traffic conditions.

[0550] "Means of providing" refers to methods and devices for transmitting specific information to users, particularly system configurations for providing personalized information in real time.

[0551] This invention is a system for optimizing traffic conditions and providing personalized information. The system aims to improve user satisfaction through the coordinated operation of the server, terminal, and user units.

[0552] The server receives image information in real time from traffic cameras and sensors through data collection methods. The received data is processed using image processing libraries such as OpenCV. Furthermore, TensorFlow is used as a machine learning method to analyze the data and understand traffic flow and congestion levels.

[0553] The server uses LSTM models and other tools to predict future traffic conditions based on past traffic data. This prediction enables the control of traffic signals and the optimization of routes.

[0554] In addition, the server activates an emotion engine to analyze facial and voice data obtained from the user's device. This allows the system to identify the user's stress and satisfaction levels, enabling personalized information delivery.

[0555] The terminal receives control instructions from the server and sends instructions to traffic control devices such as traffic lights. Furthermore, the terminal utilizes the Google Maps API and other tools to deliver personalized traffic information to the user's device. As a result, users are offered routes that avoid congestion and comfortable travel paths.

[0556] Users receive real-time traffic information provided by their devices through their smart devices. This reduces the stress of travel and allows users to have a safe and comfortable experience.

[0557] For example, if congestion is predicted during rush hour, the server uses a prediction algorithm to adjust the timing of traffic signals to smooth traffic flow. Simultaneously, the emotion engine detects the user's stress level, and the terminal notifies the user with suggestions to encourage the use of public transportation. In this way, it becomes possible to provide information that takes into account both traffic conditions and the user's psychological state.

[0558] Examples of prompts generated using AI models include, "Please tell me how to provide traffic information that takes the user's emotions into consideration." This enables a variety of approaches to improving the user's travel experience.

[0559] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0560] Step 1:

[0561] The server acquires image information in real time from traffic cameras and sensors via data collection methods. The input image information is preprocessed using image processing libraries such as OpenCV. Specifically, this includes image normalization and vehicle detection. The output is image data prepared for analysis.

[0562] Step 2:

[0563] The server uses TensorFlow to analyze image data with a machine learning model to understand traffic flow and vehicle density. The input here is the output data from step 1, and the number and location of traffic participants are identified through the analysis. The output is numerical data indicating traffic conditions and information on traffic patterns.

[0564] Step 3:

[0565] The server uses an LSTM model to combine historical traffic data with current data from step 2 to predict future traffic conditions. The input consists of historical traffic data and current traffic data. Through data calculations, the prediction algorithm estimates future congestion and delays. The output is the predicted traffic condition for the following time period.

[0566] Step 4:

[0567] The server acquires user emotional information and analyzes it using an emotion engine. Inputs include audio and facial expression data from the user's device. The analysis results quantify the user's stress level and satisfaction level. The output is numerical information about the user's emotional state.

[0568] Step 5:

[0569] The server generates instructions for adjusting traffic signals and providing route guidance based on the analysis results and predictive information. It then executes a control algorithm using the outputs from steps 2, 3, and 4 as input. Specifically, it generates instructions to adjust the timing of traffic signals to alleviate congestion. The output consists of instructions related to traffic control.

[0570] Step 6:

[0571] The terminal receives control instructions from the server and transmits them to the actual traffic control device. The input is the output information from step 5. Specific actions include switching signals and updating traffic information. The output is the traffic environment resulting from the execution of the control instructions.

[0572] Step 7:

[0573] The device provides users with personalized traffic information. Inputs include traffic data and sentiment analysis results from a server. Using this information, and leveraging the Google Maps API, the device notifies users of optimal routes and travel advice tailored to their device. Output is customized information displayed on the user's device.

[0574] Step 8:

[0575] The user makes a move based on information received from the device. The input is the information provided in step 7. For example, a specific action would be to select a route that avoids congestion and move accordingly. The output is an improvement in the user's travel experience.

[0576] (Application Example 2)

[0577] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0578] While understanding and managing traffic conditions has been done with conventional technologies, providing information that takes into account the emotional state of users has been insufficient. In particular, with the spread of autonomous vehicles, there is a need for technologies that reduce user stress and anxiety and provide safer and more comfortable travel. This invention aims to solve these problems and realize personalized traffic information provision that responds to the emotions of users.

[0579] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0580] In this invention, the server includes means for receiving video information acquired from a data acquisition device, means for analyzing the emotional state, and means for providing emotionally-based routes to the user via an in-vehicle device. This makes it possible to travel in a way that takes the user's emotional state into consideration and reduces stress and anxiety.

[0581] A "data acquisition device" is a device that collects video information necessary to recognize traffic conditions.

[0582] A "machine learning model" uses statistical methods to analyze patterns in traffic conditions based on large amounts of data and predict future situations.

[0583] "Analysis means" refers to methods for identifying the characteristics and circumstances of traffic participants based on acquired video information.

[0584] A "predictive tool" is a means of predicting future traffic conditions based on past traffic data and current conditions.

[0585] "Instruction generation means" refers to means for generating instructions for traffic control based on predictive information and analysis results.

[0586] "Control means" refers to means that receive generated instructions and transmit appropriate commands to the traffic control device.

[0587] "Recording means" refers to means for storing data and making it available for long-term analysis.

[0588] "Means of provision" refers to the means of providing recorded information or analysis results to a user or some system.

[0589] An "emotion recognition tool" is a means of providing information tailored to individual users by analyzing their emotional state.

[0590] An "in-vehicle device" is a device installed inside a vehicle that functions as an interface with the user and displays traffic information and route suggestions.

[0591] To implement this invention, a system for optimizing traffic conditions is constructed. First, the server receives video information from a data acquisition device and analyzes this information using a machine learning model. Based on the analyzed data, it predicts traffic flow and the occurrence of congestion. Based on this, the server generates instructions for traffic lights and other traffic control devices and transmits them through the control means.

[0592] Furthermore, the server analyzes the user's emotional state using emotion recognition tools. This process involves acquiring facial and voice data using cameras and microphones, and utilizing an emotion analysis engine (for example, a common cloud-based emotion analysis API). The analyzed emotion information forms the basis for the server to provide personalized traffic information to each user.

[0593] The terminal displays information sent from the server to the user in real time via an in-vehicle device. This information includes route suggestions that take the user's emotions into consideration. The in-vehicle interface is designed to be operable via touch panel and voice recognition.

[0594] Users can receive information on their smartphones or in-vehicle devices and travel comfortably by utilizing emotionally sensitive options. For example, the system suggests routes that avoid congestion to users experiencing high stress levels. This is particularly useful for users commuting or traveling.

[0595] An example of a prompt to a generative AI model is, "The user is planning a holiday drive; please suggest a relaxing route with beautiful scenery." This prompt forms the basis for providing information that matches the user's desired experience.

[0596] In this way, the present invention can be implemented as a system in which the server, terminal, and user each play their own unique roles and work together to optimize traffic conditions and improve the user experience.

[0597] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0598] Step 1:

[0599] The server receives video information acquired from data acquisition devices. Video data from cameras installed on roads is used as input. The server collects this data and stores it in a database for analysis in a digital format. The output is data ready for the next analysis step.

[0600] Step 2:

[0601] The server analyzes video information using a machine learning model. The input is the video data organized in Step 1. Based on this data, the server recognizes the number, speed, and location of traffic participants. This analysis is performed to identify traffic flow and potential congestion information, and the output generates information on the current state of traffic.

[0602] Step 3:

[0603] The server predicts future traffic conditions based on historical traffic data. Input includes not only current traffic information but also historical data. The server analyzes this data using statistical algorithms. The output is a prediction of traffic conditions for the near future.

[0604] Step 4:

[0605] The server generates instructions based on the analysis results and predictive information. The traffic conditions and predictive information obtained up to step 3 are used as input. Based on this, the server formulates specific traffic control instructions, such as adjusting traffic light timings and suggesting alternative routes. Specific control commands are generated as output.

[0606] Step 5:

[0607] The server collects and analyzes data to recognize the user's emotional state. Data from the user's smartphone or cameras and microphones installed in the vehicle are used as input. An emotion analysis engine extracts emotional information from the user's facial expressions and voice. This process identifies the user's emotional state, and that information is provided as output.

[0608] Step 6:

[0609] The server provides customized traffic information based on emotional data. It considers emotional states and traffic data as input to generate optimal routes and traffic-related suggestions for the user. Specifically, it can suggest routes with relaxing scenery, for example. The output is customized traffic information displayed on the user's device.

[0610] Step 7:

[0611] The terminal displays control instructions and emotion-sensitive suggestions from the server on the in-vehicle device. Inputs are control instructions and customized traffic information sent from the server. The terminal notifies the user of these and helps them take appropriate action. Outputs include information to guide the user's actions, provided through the in-vehicle display and audio output.

[0612] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0613] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0614] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0615] [Fourth Embodiment]

[0616] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0617] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0618] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0619] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0620] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0621] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0622] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0623] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0624] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0625] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0626] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0627] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0628] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0629] This invention is a system for analyzing traffic conditions in real time and optimizing traffic flow. This system receives video information from data acquisition devices installed on the streets and uses machine learning models to analyze it, thereby identifying fluctuations in traffic volume and identifying traffic participants.

[0630] The server plays a central role in this system, receiving video information transmitted from data acquisition devices. The received data is input into a machine learning model, which analyzes information such as the number, speed, and location of vehicles in real time. Based on this analysis, past traffic data is used to predict future traffic conditions. Furthermore, the server uses the results of the analysis and prediction to generate control instructions, including changes to traffic signals, for traffic control devices such as traffic lights. These control instructions aim to smooth traffic flow and reduce the risk of congestion and traffic accidents.

[0631] The terminal's role is to execute control instructions received from the server. Specifically, it sends instructions to traffic control devices to adjust traffic signal timings and operate traffic guidance systems. This operation includes not only adjusting traffic signals but also changing the content of road signs and information boards as needed.

[0632] The user is the ultimate beneficiary of the information. The server provides analyzed traffic information and future predictions to the user's device. Based on this information, the user can choose the optimal route and reduce travel time. For example, it can help the user change their departure time to avoid peak hours. Real-time warnings such as accidents and congestion are also sent to the user, helping them to travel more safely.

[0633] This system automates a series of processes, from data collection and analysis to communication and instruction execution, thereby achieving efficient traffic management and contributing to the smooth operation of urban traffic. For example, if sudden congestion occurs at a particular intersection, the server immediately detects the change and adjusts the timing of traffic lights to improve traffic flow. This adjustment predicts an increase in vehicles using alternative routes and sends appropriate instructions to other intersections. As a result, larger-scale traffic route optimization is achieved.

[0634] The following describes the processing flow.

[0635] Step 1:

[0636] The server receives video information from data acquisition devices such as street cameras. This video information includes road conditions and the movements of traffic participants.

[0637] Step 2:

[0638] The server inputs the received video information into a machine learning model and performs analysis in real time. This analysis identifies the number, speed, and location of vehicles, and helps to understand traffic flow.

[0639] Step 3:

[0640] The server references historical traffic data and predicts future traffic conditions based on the analysis results. This prediction takes into account patterns specific to certain times of day and days of the week.

[0641] Step 4:

[0642] The server generates control instructions for traffic lights and traffic signs based on the analysis and prediction results. This includes adjusting the color and timing of the signals.

[0643] Step 5:

[0644] The terminal receives control instructions from the server and executes those instructions on the target traffic control device. This adjusts the timing of traffic lights and smooths the flow of traffic.

[0645] Step 6:

[0646] The server records analysis results and predictive data in a database for long-term analysis. This data can be used for future urban planning and traffic management.

[0647] Step 7:

[0648] Users receive real-time traffic information from a server via their devices. Based on this information, users can select the optimal route and use it to plan their travel.

[0649] (Example 1)

[0650] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0651] In urban areas, optimizing traffic flow and reducing congestion is a critical challenge. Traditional methods often fail to fully utilize real-time traffic information, resulting in inefficient traffic control and consequently increased travel times and accident risks. Therefore, a system is needed that can analyze traffic conditions more accurately and quickly to achieve optimal traffic control.

[0652] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0653] In this invention, the server includes an analysis means that receives visual information acquired from a data collection device and processes it using machine learning techniques, a prediction means that predicts future movement conditions based on past movement data, and a command generation means that creates commands based on the analysis results and prediction information. This enables the optimization of traffic flow within the city by responding to changes in traffic conditions in real time and efficiently adjusting traffic signals and guidance displays, thereby mitigating congestion and improving safety.

[0654] A "data collection device" is a device installed at various locations in urban areas to acquire visual information and plays a role in monitoring traffic conditions.

[0655] "Visual information" refers to video and image data acquired through cameras and sensors, and represents the real-time situation of traffic.

[0656] "Machine learning techniques" are technologies that use algorithms to automatically extract features from large amounts of data and perform analysis and predictions.

[0657] "Analysis means" refers to a mechanism that uses machine learning techniques to process visual information and identify the quantity, speed, and location of traffic participants.

[0658] A "predictive means" is a mechanism for predicting future traffic conditions based on past travel data, and provides predictions to improve traffic flow.

[0659] "Command generation means" refers to a mechanism for creating commands for traffic control devices based on analysis results and predictive information.

[0660] "Mobility control devices" refer to devices and systems necessary for managing and controlling the flow of traffic, such as traffic signals and information displays.

[0661] "Recording means" refers to a mechanism for recording analysis results and predictive information and conducting long-term analysis.

[0662] The "supply means" refers to a mechanism for providing recorded analysis results to users, enabling real-time sharing of traffic information.

[0663] This invention is a system for analyzing and optimizing traffic flow in real time, and utilizes information from data collection devices to achieve more efficient traffic control.

[0664] The server plays a central role in the system. It receives visual information from data collection devices located on-site and uses this information to perform analysis using machine learning techniques. Specifically, it uses software such as Python and TensorFlow to analyze the acquired video data. This allows for the extraction of information such as the number, speed, and location of vehicles in real time, enabling a detailed understanding of traffic conditions.

[0665] The server then uses a generative AI model, referencing historical travel data, to predict future traffic trends. During this process, prompts may be used with the generative AI model. For example, providing specific instructions such as "Show the expected traffic volume fluctuation pattern for the next 30 minutes" can improve prediction accuracy.

[0666] Based on the analysis and prediction results, the server generates commands for traffic control systems. These commands include adjusting traffic light timing and updating road signs. This allows the server to optimize urban traffic flow and provide a safer traffic environment aimed at reducing congestion and preventing accidents.

[0667] The terminal receives commands from the server and puts them into action. Specifically, it adjusts the timing and updates the display content of traffic lights and information boards, which are the targets of its control, according to the commands from the server. In this way, the terminal controls and supports the optimization of traffic flow in real time.

[0668] Users can utilize traffic information analyzed and predicted by the server. Through their devices, users can receive real-time traffic information and use it to select the optimal travel route. For example, users can receive congestion information and change their planned route to reduce travel time. Furthermore, they can receive priority warnings in the event of accidents or other emergencies, supporting safer travel.

[0669] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0670] Step 1:

[0671] The server receives visual information in real time from data acquisition devices. The input consists of video data from cameras placed at multiple intersections and major roads. Upon receiving this video data, the server converts it to a specific format, stores it temporarily, and prepares it for analysis.

[0672] Step 2:

[0673] The server analyzes the received video data using machine learning techniques. The input is the video data temporarily stored in Step 1. The server uses software such as Python and TensorFlow to analyze this data and extract the number, speed, and location of traffic participants. The output is the extracted real-time traffic information.

[0674] Step 3:

[0675] The server uses a generative AI model to predict future traffic conditions based on the analyzed traffic information. This process takes historical travel data and current traffic information as input, and uses prompts to provide specific prediction instructions to the generative AI model. For example, it might use a prompt like, "Show the expected traffic light waiting time for the next hour." The output is predicted traffic fluctuation data.

[0676] Step 4:

[0677] The server generates commands for the traffic control system based on the analysis results and predictive information. The input is the data obtained from steps 2 and 3. The server generates specific commands, such as adjusting the sequence of traffic signals or updating information boards, and prepares to send them to the terminal. The output is the generated command data.

[0678] Step 5:

[0679] The terminal receives commands sent from the server and executes them. The input is the command data generated in step 4. The terminal plays a role in controlling the actual flow of traffic by making specific adjustments to various traffic control devices. The output is an update of the operating status of the control devices.

[0680] Step 6:

[0681] The user receives and utilizes traffic information analyzed and predicted by the server. The input is real-time traffic information and prediction data provided by the server. The user uses this information to select the optimal travel route. They can also adjust departure times as needed to reduce travel time. The output is the optimization of the user's travel plan.

[0682] (Application Example 1)

[0683] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0684] Urban traffic experiences significant fluctuations depending on the time of day, resulting in daily congestion and traffic accidents. This increases commuting and school travel times, and also hinders the rapid response of emergency vehicles to the scene. Furthermore, it is difficult to effectively predict future traffic conditions and provide users with useful information in real time. This invention aims to solve these problems, smooth traffic flow, and provide users with optimal travel routes and times.

[0685] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0686] In this invention, the server includes means for receiving video information acquired from a data acquisition device and analyzing it using a machine learning model, means for predicting future traffic conditions based on past traffic data, and means for generating control instructions based on the analysis results and predicted information and transmitting the information to the user terminal. This enables real-time analysis and prediction of traffic conditions, as well as personalized route suggestions and the presentation of optimal travel times.

[0687] A "data acquisition device" is a device installed on streets and at various transportation hubs to acquire video information for understanding traffic conditions.

[0688] A "machine learning model" is a learning system equipped with algorithms used to analyze traffic conditions and vehicle movements.

[0689] "Analysis means" refers to a method of analyzing video information received from a data acquisition device using a machine learning model to identify the number, speed, and location of traffic participants.

[0690] A "predictive method" is a process for predicting future traffic conditions based on past traffic data.

[0691] The "instruction generation means" is a function that generates control instructions for traffic control devices and users based on analysis results and predictive information.

[0692] A "traffic control system" is a device that controls the operation of traffic lights and road signs, and adjusts the flow of traffic.

[0693] "Recording methods" refer to methods for long-term storage and analysis of analysis results and predictive information to be used for future traffic improvements.

[0694] "Provisioning means" refers to the process of providing the analysis results from the recording means to the user's information processing device, enabling personalized route suggestions.

[0695] A "user terminal" is a computer device that receives analyzed traffic information and provides it to users in real time.

[0696] Regarding the embodiment for carrying out the invention, this system analyzes real-time traffic conditions and utilizes that information to perform traffic control and provide optimal travel suggestions to users. This system mainly consists of three components: a server, a terminal, and a user.

[0697] The server receives video information acquired from data acquisition devices installed on the streets. This video information is captured using Python's OpenCV. Next, the server uses a machine learning model employing TensorFlow to analyze the number, speed, and location of traffic participants in real time. The analyzed data is stored in the backend using the Django framework and used by a prediction algorithm to estimate future traffic conditions. Furthermore, the server generates instructions based on these analysis results and sends them to traffic control devices and users.

[0698] The terminal makes appropriate adjustments to traffic lights and signs based on control instructions sent from the server. Furthermore, analyzed traffic information is provided to the user's mobile device in real time through an application developed with React Native. This application suggests optimal routes and departure times to the user and provides recommendations to avoid congestion.

[0699] Users can utilize the information provided through their mobile devices to create optimized travel plans. For example, this could allow users to adjust their departure time based on traffic warnings provided by the app, thereby shortening their commute time.

[0700] By utilizing a generative AI model, it is also possible to suggest the most appropriate prompt message based on the current traffic conditions. An example of a prompt message would be: "Traffic volume is increasing at the current intersection. Please generate the best prompt to suggest a safe and smooth alternative route to the app user."

[0701] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0702] Step 1:

[0703] The server receives video information streamed from the data acquisition device. The received data is raw video to understand traffic flow in real time. Specifically, the OpenCV library in Python is used to capture the feed from the camera and prepare each frame for analysis.

[0704] Step 2:

[0705] The server uses TensorFlow to input video data into a machine learning model and analyze the number, speed, and location of vehicles. First, the received frames are preprocessed and converted into a format suitable for the model (e.g., resizing and normalization). Then, the data is fed into the model, and vehicle detection and tracking are performed to determine the number, location, and speed of vehicles.

[0706] Step 3:

[0707] The server uses the Django framework to store the analysis results in a database and combines them with historical traffic data to predict future traffic conditions. Specifically, it applies a time-series model using current traffic data to instantly calculate future traffic patterns and predict the occurrence and mitigation of congestion.

[0708] Step 4:

[0709] The server uses a generative AI model based on analysis results and predictive information to create prompt messages and send them to traffic control devices and user terminals. Instruction generation includes early warnings of predicted congestion points and suggestions for adjusting traffic light timings. This step leverages the generative AI model to create automated prompts and suggest optimal travel routes and times for users.

[0710] Step 5:

[0711] The terminal receives instructions from the server and performs specific traffic control actions. This includes, for example, shortening or extending the phase of traffic lights or dynamically updating traffic signs. Furthermore, the user's information processing device is provided with real-time suggestions for the optimal route and departure time through the application.

[0712] Step 6:

[0713] The user adjusts their travel plan using information provided by their device. For example, based on a provided prompt message such as, "Traffic is increasing at the current intersection. Generate the best prompt to suggest a safe and smooth alternative route to the app user," the system facilitates the selection of a smoother route and avoidance of traffic congestion.

[0714] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0715] This invention aims to provide optimal traffic information to users by incorporating an emotion engine that recognizes the emotional state of users into a system for improving traffic conditions in real time. In addition to basic traffic management functions such as receiving, analyzing, predicting, and controlling video information from data acquisition devices, this system also provides information tailored to individual users by analyzing the emotions of the users.

[0716] After receiving video information, the server performs analysis using a machine learning model to understand traffic flow. It then combines this with historical data to predict future traffic conditions and adjusts traffic signals based on the analysis results and predictions. Furthermore, the server activates an emotion engine to receive emotional information from the user's device. This emotion engine analyzes the user's facial expressions and voice data to identify emotional states such as stress, dissatisfaction, and satisfaction.

[0717] The terminal receives control instructions from the server and sends those instructions to the actual traffic control system. This process guides traffic flow appropriately and alleviates congestion. Furthermore, based on the analysis results of the emotion engine, the terminal provides traffic information and route suggestions that take the user's emotions into consideration. For example, for a user experiencing high stress levels, it can suggest routes that avoid congestion or quieter paths.

[0718] For users, real-time traffic information provided by the server can be received via devices such as smartphones. By receiving customized information tailored to the user's emotional state, a more comfortable and safer journey becomes possible. This system aims to enhance user satisfaction with their travel experiences.

[0719] As a concrete example, if congestion is predicted during rush hour, the server will grasp the situation early and adjust traffic signals accordingly. At the same time, it will detect increased user stress through an emotion engine. Based on this information, the terminal will suggest measures to reduce stress, such as recommending the use of public transportation or alternative departure times. In this way, the system optimizes traffic flow while providing information that takes individual emotions into consideration.

[0720] The following describes the processing flow.

[0721] Step 1:

[0722] The server receives video information transmitted from the data acquisition device. This includes camera data used to understand road conditions, among other things.

[0723] Step 2:

[0724] The server inputs the received video information into a machine learning model and performs analysis in real time. This analysis allows for the identification of the number, speed, and location of vehicles, and enables the understanding of traffic flow.

[0725] Step 3:

[0726] The server uses current analysis results based on past traffic data to predict future traffic conditions. These predictions are used to adjust traffic signals and avoid congestion.

[0727] Step 4:

[0728] The server receives emotional information from the user's device and uses an emotion engine to evaluate the user's emotional state. This is done to understand stress levels, satisfaction levels, and other factors through facial expression analysis and voice data analysis.

[0729] Step 5:

[0730] The server generates control instructions to send to the traffic control system based on analysis results, predictive information, and the user's emotional state. These instructions include changes to traffic signals and adjustments to route guidance.

[0731] Step 6:

[0732] The terminal transmits control instructions received from the server to the traffic control system, which then operates traffic lights and information boards. This optimizes the flow of traffic.

[0733] Step 7:

[0734] The server selects information based on the user's emotional state and sends it to the user's device in order to provide the user with suitable routes and traffic information in real time. This information makes the user's journey more comfortable.

[0735] Step 8:

[0736] Based on the customized traffic information they receive, users can adjust their travel plans to reach their destination more efficiently and safely.

[0737] (Example 2)

[0738] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0739] While conventional traffic management systems have the capability to analyze and control traffic flow and congestion in real time, they have limitations in providing information that takes into account the individual emotional state of users. As a result, they cannot adequately reduce user stress and dissatisfaction, making it difficult to improve satisfaction with travel.

[0740] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0741] In this invention, the server includes means for receiving image information acquired from data collection means and analyzing it using a machine learning method, means for predicting future traffic conditions based on past traffic data, and means for analyzing the user's emotional information and providing personalized traffic information. This makes it possible to provide real-time traffic information that takes into account the user's emotional state, thereby realizing comfortable and safe travel.

[0742] "Data collection means" is a general term for devices and sensors used to acquire image information and environmental data related to traffic conditions.

[0743] "Machine learning methods" refer to algorithms and models that allow computers to improve their task performance based on experience, and are particularly used in traffic analysis for recognizing image and audio data.

[0744] "Analysis means" refers to software or algorithms that process received data to identify current traffic flow and the emotional state of users.

[0745] "Predictive methods" refer to algorithms and processing methods used to predict future traffic conditions based on past data.

[0746] "User emotional information" refers to data on the user's psychological state obtained from their facial expressions and voice, and is used to quantify stress levels, satisfaction levels, and other factors.

[0747] "Personalized traffic information" refers to the provision of travel routes and traffic information that are customized based on the user's emotions and current traffic conditions.

[0748] "Means of providing" refers to methods and devices for transmitting specific information to users, particularly system configurations for providing personalized information in real time.

[0749] This invention is a system for optimizing traffic conditions and providing personalized information. The system aims to improve user satisfaction through the coordinated operation of the server, terminal, and user units.

[0750] The server receives image information in real time from traffic cameras and sensors through data collection methods. The received data is processed using image processing libraries such as OpenCV. Furthermore, TensorFlow is used as a machine learning method to analyze the data and understand traffic flow and congestion levels.

[0751] The server uses LSTM models and other tools to predict future traffic conditions based on past traffic data. This prediction enables the control of traffic signals and the optimization of routes.

[0752] In addition, the server activates an emotion engine to analyze facial and voice data obtained from the user's device. This allows the system to identify the user's stress and satisfaction levels, enabling personalized information delivery.

[0753] The terminal receives control instructions from the server and sends instructions to traffic control devices such as traffic lights. Furthermore, the terminal utilizes the Google Maps API and other tools to deliver personalized traffic information to the user's device. As a result, users are offered routes that avoid congestion and comfortable travel paths.

[0754] Users receive real-time traffic information provided by their devices through their smart devices. This reduces the stress of travel and allows users to have a safe and comfortable experience.

[0755] For example, if congestion is predicted during rush hour, the server uses a prediction algorithm to adjust the timing of traffic signals to smooth traffic flow. Simultaneously, the emotion engine detects the user's stress level, and the terminal notifies the user with suggestions to encourage the use of public transportation. In this way, it becomes possible to provide information that takes into account both traffic conditions and the user's psychological state.

[0756] Examples of prompts generated using AI models include, "Please tell me how to provide traffic information that takes the user's emotions into consideration." This enables a variety of approaches to improving the user's travel experience.

[0757] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0758] Step 1:

[0759] The server acquires image information in real time from traffic cameras and sensors via data collection methods. The input image information is preprocessed using image processing libraries such as OpenCV. Specifically, this includes image normalization and vehicle detection. The output is image data prepared for analysis.

[0760] Step 2:

[0761] The server uses TensorFlow to analyze image data with a machine learning model to understand traffic flow and vehicle density. The input here is the output data from step 1, and the number and location of traffic participants are identified through the analysis. The output is numerical data indicating traffic conditions and information on traffic patterns.

[0762] Step 3:

[0763] The server uses an LSTM model to combine historical traffic data with current data from step 2 to predict future traffic conditions. The input consists of historical traffic data and current traffic data. Through data calculations, the prediction algorithm estimates future congestion and delays. The output is the predicted traffic condition for the following time period.

[0764] Step 4:

[0765] The server acquires user emotional information and analyzes it using an emotion engine. Inputs include audio and facial expression data from the user's device. The analysis results quantify the user's stress level and satisfaction level. The output is numerical information about the user's emotional state.

[0766] Step 5:

[0767] The server generates instructions for adjusting traffic signals and providing route guidance based on the analysis results and predictive information. It then executes a control algorithm using the outputs from steps 2, 3, and 4 as input. Specifically, it generates instructions to adjust the timing of traffic signals to alleviate congestion. The output consists of instructions related to traffic control.

[0768] Step 6:

[0769] The terminal receives control instructions from the server and transmits them to the actual traffic control device. The input is the output information from step 5. Specific actions include switching signals and updating traffic information. The output is the traffic environment resulting from the execution of the control instructions.

[0770] Step 7:

[0771] The device provides users with personalized traffic information. Inputs include traffic data and sentiment analysis results from a server. Using this information, and leveraging the Google Maps API, the device notifies users of optimal routes and travel advice tailored to their device. Output is customized information displayed on the user's device.

[0772] Step 8:

[0773] The user makes a move based on information received from the device. The input is the information provided in step 7. For example, a specific action would be to select a route that avoids congestion and move accordingly. The output is an improvement in the user's travel experience.

[0774] (Application Example 2)

[0775] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0776] While understanding and managing traffic conditions has been done with conventional technologies, providing information that takes into account the emotional state of users has been insufficient. In particular, with the spread of autonomous vehicles, there is a need for technologies that reduce user stress and anxiety and provide safer and more comfortable travel. This invention aims to solve these problems and realize personalized traffic information provision that responds to the emotions of users.

[0777] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0778] In this invention, the server includes means for receiving video information acquired from a data acquisition device, means for analyzing the emotional state, and means for providing emotionally-based routes to the user via an in-vehicle device. This makes it possible to travel in a way that takes the user's emotional state into consideration and reduces stress and anxiety.

[0779] A "data acquisition device" is a device that collects video information necessary to recognize traffic conditions.

[0780] A "machine learning model" uses statistical methods to analyze patterns in traffic conditions based on large amounts of data and predict future situations.

[0781] "Analysis means" refers to methods for identifying the characteristics and circumstances of traffic participants based on acquired video information.

[0782] A "predictive tool" is a means of predicting future traffic conditions based on past traffic data and current conditions.

[0783] "Instruction generation means" refers to means for generating instructions for traffic control based on predictive information and analysis results.

[0784] "Control means" refers to means that receive generated instructions and transmit appropriate commands to the traffic control device.

[0785] "Recording means" refers to means for storing data and making it available for long-term analysis.

[0786] "Means of provision" refers to the means of providing recorded information or analysis results to a user or some system.

[0787] An "emotion recognition tool" is a means of providing information tailored to individual users by analyzing their emotional state.

[0788] An "in-vehicle device" is a device installed inside a vehicle that functions as an interface with the user and displays traffic information and route suggestions.

[0789] To implement this invention, a system for optimizing traffic conditions is constructed. First, the server receives video information from a data acquisition device and analyzes this information using a machine learning model. Based on the analyzed data, it predicts traffic flow and the occurrence of congestion. Based on this, the server generates instructions for traffic lights and other traffic control devices and transmits them through the control means.

[0790] Furthermore, the server analyzes the user's emotional state using emotion recognition tools. This process involves acquiring facial and voice data using cameras and microphones, and utilizing an emotion analysis engine (for example, a common cloud-based emotion analysis API). The analyzed emotion information forms the basis for the server to provide personalized traffic information to each user.

[0791] The terminal displays information sent from the server to the user in real time via an in-vehicle device. This information includes route suggestions that take the user's emotions into consideration. The in-vehicle interface is designed to be operable via touch panel and voice recognition.

[0792] Users can receive information on their smartphones or in-vehicle devices and travel comfortably by utilizing emotionally sensitive options. For example, the system suggests routes that avoid congestion to users experiencing high stress levels. This is particularly useful for users commuting or traveling.

[0793] An example of a prompt to a generative AI model is, "The user is planning a holiday drive; please suggest a relaxing route with beautiful scenery." This prompt forms the basis for providing information that matches the user's desired experience.

[0794] In this way, the present invention can be implemented as a system in which the server, terminal, and user each play their own unique roles and work together to optimize traffic conditions and improve the user experience.

[0795] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0796] Step 1:

[0797] The server receives video information acquired from data acquisition devices. Video data from cameras installed on roads is used as input. The server collects this data and stores it in a database for analysis in a digital format. The output is data ready for the next analysis step.

[0798] Step 2:

[0799] The server analyzes video information using a machine learning model. The input is the video data organized in Step 1. Based on this data, the server recognizes the number, speed, and location of traffic participants. This analysis is performed to identify traffic flow and potential congestion information, and the output generates information on the current state of traffic.

[0800] Step 3:

[0801] The server predicts future traffic conditions based on historical traffic data. Input includes not only current traffic information but also historical data. The server analyzes this data using statistical algorithms. The output is a prediction of traffic conditions for the near future.

[0802] Step 4:

[0803] The server generates instructions based on the analysis results and predictive information. The traffic conditions and predictive information obtained up to step 3 are used as input. Based on this, the server formulates specific traffic control instructions, such as adjusting traffic light timings and suggesting alternative routes. Specific control commands are generated as output.

[0804] Step 5:

[0805] The server collects and analyzes data to recognize the user's emotional state. Data from the user's smartphone or cameras and microphones installed in the vehicle are used as input. An emotion analysis engine extracts emotional information from the user's facial expressions and voice. This process identifies the user's emotional state, and that information is provided as output.

[0806] Step 6:

[0807] The server provides customized traffic information based on emotional data. It considers emotional states and traffic data as input to generate optimal routes and traffic-related suggestions for the user. Specifically, it can suggest routes with relaxing scenery, for example. The output is customized traffic information displayed on the user's device.

[0808] Step 7:

[0809] The terminal displays control instructions and emotion-sensitive suggestions from the server on the in-vehicle device. Inputs are control instructions and customized traffic information sent from the server. The terminal notifies the user of these and helps them take appropriate action. Outputs include information to guide the user's actions, provided through the in-vehicle display and audio output.

[0810] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0811] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0812] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0813] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0814] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0815] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0816] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0817] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0818] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0819] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0820] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0821] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0822] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0823] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0824] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0825] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using this memory.

[0826] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0827] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0828] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0829] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0830] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0831] The following is further disclosed regarding the embodiments described above.

[0832] (Claim 1)

[0833] Receive video information acquired from the data acquisition device.

[0834] An analysis means for analyzing the aforementioned video information using a machine learning model,

[0835] A predictive method that predicts future traffic conditions based on past traffic data,

[0836] Instruction generation means that generates control instructions based on analysis results and predictive information,

[0837] Control means for transmitting the control instruction to the traffic control device,

[0838] A recording means for recording the aforementioned analysis results and predictive information and for conducting long-term analysis,

[0839] A providing means for providing the analysis results of the recording means,

[0840] A system that includes this.

[0841] (Claim 2)

[0842] The analysis means further comprises means for identifying the number, speed, and location of traffic participants.

[0843] The system according to claim 1, wherein the instruction generating means includes means for generating instructions to adjust traffic control signals.

[0844] (Claim 3)

[0845] The system according to claim 1, wherein the providing means includes means for providing traffic information in real time through the user's computing device.

[0846] "Example 1"

[0847] (Claim 1)

[0848] Receive visual information acquired from a data acquisition device,

[0849] An analysis means for processing the aforementioned visual information using a machine learning method,

[0850] A predictive means for forecasting future travel patterns based on past travel data,

[0851] Command generation means that creates commands based on analysis results and predictive information,

[0852] Control means for transmitting the command to the mobile control device,

[0853] A recording means for recording the aforementioned analysis results and predictive information and for performing long-term analysis,

[0854] A supply means for providing the analysis results of the recording means,

[0855] A system that includes this.

[0856] (Claim 2)

[0857] The analysis means further comprises means for identifying the quantity, speed, and location of the moving participants.

[0858] The system according to claim 1, wherein the command generation means includes means for generating commands to adjust movement control signals.

[0859] (Claim 3)

[0860] The system according to claim 1, wherein the supply means includes means for providing movement information immediately through the user's computing device.

[0861] "Application Example 1"

[0862] (Claim 1)

[0863] Receive video information acquired from the data acquisition device.

[0864] The means for analyzing the aforementioned video information using a machine learning model,

[0865] A means of predicting future traffic conditions based on past traffic data,

[0866] A means for generating control instructions based on analysis results and predictive information and transmitting the information to the user terminal,

[0867] Means for transmitting the aforementioned control instruction to a traffic control device,

[0868] A means for recording the aforementioned analysis results and predictive information and conducting long-term analysis,

[0869] A means for providing the analysis results of the recording means and making personalized route suggestions,

[0870] A system that includes this.

[0871] (Claim 2)

[0872] The analysis means further comprises means for identifying the number, speed, and location of traffic participants.

[0873] The system according to claim 1, wherein the instruction generation means includes means for generating instructions to adjust traffic control signals and for proposing an optimized travel path.

[0874] (Claim 3)

[0875] The system according to claim 1, wherein the providing means includes means for providing traffic information in real time through the user's information processing device and for suggesting a recommended time to avoid congestion.

[0876] "Example 2 of combining an emotion engine"

[0877] (Claim 1)

[0878] Receiving image information acquired from data collection means,

[0879] An analysis means for analyzing the aforementioned image information using a machine learning method,

[0880] A predictive method that predicts future traffic conditions based on past traffic data,

[0881] Instruction generation means that generates control instructions based on analysis results and predictive information,

[0882] The instruction generation means also analyzes the user's emotional information and provides personalized traffic information.

[0883] Control means for transmitting the control instruction to the traffic control device,

[0884] A recording means for recording the aforementioned analysis results and predictive information and for conducting long-term analysis,

[0885] A providing means for providing the analysis results of the recording means,

[0886] A system that includes this.

[0887] (Claim 2)

[0888] The analysis means further comprises means for identifying the number, speed, and location of traffic participants.

[0889] The system according to claim 1, wherein the instruction generating means includes means for generating instructions to adjust traffic control signals.

[0890] (Claim 3)

[0891] The system according to claim 1, wherein the providing means includes means for providing personalized traffic information based on emotional information in real time through the user's computing device.

[0892] "Application example 2 when combining with an emotional engine"

[0893] (Claim 1)

[0894] Receive video information acquired from the data acquisition device.

[0895] An analysis means for analyzing the aforementioned video information using a machine learning model,

[0896] A predictive method that predicts future traffic conditions based on past traffic data,

[0897] Instruction generation means that generates control instructions based on analysis results and predictive information,

[0898] Control means for transmitting the control instruction to the traffic control device,

[0899] A recording means for recording the aforementioned analysis results and predictive information and for conducting long-term analysis,

[0900] A providing means for providing the analysis results of the recording means,

[0901] An emotion recognition means that recognizes the emotional state of the user and provides traffic information corresponding to that emotion,

[0902] A means of providing emotionally sensitive routes to users via in-vehicle devices,

[0903] A system that includes this.

[0904] (Claim 2)

[0905] The analysis means further comprises means for identifying the number, speed, and location of traffic participants.

[0906] The system according to claim 1, wherein the instruction generating means includes means for generating instructions to adjust traffic control signals.

[0907] (Claim 3)

[0908] The system according to claim 1, wherein the providing means includes means for providing emotion-based traffic information in real time through the user's computing device. [Explanation of Symbols]

[0909] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Receive video information acquired from the data acquisition device. An analysis means for analyzing the aforementioned video information using a machine learning model, A predictive method that predicts future traffic conditions based on past traffic data, Instruction generation means that generates control instructions based on analysis results and predictive information, Control means for transmitting the control instruction to the traffic control device, A recording means for recording the aforementioned analysis results and predictive information and for conducting long-term analysis, A providing means for providing the analysis results of the recording means, A system that includes this.

2. The analysis means further comprises means for identifying the number, speed, and location of traffic participants. The system according to claim 1, wherein the instruction generating means includes means for generating instructions to adjust traffic control signals.

3. The system according to claim 1, wherein the providing means includes means for providing traffic information in real time through the user's computing device.

Images