System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving real-time video data from a camera; means for pre-processing the video data; means for identifying vehicles using a generative artificial intelligence model; means for storing identified vehicle data in a traffic database; means for requesting traffic data for a specified time period or road segment and responsively retrieving the relevant data from an information storage device; and means for analyzing and displaying the received data as a statistical graph or heat map.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Current road traffic volume survey methods use deep learning technology, but the accuracy of vehicle identification decreases at night, making accurate data collection difficult. Furthermore, surveys rely on labor-intensive manual counting, making efficient and accurate data collection necessary. Furthermore, analyzing the collected data and deriving insights from it is difficult, resulting in the data not being fully utilized in urban planning and transportation policy formulation. [Means for solving the problem]

[0005] To solve this problem, the present invention provides the following means: a system including a means for receiving real-time video data from a camera, a means for preprocessing the video data, a means for identifying vehicles using a generative artificial intelligence model, a means for storing the identified vehicle data in a traffic volume database, and a means for analyzing and visualizing the traffic volume data. This system enables highly accurate vehicle identification even at night, realizing efficient and accurate collection and analysis of traffic volume data. Furthermore, the system derives specific insights from the analyzed data and provides them in an intuitively understandable format, thereby supporting the formulation of urban plans and transportation policies.

[0006] A "camera" is an optical device that captures video data in real time and transmits that information to a processing system.

[0007] "Real-time video data" refers to video data received from a camera almost immediately, without delay, and in a format that can be processed instantly by the system.

[0008] "Preprocessing" refers to a series of steps to convert video data into a format suitable for the generative AI model, and specifically includes image resizing and noise removal.

[0009] A "generative AI model" is a machine learning model that is trained using large amounts of data to identify vehicles in video footage.

[0010] "Vehicle identification" refers to the use of generative AI models to automatically detect vehicles in video data and classify their type (e.g., car, truck, motorcycle, etc.).

[0011] A "traffic volume database" is a database for systematically storing and managing identified vehicle data.

[0012] "Analysis" is a series of actions that process stored traffic data and extract statistical information and patterns.

[0013] "Visualization" is the process of displaying analyzed data in a visual format such as a graph or heat map to enable users to intuitively understand it.

[0014] "Insights" refer to insights and discoveries gained from data analysis, and are information that can serve as a guide for specific decisions and actions.

[0015] The "system" refers to the entire system consisting of cameras, generative AI models, databases, analysis tools, visualization tools, etc., which automatically measures and analyzes traffic volume. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention provides a system called "Traffic AI" that highly automates road traffic volume surveys. This system receives video data from cameras in real time, identifies vehicles using a generative AI model, and measures and analyzes road traffic volume. The specific roles of the server, terminal, and user, and the processing involved, are explained in detail below.

[0038] Server Roles and Operations

[0039] Video data reception and preprocessing

[0040] The server first receives real-time video data from the camera. This data is sent to the server with minimal delay, and processing begins immediately. The received video data is pre-processed into a format suitable for the generative AI model. This pre-processing includes cropping the image frame by frame, removing noise, and adjusting the resolution.

[0041] Vehicle identification and storage in a database

[0042] The preprocessed video data is then fed into a generative AI model. This model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. The identified vehicle types (cars, trucks, motorcycles, etc.) and their counts are output in JSON format and stored in a traffic database. This data also includes timestamps and vehicle location information.

[0043] Data analysis and visualization

[0044] The data stored in the traffic database is analyzed on an ongoing basis. This analysis involves generating statistical data and extracting patterns. The analyzed data is then visualized using visualization tools. Specifically, it is displayed as graphs or heat maps showing fluctuations in traffic volume.

[0045] Terminal roles and processing

[0046] Data Request and Receipt

[0047] In response to a user request, the terminal sends a request for traffic volume data to the server. The request includes a specification of a specific time period and road section. In response, the server retrieves the relevant data from the traffic volume database and sends it to the terminal.

[0048] Data analysis and presentation

[0049] The device analyzes the received traffic volume data and displays it in a user-friendly format, using statistical graphs and heat maps, allowing users to intuitively grasp the traffic situation.

[0050] User operations

[0051] Operating the application and checking the data

[0052] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section. For example, if the user wants to check traffic volume on a specific road at night, they set the condition "traffic volume on major road A from 22:00 to 22:30" in the app. Based on this condition, the device sends a request to the server and displays the retrieved data to the user.

[0053] Decision-making and action

[0054] Users can view the visualized data and make adaptive decisions as needed, such as choosing an alternative route to avoid heavy traffic on major road A. Urban planners can also use the insights gained to formulate transportation policies and infrastructure development plans.

[0055] The present invention makes it possible to carry out efficient and accurate surveys and analyses of road traffic volumes, which can be of great help in formulating urban plans and transportation policies.

[0056] The processing flow will be explained below.

[0057] Now, let me explain the program step by step.

[0058] Server Processing

[0059] Step 1:

[0060] The server receives real-time video data from the camera. The video data is transmitted at a constant frame rate, and the server receives it sequentially.

[0061] Specific operation: Receives video data from the camera at 10 fps (frames per second).

[0062] Step 2:

[0063] The server preprocesses the received video data, including cropping the image frame by frame, removing noise, and adjusting the resolution, converting it into a format suitable for the generative AI model.

[0064] Specific operation: Resize each frame to 256x256 pixels and remove noise from the image.

[0065] Step 3:

[0066] The server inputs the preprocessed data into a generative AI model to identify vehicles. The generative AI model then identifies vehicles in the video with high accuracy and determines their type (cars, trucks, motorcycles, etc.) and number.

[0067] Specific operation: In the first frame, two cars and one truck are identified, and in the second frame, one motorcycle is identified.

[0068] Step 4:

[0069] The server stores the identified vehicle data in a traffic database, including vehicle type, number, time stamp, and location information.

[0070] Specific operation: Save the data of two cars and one truck at 2023-10-01 22:05:10 in the database.

[0071] Terminal handling

[0072] Step 5:

[0073] The terminal sends a request for traffic data to the server based on the user's request, which includes a specification of a particular time period and road section.

[0074] Specific operation: Send a message to the server requesting traffic volume data for major road A from 22:00 to 22:30.

[0075] Step 6:

[0076] The terminal analyzes the traffic volume data received from the server, which includes formatting the data and statistical compilation.

[0077] Specific operation: Calculate the total number of vehicles and their breakdown on main road A from 22:00 to 22:30.

[0078] Step 7:

[0079] The device displays the analysis results in a user-friendly format, including generating graphs and heat maps showing traffic fluctuations.

[0080] Specific operation: Displays the traffic volume on major road A as a line graph, showing fluctuations by time period.

[0081] User operations

[0082] Step 8:

[0083] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section.

[0084] Specific operation: Set the condition "traffic volume on major road A from 22:00 to 22:30" on the application.

[0085] Step 9:

[0086] Users can view visualized data and make on-the-fly decisions as needed.

[0087] Specific operation: Select a route that avoids times when traffic on major road A is heavy.

[0088] These are the specific processing steps of the server, terminal, and user, which enable efficient and accurate survey and analysis of road traffic volume.

[0089] Example 1

[0090] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0091] Conventional road traffic volume surveys require manual analysis of large amounts of video data, which is time-consuming and labor-intensive. It is also difficult to accurately identify vehicles in nighttime video data. Furthermore, there is a lack of systems for analyzing and visualizing traffic volume data in real time, which hinders rapid decision-making in urban planning and transportation policies.

[0092] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0093] In this invention, the server includes a means for receiving real-time video data from the camera, a means for preprocessing the video data, and a means for identifying vehicles using a generative artificial intelligence model. This enables automatic processing of video data and highly accurate vehicle identification. It also includes a means for storing the identified vehicle data in an information storage device, a means for analyzing and visualizing traffic volume data, a means for requesting traffic volume data for a specified time period or road section and retrieving the corresponding data from the information storage device, and a means for analyzing the received data and displaying it as a statistical graph or heat map. This enables real-time analysis and visualization of traffic volume, enabling rapid decision-making in urban planning and transportation policies.

[0094] A "camera" is a device that captures video data and transmits it in real time.

[0095] "Video data" is a digital record of visual information captured by a camera.

[0096] "Preprocessing" is the process of converting video data into a format that can be used by a generative AI model, and includes processes such as noise removal and resolution adjustment.

[0097] A "generative artificial intelligence model" is a machine learning model that is trained using large amounts of data and can perform a specific task (e.g., vehicle identification) with high accuracy.

[0098] "Vehicle identification" is the process of identifying vehicles contained in video data and determining their type and location.

[0099] "Information storage device" means a digital storage system for long-term preservation of identified data.

[0100] "Traffic volume data" is a collection of information about vehicle traffic in a particular area or time period.

[0101] "Analysis" is the process of processing data using statistical methods and pattern recognition techniques to extract meaningful information.

[0102] "Visualization" means displaying the analyzed data in a visual format such as a graph or heat map.

[0103] A "statistical graph" is a diagram that visually represents numerical data and is used to show fluctuations and trends in traffic volume.

[0104] A "heat map" is a graphical tool that uses shades of color to represent data density, showing the concentration of traffic in a particular area or time period.

[0105] "User interface" refers to the screen and operating means by which a user interacts with a system and inputs and outputs information.

[0106] This invention relates to "Traffic AI," a system for highly automated road traffic volume surveys. This system receives real-time video data from cameras, identifies vehicles using generative AI models, and analyzes and visualizes traffic volume. For example, the hardware used includes cameras, servers, and terminals (e.g., mobile devices and PCs), and the software and libraries used include OpenCV, YOLOv5, Pandas, Grafana, HTTP protocol, and MongoDB.

[0107] Server Roles and Operations

[0108] The server first receives real-time video data from the camera. This data is then sent to the server via a streaming protocol. The received video data is immediately pre-processed using the OpenCV library. This pre-processing includes frame-by-frame image extraction, noise reduction, and resolution adjustment.

[0109] The preprocessed video data is input into a generative AI model (e.g., YOLOv5). This AI model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. The type and number of identified vehicles are output in JSON format along with timestamps and location information and stored in an information storage device (e.g., MongoDB).

[0110] The data stored in the traffic database is analyzed using the Pandas library. The analysis includes generating statistical data and extracting patterns, and the resulting data is visualized using visualization tools such as Grafana. Specifically, graphs and heat maps showing fluctuations in traffic volume are generated and displayed in a format that users can intuitively understand.

[0111] Terminal roles and processing

[0112] The device sends a data request to the server in response to a user request. This request includes information about a specific time period and road section. The server processes the request, retrieves the corresponding traffic data from the information storage device, and sends it to the device.

[0113] The received data is further analyzed on the device using Python libraries such as Matplotlib and Seaborn, and the data is displayed as statistical graphs and heat maps, allowing users to intuitively understand the traffic situation.

[0114] User operations

[0115] The user launches the application and sets the conditions for checking traffic volume data for a specific time period or road section. For example, the user may want to check "traffic volume on major road B from 07:00 to 09:00 on October 1, 2023," and specify this in the app. An example of this prompt is:

[0116] "Please display a graph showing the traffic volume and its fluctuations by vehicle type on major road B from 07:00 to 09:00 on October 1, 2023."

[0117] The device sends a request to the server based on these conditions and displays the retrieved data to the user. The user can then refer to the displayed graphs and heat maps and make appropriate decisions as needed. For example, they can change their departure time or choose an alternative route to avoid heavy traffic. Urban planners can also use the data to create transportation policies and infrastructure development plans.

[0118] As described above, "Traffic AI" is a system that realizes efficient and accurate surveys and analysis of road traffic volume and supports rapid decision-making in urban planning and transportation policy. By accurately performing each processing step based on the specific embodiment of this invention, advanced traffic volume management becomes possible.

[0119] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0120] Step 1:

[0121] The server receives real-time video data from the camera. The input is the video data from the camera, and the output is the video data transferred to the server. This video data is processed frame by frame and immediately moves to the next step. For example, we use the RTSP protocol to get the video stream and split it into frames using OpenCV.

[0122] Step 2:

[0123] The server preprocesses the received video data. The input is the video data received in step 1, and the output is the preprocessed video data. Preprocessing includes noise removal, resolution adjustment, and frame-by-frame image cropping. Specifically, the OpenCV library is used to remove noise with Gaussian blur and resize the video to the specified resolution.

[0124] Step 3:

[0125] The server inputs the preprocessed video data into a generative AI model. The input is the preprocessed video data, and the output is the identified vehicle data. The generative AI model (e.g., YOLOv5) identifies vehicles in the video with high accuracy and outputs the vehicle type and location information. For example, the YOLOv5 model can be used to identify cars, trucks, motorcycles, etc.

[0126] Step 4:

[0127] The server stores the identified vehicle data in an information storage device. The input is the identified vehicle data, and the output is the data stored in a database. The data is stored in JSON format along with timestamps and location information. Specifically, MongoDB is used to store the identification results.

[0128] Step 5:

[0129] The server analyzes the data stored in the traffic volume database. The input is the vehicle data stored in the database, and the output is the analysis results. The analysis includes generating statistical data and extracting patterns. Specifically, Pandas is used to aggregate the data and extract fluctuations and trends in traffic volume by time period.

[0130] Step 6:

[0131] The server visualizes the analysis results. The input is the analysis results, and the output is the visualized data. The visualization includes graphs and heat maps showing traffic volume fluctuations. Specifically, Grafana is used to create interactive dashboards.

[0132] Step 7:

[0133] The terminal sends a data request from the user to the server. The input is the user's request conditions (e.g., a specific time period or road section), and the output is the data request to the server. The request is sent using the HTTP protocol.

[0134] Step 8:

[0135] The terminal analyzes the traffic data received from the server. The input is the traffic data from the server, and the output is the analysis results. The analysis includes generating statistical graphs and heat maps. Specifically, the data is visualized using Python's Matplotlib and Seaborn.

[0136] Step 9:

[0137] The terminal displays the analysis results to the user. The input is the analysis results, and the output is visualization data displayed to the user. Dashboards, statistical graphs, and heat maps are used for display, allowing the user to intuitively understand the traffic situation.

[0138] Step 10:

[0139] The user launches the application and sets the conditions for checking traffic volume data. The input is the user's requested conditions, and the output is the condition settings to the device. For example, if the user wants to check "traffic volume on major road B from 07:00 to 09:00 on October 1, 2023," they specify this in the app.

[0140] Step 11:

[0141] Users make appropriate decisions based on the displayed traffic data. The input is the displayed visualization data, and the output is the user's decision. For example, they may change their departure time to avoid heavy traffic hours or choose an alternative route. Urban planners also use the data to create transportation policies and infrastructure development plans.

[0142] (Application example 1)

[0143] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0144] Conventional road traffic volume survey systems require a great deal of manpower and time to grasp traffic volume and identify vehicles, making real-time analysis difficult, especially when collecting data at night. Furthermore, they are inadequate in providing optimal detour routes based on the analyzed data, and in presenting information intuitively to drivers. New technologies are needed to solve these issues.

[0145] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0146] In this invention, the server includes means for receiving real-time video data from a camera, means for preprocessing the video data, means for identifying vehicles using a generative artificial intelligence model, means for storing the identified vehicle data in a traffic volume database, means for analyzing and visualizing the traffic volume data, and means for proposing an optimal route based on the analyzed traffic volume data, thereby enabling real-time traffic volume understanding, provision of optimal detour routes, and intuitive information presentation.

[0147] A "camera" is a device for acquiring real-time image data on a road.

[0148] "Real-time video data" refers to current video information continuously acquired through a camera.

[0149] "Video data preprocessing" refers to the process of converting raw data acquired from a camera into a format suitable for a generative artificial intelligence model.

[0150] A "generative artificial intelligence model" is an AI model trained using large amounts of data and capable of identifying vehicles in video footage.

[0151] "Vehicle identification" is the process of recognizing and classifying vehicles such as cars, trucks, and motorcycles from video data.

[0152] A "traffic database" is a data storage system for storing identified vehicle data for later analysis.

[0153] "Traffic data analysis" refers to the statistical processing of data stored in traffic databases to extract meaningful patterns and insights.

[0154] "Visualization" refers to the presentation of analyzed traffic data in a visual format such as a graph or heat map.

[0155] "Optimal route proposal" means selecting and presenting the most efficient and least congested route based on real-time traffic analysis data.

[0156] This invention provides a system called "Traffic Navigator" based on "Traffic AI," which highly automates road traffic volume surveys. This system receives video data from cameras in real time, identifies vehicles using a generative AI model, measures and analyzes traffic volume, and proposes optimal routes.

[0157] Server Roles and Operations

[0158] The server first receives real-time video data from the camera. This data is continuously acquired through the camera and processing begins immediately. The received video data is pre-processed into a format suitable for the generative AI model. This pre-processing includes cropping the image for each frame, removing noise, and adjusting the resolution. Libraries such as OpenCV are used for pre-processing.

[0159] The preprocessed video data is then fed into a generative AI model. This generative AI model is trained on a large amount of data and is able to identify vehicles in the video with high accuracy. The identified vehicle types (cars, trucks, motorcycles, etc.) and their counts are output in JSON format and stored in a traffic database. This data also includes timestamps and vehicle location information.

[0160] The data stored in the traffic database is analyzed on an ongoing basis. This analysis involves generating statistical data and extracting patterns. The analyzed data is then visualized using visualization tools. Specifically, it is displayed as graphs or heat maps showing fluctuations in traffic volume. Libraries such as Matplotlib are used at this stage.

[0161] The server then performs calculations to suggest optimal routes based on the analyzed traffic data, and presents the most efficient detour route to the driver based on the real-time analysis results.

[0162] Terminal roles and processing

[0163] The device sends a request for traffic volume data to the server in response to a user request. The request includes a specific time period and road section. The server then retrieves the relevant data from the traffic volume database and sends it to the device. The device then analyzes the received traffic volume data and displays it in an easy-to-understand format, such as a statistical graph or heat map.

[0164] User operations

[0165] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section. For example, if a user wants to check traffic volume on a specific road at night, they can set the condition "traffic volume on major road A from 22:00 to 22:30" on the app. Based on these conditions, the server provides analysis results and suggests the optimal detour route.

[0166] Specific examples

[0167] For example, it can be seen in real time that traffic volume on major road A is heavy at night. In this case, the system can immediately suggest optimal detour routes to help drivers avoid traffic jams. Urban planners can also use heat maps and statistical graphs to intuitively understand the traffic situation in specific areas and formulate appropriate traffic policies.

[0168] Prompt Sentence Examples

[0169] "Analyze the traffic volume on major road A from 10:00 PM to 10:30 PM at night and generate a heat map."

[0170] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0171] Step 1:

[0172] The server receives real-time video data from the camera. The video information continuously acquired through the camera is input and immediately processed. The received video data is used in subsequent processing steps.

[0173] Step 2:

[0174] The server preprocesses the received video data. This preprocessing involves cropping the image frame by frame, removing noise, and adjusting the resolution. Using an image processing library such as OpenCV, the quality of the input data is converted into a format suitable for the generative AI model. The preprocessed image data is then output.

[0175] Step 3:

[0176] The server inputs the preprocessed video data into a generative AI model to identify vehicles. The generative AI model uses a large amount of training data to perform highly accurate vehicle identification. The model outputs the identified vehicle type (car, truck, motorcycle, etc.) and its number, and this information is used in subsequent steps.

[0177] Step 4:

[0178] The server stores the identified vehicle data in a traffic database. The identification results are stored in JSON format, including timestamps and vehicle location information. The identified vehicle data is given as input and recorded in the traffic database as output.

[0179] Step 5:

[0180] The server analyzes the data stored in the traffic database. This analysis includes generating statistics and extracting patterns. The input data is the contents of the traffic database, and the output data is the analysis results. Statistical calculation methods are used.

[0181] Step 6:

[0182] The server visualizes the analyzed data. Using a visualization tool, the data is output as graphs and heat maps showing fluctuations in traffic volume. The Matplotlib library is used to visually display the analysis results.

[0183] Step 7:

[0184] The server proposes the optimal route based on the analysis results. Based on the analysis results in real time, it calculates the most efficient detour route for the driver. The input data is the analysis results, and the output data is the proposed optimal route.

[0185] Step 8:

[0186] The terminal receives the data sent from the server and requests specific traffic volume data according to the user's request. The request includes the specification of a specific time period or road section, and obtains the corresponding data from the server.

[0187] Step 9:

[0188] The device analyzes the traffic volume data it receives and displays it in statistical graphs and heat maps, allowing users to intuitively understand the data and assisting them in selecting the optimal route.

[0189] Step 10:

[0190] The user selects traffic volume data for a specific time period or road section through the application, and the device sends a request to the server. Based on the acquired data, the user can intuitively understand the traffic situation and act accordingly.

[0191] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0192] This invention provides a system called "Traffic AI Plus," which combines a highly automated system for road traffic volume surveys with an emotion engine that recognizes user emotions. This system receives video data from a camera in real time, identifies vehicles using a generative AI model, and measures and analyzes traffic volume. The emotion engine also recognizes user emotions and customizes the presentation of analysis results. The specific roles of the server, terminal, and user, as well as the processing involved, are described in detail below.

[0193] Server Roles and Operations

[0194] Video data reception and preprocessing

[0195] The server receives real-time video data from the camera, which is then processed immediately. The received video data is pre-processed into a format suitable for the generative AI model. Pre-processing includes cropping the image frame by frame, removing noise, and adjusting the resolution.

[0196] Vehicle identification and storage in a database

[0197] The preprocessed video data is then fed into a generative AI model. This model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. The identified vehicle types (cars, trucks, motorcycles, etc.) and their counts are output in JSON format and stored in a traffic database. The data also includes timestamps and vehicle location information.

[0198] Data analysis and visualization

[0199] The data stored in the traffic volume database is analyzed on an ongoing basis using analytical tools. This analysis includes generating statistical data and extracting patterns. The analyzed data is then visualized using visualization tools, and displayed as graphs and heat maps showing fluctuations in traffic volume.

[0200] Terminal roles and processing

[0201] Data Request and Receipt

[0202] The terminal sends a request for traffic volume data to the server in response to a user request. The request includes a specification of a specific time period and road section. The server then retrieves the relevant data from the traffic volume database and sends it to the terminal.

[0203] Data analysis and presentation

[0204] The device analyzes the received traffic data and displays it in a user-friendly format, using statistical graphs and heat maps. Furthermore, an emotion engine customizes the analysis results to reflect the user's emotions.

[0205] User operations

[0206] Operating the application and checking the data

[0207] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section. For example, if the user wants to check traffic volume on a specific road at night, they set the condition "traffic volume on major road A from 22:00 to 22:30" in the app. Based on this condition, the device sends a request to the server and displays the retrieved data to the user.

[0208] Emotion recognition and customized alerts

[0209] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions, and then customizes traffic information displays and alerts accordingly. For example, if the user is nervous, the device will suggest a relaxing route, and if the user is happy, the device will suggest the shortest route.

[0210] Decision-making and action

[0211] Users can view the visualized data and make on-the-fly decisions as needed, such as choosing an alternative route to avoid heavy traffic on major road A. Urban planners can also use the insights gained to formulate transportation policies and infrastructure development plans.

[0212] This invention not only enables efficient and accurate survey and analysis of road traffic volume, but also enables the provision of customized traffic information that takes user emotions into consideration, which will greatly support the formulation of urban planning and transportation policies and improve user satisfaction.

[0213] The processing flow will be explained below.

[0214] Server Processing

[0215] Step 1:

[0216] The server receives real-time video data from the camera. The video data is transmitted at a constant frame rate, and the server receives it sequentially.

[0217] Specific operation: Receives video data from the camera at 10 fps (frames per second).

[0218] Step 2:

[0219] The server pre-processes the received video data, including cutting out the image frame by frame, removing noise, and adjusting the resolution.

[0220] Specific operation: Resize each frame to 256x256 pixels and remove noise from the image.

[0221] Step 3:

[0222] The server inputs the preprocessed data into a generative AI model to identify vehicles. The generative AI model then identifies vehicles in the video with high accuracy and determines their type (cars, trucks, motorcycles, etc.) and number.

[0223] Specific operation: In the first frame, two cars and one truck are identified, and in the second frame, one motorcycle is identified.

[0224] Step 4:

[0225] The server stores the identified vehicle data in a traffic database, including vehicle type, number, time stamp, and location information.

[0226] Specific operation: Save the data of two cars and one truck at 2023-10-01 22:05:10 in the database.

[0227] Terminal handling

[0228] Step 5:

[0229] The terminal sends a request for traffic data to the server based on the user's request, which includes a specification of a particular time period and road section.

[0230] Specific operation: Send a message to the server requesting traffic volume data for major road A from 22:00 to 22:30.

[0231] Step 6:

[0232] The terminal analyzes the traffic volume data received from the server, which includes formatting the data and statistical compilation.

[0233] Specific operation: Calculate the total number of vehicles and their breakdown on main road A from 22:00 to 22:30.

[0234] Step 7:

[0235] The device displays the analysis results in a user-friendly format, including generating graphs and heat maps showing traffic fluctuations.

[0236] Specific operation: Displays the traffic volume on major road A as a line graph, showing fluctuations by time period.

[0237] Step 8:

[0238] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions, and displays and alerts are customized based on the recognized emotions.

[0239] What it does: If the user is nervous, provide them with directions and travel information to help them relax.

[0240] User operations

[0241] Step 9:

[0242] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section.

[0243] Specific operation: Set the condition "traffic volume on major road A from 22:00 to 22:30" on the application.

[0244] Step 10:

[0245] Users can view visualized data and make on-the-fly decisions as needed.

[0246] Specific operation: Select a route that avoids times when traffic on major road A is heavy.

[0247] These are the specific processing steps of the server, terminal, and user. This not only enables efficient and accurate survey and analysis of road traffic volume, but also provides customized traffic information that takes user emotions into consideration.

[0248] Example 2

[0249] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0250] While conventional road traffic volume survey systems can measure traffic volume using real-time video data, they have the problem of being unable to provide customized information that takes user emotions into account.Furthermore, they have a problem of not being able to identify vehicles with a high degree of accuracy even in nighttime video data.

[0251] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0252] In this invention, the server includes means for receiving real-time video data from a camera, means for preprocessing the video data, means for identifying vehicles using a generative artificial intelligence model, means for storing the identified vehicle data in a traffic volume database, means for analyzing and visualizing the traffic volume data, and means for recognizing a user's emotions and customizing the presentation method of the analysis results based on the emotions. This not only enables efficient and accurate road traffic volume survey and analysis, but also enables the provision of customized traffic information that takes the user's emotions into consideration.

[0253] A "camera" is a photographing device for capturing video data in real time.

[0254] "Real-time video data" refers to data that is obtained and transmitted immediately as video information at the current time.

[0255] "Preprocessing" refers to a series of operations that convert video data into a format suitable for subsequent analysis and recognition.

[0256] A "generative artificial intelligence model" is a machine learning model that learns from data and performs a specific task (e.g., vehicle identification).

[0257] "Means for identifying vehicles" refers to technologies or algorithms for detecting vehicles in video data and identifying their type and number.

[0258] "Vehicle data" refers to data including the type, number, timestamp, location information, etc. of identified vehicles.

[0259] A "traffic database" is a database system for storing identified vehicle data.

[0260] "Analysis" refers to the process of generating statistical data and extracting patterns based on traffic volume data.

[0261] "Visualization" refers to displaying analyzed data in a visual format such as a graph or heat map.

[0262] "Means for recognizing user emotions" refers to technology or devices that analyze emotions from the user's facial expressions and tone of voice.

[0263] "Customization means" refers to technologies and algorithms that change the way information is presented and the format of the display based on the user's emotions.

[0264] This invention relates to a system called "Traffic AI Plus," which combines a highly automated system for road traffic volume surveys with a function for recognizing user emotions. This system receives video data from a camera in real time, identifies vehicles using a generative AI model, and measures and analyzes traffic volume. It also has a function for recognizing user emotions using an emotion engine and customizing the way the analysis results are presented.

[0265] Server Roles

[0266] Video data reception and preprocessing

[0267] The server receives real-time video data from the camera. This video data is captured at 24 frames per second and temporarily stored in the server's buffer. The video data is then pre-processed. This includes cropping the image frame by frame using the OpenCV library, applying a noise reduction filter, and unifying the resolution.

[0268] Vehicle Identification

[0269] The preprocessed video data is input into a generative AI model. This generative AI model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. Specifically, it uses the prompt "How many vehicles are there in this frame?" to identify the type and number of vehicles.

[0270] Saving to a database

[0271] The identified vehicle data is converted to JSON format and stored in a traffic database. The data includes timestamps and vehicle location information. The server uses a NoSQL database (e.g., MongoDB) to store this data.

[0272] Data analysis and visualization

[0273] The data stored in the traffic volume database is analyzed using an analysis tool. For analysis, the Python pandas library is used to generate statistical data, and the matplotlib library is used to visualize the data, generating graphs and heat maps showing fluctuations in traffic volume.

[0274] Device Role

[0275] Sending and Receiving Data Requests

[0276] In response to a user request, the device requests traffic volume data for a specific time period or road section from the server. This request uses the HTTP protocol, and the corresponding traffic volume data is received from the server in JSON format.

[0277] Data analysis and presentation

[0278] The received traffic volume data is analyzed on the device, using the JavaScript D3.js library to generate interactive graphs and heat maps. Furthermore, the device's emotion engine customizes the display format based on the user's emotions.

[0279] User operations

[0280] Launching the application and checking the data

[0281] The user launches the application and sets conditions for checking specific traffic volume data, for example, "traffic volume on major road A from 22:00 to 22:30 on October 1, 2023."

[0282] Emotion Recognition and Customization

[0283] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions. Based on this, the display format of traffic information and alerts can be customized. For example, if the user is feeling stressed, the device will display information in a color scheme that has a relaxing effect.

[0284] Specific examples

[0285] It receives video data in real time, preprocesses it, and then inputs a prompt to the generative AI model: "How many vehicles are there in this frame?" The resulting data is then stored in a traffic database and analyzed using analytical tools, providing traffic information to users in an easy-to-understand format.

[0286] In this way, Traffic AI Plus is a system that realizes efficient and accurate surveys and analysis of road traffic volume, and also provides customized traffic information according to user sentiment. This system will greatly contribute to the formulation of urban planning and transportation policies, and will improve user satisfaction.

[0287] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0288] Step 1:

[0289] Receiving real-time video data

[0290] The server receives real-time video data from the camera. Specifically, the camera transmits video at 24 frames per second, and the server temporarily stores the data in a buffer. The input is the video data from the camera, and the output is the video data stored in the buffer. This data is used for subsequent processing steps.

[0291] Step 2:

[0292] Video data preprocessing

[0293] The server preprocesses the video data stored in the buffer. Specific preprocessing tasks include using the OpenCV library to crop the image frame by frame, applying a noise reduction filter, and unifying the resolution to 1080p. The input is the video data in the buffer, and the output is the preprocessed image data.

[0294] Step 3:

[0295] Vehicle Identification with Generative AI Models

[0296] The server inputs the preprocessed video data into a generative AI model, which then inputs the prompt "How many vehicles are there in this frame?" to identify the vehicles. The input is the preprocessed image data and the prompt, and the output is JSON-formatted data containing the type and number of identified vehicles.

[0297] Step 4:

[0298] Data storage

[0299] The server stores the identified vehicle data in a traffic volume database. Specifically, it uses a NoSQL database (e.g., MongoDB) to store JSON data including vehicle type, number, timestamp, and location information. The input is the identified vehicle data, and the output is the traffic volume data stored in the database.

[0300] Step 5:

[0301] Data analysis and visualization

[0302] The server analyzes and visualizes the data stored in the traffic volume database. For analysis, it uses Python's pandas library to generate statistical data and the matplotlib library to create visualizations such as heat maps and graphs. The input is the data in the traffic volume database, and the output is the visual data as the analysis results.

[0303] Step 6:

[0304] Sending a Data Request

[0305] In response to a user request, the terminal requests traffic volume data for a specific time period or road section from the server. Specific conditions are entered using an HTTP request, and the corresponding data is received from the server. The input is the conditions set by the user, and the output is the traffic volume data received from the server.

[0306] Step 7:

[0307] Data reception and analysis

[0308] The terminal analyzes the traffic volume data received from the server. It uses the D3.js library to create interactive graphs and heat maps and display them to the user. The input is the traffic volume data received from the server, and the output is the visualized data.

[0309] Step 8:

[0310] Emotion-aware customization

[0311] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions. Based on this, the display format is customized. For example, if the user is feeling stressed, the data is displayed in a color scheme that has a relaxing effect. The input is the user's facial expressions and voice data, and the output is a customized display format.

[0312] Step 9:

[0313] Application launch and condition settings

[0314] The user launches the application and sets a condition to check specific traffic volume data, for example, "traffic volume on major road A from 22:00 to 22:30 on October 1, 2023." The input is the condition setting, and the output is a data request based on the condition setting.

[0315] Step 10:

[0316] Data review and decision making

[0317] The user checks the visualized traffic data displayed on the device and decides on actions as necessary, for example, changing the route to avoid heavy traffic periods. The input is the visualized traffic data, and the output is the user's decision and actions.

[0318] In this way, Traffic AI Plus conducts efficient and accurate road traffic volume surveys and analysis through each step, and also provides information that responds to the user's emotions.

[0319] (Application example 2)

[0320] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0321] Conventional road traffic volume survey and analysis systems provide real-time traffic volume information, but they have not been able to provide customized information that takes user emotions into account. Furthermore, when proposing routes for autonomous vehicles, they lack the ability to respond to user emotions, making it difficult to ensure safe and comfortable driving at all times. Given this background, there is a growing need for a system that analyzes traffic volume in real time and customizes information based on user emotions.

[0322] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving real-time video data from a camera, means for preprocessing the video data, means for identifying vehicles using a generative artificial intelligence model, means for storing the identified vehicle data in a traffic volume database, means for analyzing and visualizing the traffic volume data, means for customizing the presentation method using an emotion engine that recognizes user emotions, and means for proposing a comfortable and safe route in an autonomously driven vehicle. This makes it possible to analyze traffic volume information in real time and provide customized information according to the user's emotions.

[0323] The "means for receiving real-time video data from a camera" refers to a system or device for receiving video data acquired from a camera in real-time and transmitting the data to another processing unit.

[0324] "Means for preprocessing video data" refers to a system or device that performs preprocessing such as frame cropping, noise removal, and resolution adjustment in order to convert the video data received from the camera into a format suitable for the generative AI model.

[0325] "Means for identifying vehicles using generative artificial intelligence models" refers to software or algorithms that use generative AI models trained on large amounts of data to identify vehicles in video footage with high accuracy.

[0326] "Means for storing identified vehicle data in a traffic volume database" refers to a system or device that automatically stores vehicle data (such as vehicle type, number, and location information) identified by the generative AI model in a traffic volume database in JSON format or similar.

[0327] "Means for analyzing and visualizing traffic volume data" refers to a system or device that statistically analyzes data stored in a traffic volume database and visualizes it using visualization tools as graphs or heat maps.

[0328] "Means for customizing presentation methods using an emotion engine that recognizes user emotions" refers to software or algorithms that analyze the user's facial expressions and tone of voice to recognize emotions, and customize the way traffic information and routes are presented based on that.

[0329] "Means for proposing comfortable and safe routes within an autonomous vehicle" refers to a navigation system or algorithm that proposes and executes optimal routes for an autonomous vehicle based on traffic volume data and user emotion data.

[0330] This invention, the "Traffic AI Plus" system, analyzes road traffic volume in a highly automated manner and also recognizes user emotions to provide customized information. It is particularly focused on operation in autonomous vehicles. The system has three main components: a server, a terminal, and a user.

[0331] Server Roles and Operations

[0332] The server performs the following functions:

[0333] Video data reception and preprocessing

[0334] The server receives real-time video data from the camera. This data begins processing immediately. The video data is pre-processed into a format suitable for the generative AI model. Pre-processing includes cropping the image frame by frame, removing noise, and adjusting the resolution. This allows the generative AI model to identify vehicles with high accuracy.

[0335] Vehicle identification and storage in a database

[0336] The preprocessed video data is then fed into a generative AI model. This model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. The identified vehicle types (cars, trucks, motorcycles, etc.) and their counts are output in JSON format and stored in a traffic database. The data also includes timestamps and vehicle location information.

[0337] Data analysis and visualization

[0338] The data stored in the traffic volume database is analyzed on an ongoing basis using analytical tools. This analysis includes generating statistical data and extracting patterns. The analyzed data is then visualized using visualization tools, and displayed as graphs and heat maps showing fluctuations in traffic volume.

[0339] Terminal roles and processing

[0340] The terminal implements the following functions:

[0341] Data Request and Receipt

[0342] The terminal sends a request for traffic volume data to the server in response to a user request. The request includes a specification of a specific time period and road section. The server then retrieves the relevant data from the traffic volume database and sends it to the terminal.

[0343] Data analysis and presentation

[0344] The device analyzes the received traffic data and displays it in a user-friendly format, using statistical graphs and heat maps. Furthermore, the analysis results are customized based on the user's emotions using an emotion engine.

[0345] User roles and operations

[0346] The user performs the following operations:

[0347] Operating the application and checking the data

[0348] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section. For example, if they want to check traffic volume on a specific road at night, they set the condition "traffic volume on major road A from 22:00 to 22:30" in the app. Based on this condition, the device sends a request to the server and displays the retrieved data to the user.

[0349] Emotion recognition and customized alerts

[0350] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions, and then customizes traffic information displays and alerts accordingly. For example, if the user is nervous, the device will suggest a relaxing route, and if the user is happy, the device will suggest the shortest route.

[0351] Decision-making and action

[0352] Users can view the visualized data and make on-the-fly decisions as needed, such as choosing an alternative route to avoid heavy traffic on major road A. Urban planners can also use the insights gained to formulate transportation policies and infrastructure development plans.

[0353] Specific examples

[0354] For example, suppose a child looks sleepy while a family is on a road trip. The system recognizes the child's sleepy expression through the in-car camera and selects a quiet and comfortable route. If the parent looks excited, the system suggests the shortest route to arrive smoothly. In this way, customized information is provided that takes into account the user's emotions in real time.

[0355] Prompt Sentence Examples

[0356] "Create an application that suggests a quieter route with less traffic if the user is sleepy, and selects the shortest route if the user is excited. The data used will be real-time camera footage, the user's facial expressions, and audio data."

[0357] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0358] Step 1:

[0359] The server receives real-time video data from the camera. It takes the video stream as input and outputs video data (frame by frame). Specifically, the server monitors the camera stream and immediately captures the captured video data.

[0360] Step 2:

[0361] The server preprocesses the received video data. The input is the video data acquired in step 1, and the output is noise-removed, resolution-adjusted frames (preprocessed video data). Specifically, the server uses OpenCV to remove noise and adjust the resolution, and converts each frame into an appropriate format.

[0362] Step 3:

[0363] The server uses a generative AI model to identify vehicles from the preprocessed video data. The input is the preprocessed video data output in step 2, and the output is data on identified vehicles (vehicle type, number, and location information). Specifically, the server applies the generative AI model using Keras / TensorFlow to identify vehicles in the video with high accuracy.

[0364] Step 4:

[0365] The server stores the identified vehicle data in the traffic volume database. The input is the vehicle data output in step 3, and the updated traffic volume database is output. Specifically, the server converts the vehicle data into JSON format and automatically stores it in the database.

[0366] Step 5:

[0367] The server analyzes and visualizes traffic volume data. The input is data from the traffic volume database, and the output is analysis results and visualized data (graphs, heat maps). Specifically, analysis tools are used to generate statistical data and extract patterns, and visualization tools are used to visualize the results.

[0368] Step 6:

[0369] The terminal sends a request for traffic volume data to the server in response to a user request. The input is the request conditions from the user, and a request message is output. Specifically, the user sets conditions for specific time periods and road sections within the application, and the request is made according to those conditions.

[0370] Step 7:

[0371] The server acquires traffic volume data in response to a request from the terminal and sends it to the terminal. The input is the request message sent in step 6, and the corresponding traffic volume data is output. Specifically, the server acquires the relevant data from the traffic volume database and sends it back to the terminal.

[0372] Step 8:

[0373] The terminal analyzes the received traffic volume data and displays it in a format that is easy for the user to understand. The input is the traffic volume data acquired in step 7, and visualized data (statistical graphs, heat maps) is output. Specifically, the terminal processes the received data using analysis tools and displays it visually on the user interface.

[0374] Step 9:

[0375] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize emotions. The input is raw data from the in-car camera and microphone, and the output is the user's emotional state. Specifically, the device processes the data using an emotion recognition algorithm to determine the user's emotions in real time.

[0376] Step 10:

[0377] The device customizes the way traffic information and routes are presented based on the user's emotions. The input is the emotion data recognized in step 9 and the traffic volume data acquired in step 8, and customized route suggestions and display formats are output. Specifically, if the user is nervous, a relaxing route is suggested, and if the user is happy, the shortest route is displayed.

[0378] Step 11:

[0379] The user checks the visualized data and selects a route. The input is the visualized data and route proposals presented by the terminal, and the selected route is output. Specifically, the user looks at the application, selects the optimal route, and makes another request if necessary.

[0380] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0381] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0382] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0383] [Second embodiment]

[0384] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0385] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0386] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0387] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0388] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0389] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0390] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0391] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0392] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0393] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0394] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0395] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0396] This invention provides a system called "Traffic AI" that highly automates road traffic volume surveys. This system receives video data from cameras in real time, identifies vehicles using a generative AI model, and measures and analyzes road traffic volume. The specific roles of the server, terminal, and user, and the processing involved, are explained in detail below.

[0397] Server Roles and Operations

[0398] Video data reception and preprocessing

[0399] The server first receives real-time video data from the camera. This data is sent to the server with minimal delay, and processing begins immediately. The received video data is pre-processed into a format suitable for the generative AI model. This pre-processing includes cropping the image frame by frame, removing noise, and adjusting the resolution.

[0400] Vehicle identification and storage in a database

[0401] The preprocessed video data is then fed into a generative AI model. This model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. The identified vehicle types (cars, trucks, motorcycles, etc.) and their counts are output in JSON format and stored in a traffic database. This data also includes timestamps and vehicle location information.

[0402] Data analysis and visualization

[0403] The data stored in the traffic database is analyzed on an ongoing basis. This analysis involves generating statistical data and extracting patterns. The analyzed data is then visualized using visualization tools. Specifically, it is displayed as graphs or heat maps showing fluctuations in traffic volume.

[0404] Terminal roles and processing

[0405] Data Request and Receipt

[0406] In response to a user request, the terminal sends a request for traffic volume data to the server. The request includes a specification of a specific time period and road section. In response, the server retrieves the relevant data from the traffic volume database and sends it to the terminal.

[0407] Data analysis and presentation

[0408] The device analyzes the received traffic volume data and displays it in a user-friendly format, using statistical graphs and heat maps, allowing users to intuitively grasp the traffic situation.

[0409] User operations

[0410] Operating the application and checking the data

[0411] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section. For example, if the user wants to check traffic volume on a specific road at night, they set the condition "traffic volume on major road A from 22:00 to 22:30" in the app. Based on this condition, the device sends a request to the server and displays the retrieved data to the user.

[0412] Decision-making and action

[0413] Users can view the visualized data and make adaptive decisions as needed, such as choosing an alternative route to avoid heavy traffic on major road A. Urban planners can also use the insights gained to formulate transportation policies and infrastructure development plans.

[0414] The present invention makes it possible to carry out efficient and accurate surveys and analyses of road traffic volumes, which can be of great help in formulating urban plans and transportation policies.

[0415] The processing flow will be explained below.

[0416] Now, let me explain the program step by step.

[0417] Server Processing

[0418] Step 1:

[0419] The server receives real-time video data from the camera. The video data is transmitted at a constant frame rate, and the server receives it sequentially.

[0420] Specific operation: Receives video data from the camera at 10 fps (frames per second).

[0421] Step 2:

[0422] The server preprocesses the received video data, including cropping the image frame by frame, removing noise, and adjusting the resolution, converting it into a format suitable for the generative AI model.

[0423] Specific operation: Resize each frame to 256x256 pixels and remove noise from the image.

[0424] Step 3:

[0425] The server inputs the preprocessed data into a generative AI model to identify vehicles. The generative AI model then identifies vehicles in the video with high accuracy and determines their type (cars, trucks, motorcycles, etc.) and number.

[0426] Specific operation: In the first frame, two cars and one truck are identified, and in the second frame, one motorcycle is identified.

[0427] Step 4:

[0428] The server stores the identified vehicle data in a traffic database, including vehicle type, number, time stamp, and location information.

[0429] Specific operation: Save the data of two cars and one truck at 2023-10-01 22:05:10 in the database.

[0430] Terminal handling

[0431] Step 5:

[0432] The terminal sends a request for traffic data to the server based on the user's request, which includes a specification of a particular time period and road section.

[0433] Specific operation: Send a message to the server requesting traffic volume data for major road A from 22:00 to 22:30.

[0434] Step 6:

[0435] The terminal analyzes the traffic volume data received from the server, which includes formatting the data and statistical compilation.

[0436] Specific operation: Calculate the total number of vehicles and their breakdown on main road A from 22:00 to 22:30.

[0437] Step 7:

[0438] The device displays the analysis results in a user-friendly format, including generating graphs and heat maps showing traffic fluctuations.

[0439] Specific operation: Displays the traffic volume on major road A as a line graph, showing fluctuations by time period.

[0440] User operations

[0441] Step 8:

[0442] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section.

[0443] Specific operation: Set the condition "traffic volume on major road A from 22:00 to 22:30" on the application.

[0444] Step 9:

[0445] Users can view visualized data and make on-the-fly decisions as needed.

[0446] Specific operation: Select a route that avoids times when traffic on major road A is heavy.

[0447] These are the specific processing steps of the server, terminal, and user, which enable efficient and accurate survey and analysis of road traffic volume.

[0448] Example 1

[0449] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0450] Conventional road traffic volume surveys require manual analysis of large amounts of video data, which is time-consuming and labor-intensive. It is also difficult to accurately identify vehicles in nighttime video data. Furthermore, there is a lack of systems for analyzing and visualizing traffic volume data in real time, which hinders rapid decision-making in urban planning and transportation policies.

[0451] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0452] In this invention, the server includes a means for receiving real-time video data from the camera, a means for preprocessing the video data, and a means for identifying vehicles using a generative artificial intelligence model. This enables automatic processing of video data and highly accurate vehicle identification. It also includes a means for storing the identified vehicle data in an information storage device, a means for analyzing and visualizing traffic volume data, a means for requesting traffic volume data for a specified time period or road section and retrieving the corresponding data from the information storage device, and a means for analyzing the received data and displaying it as a statistical graph or heat map. This enables real-time analysis and visualization of traffic volume, enabling rapid decision-making in urban planning and transportation policies.

[0453] A "camera" is a device that captures video data and transmits it in real time.

[0454] "Video data" is a digital record of visual information captured by a camera.

[0455] "Preprocessing" is the process of converting video data into a format that can be used by a generative AI model, and includes processes such as noise removal and resolution adjustment.

[0456] A "generative artificial intelligence model" is a machine learning model that is trained using large amounts of data and can perform a specific task (e.g., vehicle identification) with high accuracy.

[0457] "Vehicle identification" is the process of identifying vehicles contained in video data and determining their type and location.

[0458] "Information storage device" means a digital storage system for long-term preservation of identified data.

[0459] "Traffic volume data" is a collection of information about vehicle traffic in a particular area or time period.

[0460] "Analysis" is the process of processing data using statistical methods and pattern recognition techniques to extract meaningful information.

[0461] "Visualization" means displaying the analyzed data in a visual format such as a graph or heat map.

[0462] A "statistical graph" is a diagram that visually represents numerical data and is used to show fluctuations and trends in traffic volume.

[0463] A "heat map" is a graphical tool that uses shades of color to represent data density, showing the concentration of traffic in a particular area or time period.

[0464] "User interface" refers to the screen and operating means by which a user interacts with a system and inputs and outputs information.

[0465] This invention relates to "Traffic AI," a system for highly automated road traffic volume surveys. This system receives real-time video data from cameras, identifies vehicles using generative AI models, and analyzes and visualizes traffic volume. For example, the hardware used includes cameras, servers, and terminals (e.g., mobile devices and PCs), and the software and libraries used include OpenCV, YOLOv5, Pandas, Grafana, HTTP protocol, and MongoDB.

[0466] Server Roles and Operations

[0467] The server first receives real-time video data from the camera. This data is then sent to the server via a streaming protocol. The received video data is immediately pre-processed using the OpenCV library. This pre-processing includes frame-by-frame image extraction, noise reduction, and resolution adjustment.

[0468] The preprocessed video data is input into a generative AI model (e.g., YOLOv5). This AI model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. The type and number of identified vehicles are output in JSON format along with timestamps and location information and stored in an information storage device (e.g., MongoDB).

[0469] The data stored in the traffic database is analyzed using the Pandas library. The analysis includes generating statistical data and extracting patterns, and the resulting data is visualized using visualization tools such as Grafana. Specifically, graphs and heat maps showing fluctuations in traffic volume are generated and displayed in a format that users can intuitively understand.

[0470] Terminal roles and processing

[0471] The device sends a data request to the server in response to a user request. This request includes information about a specific time period and road section. The server processes the request, retrieves the corresponding traffic data from the information storage device, and sends it to the device.

[0472] The received data is further analyzed on the device using Python libraries such as Matplotlib and Seaborn, and the data is displayed as statistical graphs and heat maps, allowing users to intuitively understand the traffic situation.

[0473] User operations

[0474] The user launches the application and sets the conditions for checking traffic volume data for a specific time period or road section. For example, the user may want to check "traffic volume on major road B from 07:00 to 09:00 on October 1, 2023," and specify this in the app. An example of this prompt is:

[0475] "Please display a graph showing the traffic volume and its fluctuations by vehicle type on major road B from 07:00 to 09:00 on October 1, 2023."

[0476] The device sends a request to the server based on these conditions and displays the retrieved data to the user. The user can then refer to the displayed graphs and heat maps and make appropriate decisions as needed. For example, they can change their departure time or choose an alternative route to avoid heavy traffic. Urban planners can also use the data to create transportation policies and infrastructure development plans.

[0477] As described above, "Traffic AI" is a system that realizes efficient and accurate surveys and analysis of road traffic volume and supports rapid decision-making in urban planning and transportation policy. By accurately performing each processing step based on the specific embodiment of this invention, advanced traffic volume management becomes possible.

[0478] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0479] Step 1:

[0480] The server receives real-time video data from the camera. The input is the video data from the camera, and the output is the video data transferred to the server. This video data is processed frame by frame and immediately moves to the next step. For example, we use the RTSP protocol to get the video stream and split it into frames using OpenCV.

[0481] Step 2:

[0482] The server preprocesses the received video data. The input is the video data received in step 1, and the output is the preprocessed video data. Preprocessing includes noise removal, resolution adjustment, and frame-by-frame image cropping. Specifically, the OpenCV library is used to remove noise with Gaussian blur and resize the video to the specified resolution.

[0483] Step 3:

[0484] The server inputs the preprocessed video data into a generative AI model. The input is the preprocessed video data, and the output is the identified vehicle data. The generative AI model (e.g., YOLOv5) identifies vehicles in the video with high accuracy and outputs the vehicle type and location information. For example, the YOLOv5 model can be used to identify cars, trucks, motorcycles, etc.

[0485] Step 4:

[0486] The server stores the identified vehicle data in an information storage device. The input is the identified vehicle data, and the output is the data stored in a database. The data is stored in JSON format along with timestamps and location information. Specifically, MongoDB is used to store the identification results.

[0487] Step 5:

[0488] The server analyzes the data stored in the traffic volume database. The input is the vehicle data stored in the database, and the output is the analysis results. The analysis includes generating statistical data and extracting patterns. Specifically, Pandas is used to aggregate the data and extract fluctuations and trends in traffic volume by time period.

[0489] Step 6:

[0490] The server visualizes the analysis results. The input is the analysis results, and the output is the visualized data. The visualization includes graphs and heat maps showing traffic volume fluctuations. Specifically, Grafana is used to create interactive dashboards.

[0491] Step 7:

[0492] The terminal sends a data request from the user to the server. The input is the user's request conditions (e.g., a specific time period or road section), and the output is the data request to the server. The request is sent using the HTTP protocol.

[0493] Step 8:

[0494] The terminal analyzes the traffic data received from the server. The input is the traffic data from the server, and the output is the analysis results. The analysis includes generating statistical graphs and heat maps. Specifically, the data is visualized using Python's Matplotlib and Seaborn.

[0495] Step 9:

[0496] The terminal displays the analysis results to the user. The input is the analysis results, and the output is visualization data displayed to the user. Dashboards, statistical graphs, and heat maps are used for display, allowing the user to intuitively understand the traffic situation.

[0497] Step 10:

[0498] The user launches the application and sets the conditions for checking traffic volume data. The input is the user's requested conditions, and the output is the condition settings to the device. For example, if the user wants to check "traffic volume on major road B from 07:00 to 09:00 on October 1, 2023," they specify this in the app.

[0499] Step 11:

[0500] Users make appropriate decisions based on the displayed traffic data. The input is the displayed visualization data, and the output is the user's decision. For example, they may change their departure time to avoid heavy traffic hours or choose an alternative route. Urban planners also use the data to create transportation policies and infrastructure development plans.

[0501] (Application example 1)

[0502] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0503] Conventional road traffic volume survey systems require a great deal of manpower and time to grasp traffic volume and identify vehicles, making real-time analysis difficult, especially when collecting data at night. Furthermore, they are inadequate in providing optimal detour routes based on the analyzed data, and in presenting information intuitively to drivers. New technologies are needed to solve these issues.

[0504] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0505] In this invention, the server includes means for receiving real-time video data from a camera, means for preprocessing the video data, means for identifying vehicles using a generative artificial intelligence model, means for storing the identified vehicle data in a traffic volume database, means for analyzing and visualizing the traffic volume data, and means for proposing an optimal route based on the analyzed traffic volume data, thereby enabling real-time traffic volume understanding, provision of optimal detour routes, and intuitive information presentation.

[0506] A "camera" is a device for acquiring real-time image data on a road.

[0507] "Real-time video data" refers to current video information continuously acquired through a camera.

[0508] "Video data preprocessing" refers to the process of converting raw data acquired from a camera into a format suitable for a generative artificial intelligence model.

[0509] A "generative artificial intelligence model" is an AI model trained using large amounts of data and capable of identifying vehicles in video footage.

[0510] "Vehicle identification" is the process of recognizing and classifying vehicles such as cars, trucks, and motorcycles from video data.

[0511] A "traffic database" is a data storage system for storing identified vehicle data for later analysis.

[0512] "Traffic data analysis" refers to the statistical processing of data stored in traffic databases to extract meaningful patterns and insights.

[0513] "Visualization" refers to the presentation of analyzed traffic data in a visual format such as a graph or heat map.

[0514] "Optimal route proposal" means selecting and presenting the most efficient and least congested route based on real-time traffic analysis data.

[0515] This invention provides a system called "Traffic Navigator" based on "Traffic AI," which highly automates road traffic volume surveys. This system receives video data from cameras in real time, identifies vehicles using a generative AI model, measures and analyzes traffic volume, and proposes optimal routes.

[0516] Server Roles and Operations

[0517] The server first receives real-time video data from the camera. This data is continuously acquired through the camera and processing begins immediately. The received video data is pre-processed into a format suitable for the generative AI model. This pre-processing includes cropping the image for each frame, removing noise, and adjusting the resolution. Libraries such as OpenCV are used for pre-processing.

[0518] The preprocessed video data is then fed into a generative AI model. This generative AI model is trained on a large amount of data and is able to identify vehicles in the video with high accuracy. The identified vehicle types (cars, trucks, motorcycles, etc.) and their counts are output in JSON format and stored in a traffic database. This data also includes timestamps and vehicle location information.

[0519] The data stored in the traffic database is analyzed on an ongoing basis. This analysis involves generating statistical data and extracting patterns. The analyzed data is then visualized using visualization tools. Specifically, it is displayed as graphs or heat maps showing fluctuations in traffic volume. Libraries such as Matplotlib are used at this stage.

[0520] The server then performs calculations to suggest optimal routes based on the analyzed traffic data, and presents the most efficient detour route to the driver based on the real-time analysis results.

[0521] Terminal roles and processing

[0522] The device sends a request for traffic volume data to the server in response to a user request. The request includes a specific time period and road section. The server then retrieves the relevant data from the traffic volume database and sends it to the device. The device then analyzes the received traffic volume data and displays it in an easy-to-understand format, such as a statistical graph or heat map.

[0523] User operations

[0524] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section. For example, if a user wants to check traffic volume on a specific road at night, they can set the condition "traffic volume on major road A from 22:00 to 22:30" on the app. Based on these conditions, the server provides analysis results and suggests the optimal detour route.

[0525] Specific examples

[0526] For example, it can be seen in real time that traffic volume on major road A is heavy at night. In this case, the system can immediately suggest optimal detour routes to help drivers avoid traffic jams. Urban planners can also use heat maps and statistical graphs to intuitively understand the traffic situation in specific areas and formulate appropriate traffic policies.

[0527] Prompt Sentence Examples

[0528] "Analyze the traffic volume on major road A from 10:00 PM to 10:30 PM at night and generate a heat map."

[0529] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0530] Step 1:

[0531] The server receives real-time video data from the camera. The video information continuously acquired through the camera is input and immediately processed. The received video data is used in subsequent processing steps.

[0532] Step 2:

[0533] The server preprocesses the received video data. This preprocessing involves cropping the image frame by frame, removing noise, and adjusting the resolution. Using an image processing library such as OpenCV, the quality of the input data is converted into a format suitable for the generative AI model. The preprocessed image data is then output.

[0534] Step 3:

[0535] The server inputs the preprocessed video data into a generative AI model to identify vehicles. The generative AI model uses a large amount of training data to perform highly accurate vehicle identification. The model outputs the identified vehicle type (car, truck, motorcycle, etc.) and its number, and this information is used in subsequent steps.

[0536] Step 4:

[0537] The server stores the identified vehicle data in a traffic database. The identification results are stored in JSON format, including timestamps and vehicle location information. The identified vehicle data is given as input and recorded in the traffic database as output.

[0538] Step 5:

[0539] The server analyzes the data stored in the traffic database. This analysis includes generating statistics and extracting patterns. The input data is the contents of the traffic database, and the output data is the analysis results. Statistical calculation methods are used.

[0540] Step 6:

[0541] The server visualizes the analyzed data. Using a visualization tool, the data is output as graphs and heat maps showing fluctuations in traffic volume. The Matplotlib library is used to visually display the analysis results.

[0542] Step 7:

[0543] The server proposes the optimal route based on the analysis results. Based on the analysis results in real time, it calculates the most efficient detour route for the driver. The input data is the analysis results, and the output data is the proposed optimal route.

[0544] Step 8:

[0545] The terminal receives the data sent from the server and requests specific traffic volume data according to the user's request. The request includes the specification of a specific time period or road section, and obtains the corresponding data from the server.

[0546] Step 9:

[0547] The device analyzes the traffic volume data it receives and displays it in statistical graphs and heat maps, allowing users to intuitively understand the data and assisting them in selecting the optimal route.

[0548] Step 10:

[0549] The user selects traffic volume data for a specific time period or road section through the application, and the device sends a request to the server. Based on the acquired data, the user can intuitively understand the traffic situation and act accordingly.

[0550] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0551] This invention provides a system called "Traffic AI Plus," which combines a highly automated system for road traffic volume surveys with an emotion engine that recognizes user emotions. This system receives video data from a camera in real time, identifies vehicles using a generative AI model, and measures and analyzes traffic volume. The emotion engine also recognizes user emotions and customizes the presentation of analysis results. The specific roles of the server, terminal, and user, as well as the processing involved, are described in detail below.

[0552] Server Roles and Operations

[0553] Video data reception and preprocessing

[0554] The server receives real-time video data from the camera, which is then processed immediately. The received video data is pre-processed into a format suitable for the generative AI model. Pre-processing includes cropping the image frame by frame, removing noise, and adjusting the resolution.

[0555] Vehicle identification and storage in a database

[0556] The preprocessed video data is then fed into a generative AI model. This model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. The identified vehicle types (cars, trucks, motorcycles, etc.) and their counts are output in JSON format and stored in a traffic database. The data also includes timestamps and vehicle location information.

[0557] Data analysis and visualization

[0558] The data stored in the traffic volume database is analyzed on an ongoing basis using analytical tools. This analysis includes generating statistical data and extracting patterns. The analyzed data is then visualized using visualization tools, and displayed as graphs and heat maps showing fluctuations in traffic volume.

[0559] Terminal roles and processing

[0560] Data Request and Receipt

[0561] The terminal sends a request for traffic volume data to the server in response to a user request. The request includes a specification of a specific time period and road section. The server then retrieves the relevant data from the traffic volume database and sends it to the terminal.

[0562] Data analysis and presentation

[0563] The device analyzes the received traffic data and displays it in a user-friendly format, using statistical graphs and heat maps. Furthermore, an emotion engine customizes the analysis results to reflect the user's emotions.

[0564] User operations

[0565] Operating the application and checking the data

[0566] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section. For example, if the user wants to check traffic volume on a specific road at night, they set the condition "traffic volume on major road A from 22:00 to 22:30" in the app. Based on this condition, the device sends a request to the server and displays the retrieved data to the user.

[0567] Emotion recognition and customized alerts

[0568] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions, and then customizes traffic information displays and alerts accordingly. For example, if the user is nervous, the device will suggest a relaxing route, and if the user is happy, the device will suggest the shortest route.

[0569] Decision-making and action

[0570] Users can view the visualized data and make on-the-fly decisions as needed, such as choosing an alternative route to avoid heavy traffic on major road A. Urban planners can also use the insights gained to formulate transportation policies and infrastructure development plans.

[0571] This invention not only enables efficient and accurate survey and analysis of road traffic volume, but also enables the provision of customized traffic information that takes user emotions into consideration, which will greatly support the formulation of urban planning and transportation policies and improve user satisfaction.

[0572] The processing flow will be explained below.

[0573] Server Processing

[0574] Step 1:

[0575] The server receives real-time video data from the camera. The video data is transmitted at a constant frame rate, and the server receives it sequentially.

[0576] Specific operation: Receives video data from the camera at 10 fps (frames per second).

[0577] Step 2:

[0578] The server pre-processes the received video data, including cutting out the image frame by frame, removing noise, and adjusting the resolution.

[0579] Specific operation: Resize each frame to 256x256 pixels and remove noise from the image.

[0580] Step 3:

[0581] The server inputs the preprocessed data into a generative AI model to identify vehicles. The generative AI model then identifies vehicles in the video with high accuracy and determines their type (cars, trucks, motorcycles, etc.) and number.

[0582] Specific operation: In the first frame, two cars and one truck are identified, and in the second frame, one motorcycle is identified.

[0583] Step 4:

[0584] The server stores the identified vehicle data in a traffic database, including vehicle type, number, time stamp, and location information.

[0585] Specific operation: Save the data of two cars and one truck at 2023-10-01 22:05:10 in the database.

[0586] Terminal handling

[0587] Step 5:

[0588] The terminal sends a request for traffic data to the server based on the user's request, which includes a specification of a particular time period and road section.

[0589] Specific operation: Send a message to the server requesting traffic volume data for major road A from 22:00 to 22:30.

[0590] Step 6:

[0591] The terminal analyzes the traffic volume data received from the server, which includes formatting the data and statistical compilation.

[0592] Specific operation: Calculate the total number of vehicles and their breakdown on main road A from 22:00 to 22:30.

[0593] Step 7:

[0594] The device displays the analysis results in a user-friendly format, including generating graphs and heat maps showing traffic fluctuations.

[0595] Specific operation: Displays the traffic volume on major road A as a line graph, showing fluctuations by time period.

[0596] Step 8:

[0597] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions, and displays and alerts are customized based on the recognized emotions.

[0598] What it does: If the user is nervous, provide them with directions and travel information to help them relax.

[0599] User operations

[0600] Step 9:

[0601] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section.

[0602] Specific operation: Set the condition "traffic volume on major road A from 22:00 to 22:30" on the application.

[0603] Step 10:

[0604] Users can view visualized data and make on-the-fly decisions as needed.

[0605] Specific operation: Select a route that avoids times when traffic on major road A is heavy.

[0606] These are the specific processing steps of the server, terminal, and user. This not only enables efficient and accurate survey and analysis of road traffic volume, but also provides customized traffic information that takes user emotions into consideration.

[0607] Example 2

[0608] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0609] While conventional road traffic volume survey systems can measure traffic volume using real-time video data, they have the problem of being unable to provide customized information that takes user emotions into account.Furthermore, they have a problem of not being able to identify vehicles with a high degree of accuracy even in nighttime video data.

[0610] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0611] In this invention, the server includes means for receiving real-time video data from a camera, means for preprocessing the video data, means for identifying vehicles using a generative artificial intelligence model, means for storing the identified vehicle data in a traffic volume database, means for analyzing and visualizing the traffic volume data, and means for recognizing a user's emotions and customizing the presentation method of the analysis results based on the emotions. This not only enables efficient and accurate road traffic volume survey and analysis, but also enables the provision of customized traffic information that takes the user's emotions into consideration.

[0612] A "camera" is a photographing device for capturing video data in real time.

[0613] "Real-time video data" refers to data that is obtained and transmitted immediately as video information at the current time.

[0614] "Preprocessing" refers to a series of operations that convert video data into a format suitable for subsequent analysis and recognition.

[0615] A "generative artificial intelligence model" is a machine learning model that learns from data and performs a specific task (e.g., vehicle identification).

[0616] "Means for identifying vehicles" refers to technologies or algorithms for detecting vehicles in video data and identifying their type and number.

[0617] "Vehicle data" refers to data including the type, number, timestamp, location information, etc. of identified vehicles.

[0618] A "traffic database" is a database system for storing identified vehicle data.

[0619] "Analysis" refers to the process of generating statistical data and extracting patterns based on traffic volume data.

[0620] "Visualization" refers to displaying analyzed data in a visual format such as a graph or heat map.

[0621] "Means for recognizing user emotions" refers to technology or devices that analyze emotions from the user's facial expressions and tone of voice.

[0622] "Customization means" refers to technologies and algorithms that change the way information is presented and the format of the display based on the user's emotions.

[0623] This invention relates to a system called "Traffic AI Plus," which combines a highly automated system for road traffic volume surveys with a function for recognizing user emotions. This system receives video data from a camera in real time, identifies vehicles using a generative AI model, and measures and analyzes traffic volume. It also has a function for recognizing user emotions using an emotion engine and customizing the way the analysis results are presented.

[0624] Server Roles

[0625] Video data reception and preprocessing

[0626] The server receives real-time video data from the camera. This video data is captured at 24 frames per second and temporarily stored in the server's buffer. The video data is then pre-processed. This includes cropping the image frame by frame using the OpenCV library, applying a noise reduction filter, and unifying the resolution.

[0627] Vehicle Identification

[0628] The preprocessed video data is input into a generative AI model. This generative AI model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. Specifically, it uses the prompt "How many vehicles are there in this frame?" to identify the type and number of vehicles.

[0629] Saving to a database

[0630] The identified vehicle data is converted to JSON format and stored in a traffic database. The data includes timestamps and vehicle location information. The server uses a NoSQL database (e.g., MongoDB) to store this data.

[0631] Data analysis and visualization

[0632] The data stored in the traffic volume database is analyzed using an analysis tool. For analysis, the Python pandas library is used to generate statistical data, and the matplotlib library is used to visualize the data, generating graphs and heat maps showing fluctuations in traffic volume.

[0633] Device Role

[0634] Sending and Receiving Data Requests

[0635] In response to a user request, the device requests traffic volume data for a specific time period or road section from the server. This request uses the HTTP protocol, and the corresponding traffic volume data is received from the server in JSON format.

[0636] Data analysis and presentation

[0637] The received traffic volume data is analyzed on the device, using the JavaScript D3.js library to generate interactive graphs and heat maps. Furthermore, the device's emotion engine customizes the display format based on the user's emotions.

[0638] User operations

[0639] Launching the application and checking the data

[0640] The user launches the application and sets conditions for checking specific traffic volume data, for example, "traffic volume on major road A from 22:00 to 22:30 on October 1, 2023."

[0641] Emotion Recognition and Customization

[0642] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions. Based on this, the display format of traffic information and alerts can be customized. For example, if the user is feeling stressed, the device will display information in a color scheme that has a relaxing effect.

[0643] Specific examples

[0644] It receives video data in real time, preprocesses it, and then inputs a prompt to the generative AI model: "How many vehicles are there in this frame?" The resulting data is then stored in a traffic database and analyzed using analytical tools, providing traffic information to users in an easy-to-understand format.

[0645] In this way, Traffic AI Plus is a system that realizes efficient and accurate surveys and analysis of road traffic volume, and also provides customized traffic information according to user sentiment. This system will greatly contribute to the formulation of urban planning and transportation policies, and will improve user satisfaction.

[0646] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0647] Step 1:

[0648] Receiving real-time video data

[0649] The server receives real-time video data from the camera. Specifically, the camera transmits video at 24 frames per second, and the server temporarily stores the data in a buffer. The input is the video data from the camera, and the output is the video data stored in the buffer. This data is used for subsequent processing steps.

[0650] Step 2:

[0651] Video data preprocessing

[0652] The server preprocesses the video data stored in the buffer. Specific preprocessing tasks include using the OpenCV library to crop the image frame by frame, applying a noise reduction filter, and unifying the resolution to 1080p. The input is the video data in the buffer, and the output is the preprocessed image data.

[0653] Step 3:

[0654] Vehicle Identification with Generative AI Models

[0655] The server inputs the preprocessed video data into a generative AI model, which then inputs the prompt "How many vehicles are there in this frame?" to identify the vehicles. The input is the preprocessed image data and the prompt, and the output is JSON-formatted data containing the type and number of identified vehicles.

[0656] Step 4:

[0657] Data storage

[0658] The server stores the identified vehicle data in a traffic volume database. Specifically, it uses a NoSQL database (e.g., MongoDB) to store JSON data including vehicle type, number, timestamp, and location information. The input is the identified vehicle data, and the output is the traffic volume data stored in the database.

[0659] Step 5:

[0660] Data analysis and visualization

[0661] The server analyzes and visualizes the data stored in the traffic volume database. For analysis, it uses Python's pandas library to generate statistical data and the matplotlib library to create visualizations such as heat maps and graphs. The input is the data in the traffic volume database, and the output is the visual data as the analysis results.

[0662] Step 6:

[0663] Sending a Data Request

[0664] In response to a user request, the terminal requests traffic volume data for a specific time period or road section from the server. Specific conditions are entered using an HTTP request, and the corresponding data is received from the server. The input is the conditions set by the user, and the output is the traffic volume data received from the server.

[0665] Step 7:

[0666] Data reception and analysis

[0667] The terminal analyzes the traffic volume data received from the server. It uses the D3.js library to create interactive graphs and heat maps and display them to the user. The input is the traffic volume data received from the server, and the output is the visualized data.

[0668] Step 8:

[0669] Emotion-aware customization

[0670] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions. Based on this, the display format is customized. For example, if the user is feeling stressed, the data is displayed in a color scheme that has a relaxing effect. The input is the user's facial expressions and voice data, and the output is a customized display format.

[0671] Step 9:

[0672] Application launch and condition settings

[0673] The user launches the application and sets a condition to check specific traffic volume data, for example, "traffic volume on major road A from 22:00 to 22:30 on October 1, 2023." The input is the condition setting, and the output is a data request based on the condition setting.

[0674] Step 10:

[0675] Data review and decision making

[0676] The user checks the visualized traffic data displayed on the device and decides on actions as necessary, for example, changing the route to avoid heavy traffic periods. The input is the visualized traffic data, and the output is the user's decision and actions.

[0677] In this way, Traffic AI Plus conducts efficient and accurate road traffic volume surveys and analysis through each step, and also provides information that responds to the user's emotions.

[0678] (Application example 2)

[0679] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0680] Conventional road traffic volume survey and analysis systems provide real-time traffic volume information, but they have not been able to provide customized information that takes user emotions into account. Furthermore, when proposing routes for autonomous vehicles, they lack the ability to respond to user emotions, making it difficult to ensure safe and comfortable driving at all times. Given this background, there is a growing need for a system that analyzes traffic volume in real time and customizes information based on user emotions.

[0681] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving real-time video data from a camera, means for preprocessing the video data, means for identifying vehicles using a generative artificial intelligence model, means for storing the identified vehicle data in a traffic volume database, means for analyzing and visualizing the traffic volume data, means for customizing the presentation method using an emotion engine that recognizes user emotions, and means for proposing a comfortable and safe route in an autonomously driven vehicle. This makes it possible to analyze traffic volume information in real time and provide customized information according to the user's emotions.

[0682] The "means for receiving real-time video data from a camera" refers to a system or device for receiving video data acquired from a camera in real-time and transmitting the data to another processing unit.

[0683] "Means for preprocessing video data" refers to a system or device that performs preprocessing such as frame cropping, noise removal, and resolution adjustment in order to convert the video data received from the camera into a format suitable for the generative AI model.

[0684] "Means for identifying vehicles using generative artificial intelligence models" refers to software or algorithms that use generative AI models trained on large amounts of data to identify vehicles in video footage with high accuracy.

[0685] "Means for storing identified vehicle data in a traffic volume database" refers to a system or device that automatically stores vehicle data (such as vehicle type, number, and location information) identified by the generative AI model in a traffic volume database in JSON format or similar.

[0686] "Means for analyzing and visualizing traffic volume data" refers to a system or device that statistically analyzes data stored in a traffic volume database and visualizes it using visualization tools as graphs or heat maps.

[0687] "Means for customizing presentation methods using an emotion engine that recognizes user emotions" refers to software or algorithms that analyze the user's facial expressions and tone of voice to recognize emotions, and customize the way traffic information and routes are presented based on that.

[0688] "Means for proposing comfortable and safe routes within an autonomous vehicle" refers to a navigation system or algorithm that proposes and executes optimal routes for an autonomous vehicle based on traffic volume data and user emotion data.

[0689] This invention, the "Traffic AI Plus" system, analyzes road traffic volume in a highly automated manner and also recognizes user emotions to provide customized information. It is particularly focused on operation in autonomous vehicles. The system has three main components: a server, a terminal, and a user.

[0690] Server Roles and Operations

[0691] The server performs the following functions:

[0692] Video data reception and preprocessing

[0693] The server receives real-time video data from the camera. This data begins processing immediately. The video data is pre-processed into a format suitable for the generative AI model. Pre-processing includes cropping the image frame by frame, removing noise, and adjusting the resolution. This allows the generative AI model to identify vehicles with high accuracy.

[0694] Vehicle identification and storage in a database

[0695] The preprocessed video data is then fed into a generative AI model. This model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. The identified vehicle types (cars, trucks, motorcycles, etc.) and their counts are output in JSON format and stored in a traffic database. The data also includes timestamps and vehicle location information.

[0696] Data analysis and visualization

[0697] The data stored in the traffic volume database is analyzed on an ongoing basis using analytical tools. This analysis includes generating statistical data and extracting patterns. The analyzed data is then visualized using visualization tools, and displayed as graphs and heat maps showing fluctuations in traffic volume.

[0698] Terminal roles and processing

[0699] The terminal implements the following functions:

[0700] Data Request and Receipt

[0701] The terminal sends a request for traffic volume data to the server in response to a user request. The request includes a specification of a specific time period and road section. The server then retrieves the relevant data from the traffic volume database and sends it to the terminal.

[0702] Data analysis and presentation

[0703] The device analyzes the received traffic data and displays it in a user-friendly format, using statistical graphs and heat maps. Furthermore, the analysis results are customized based on the user's emotions using an emotion engine.

[0704] User roles and operations

[0705] The user performs the following operations:

[0706] Operating the application and checking the data

[0707] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section. For example, if they want to check traffic volume on a specific road at night, they set the condition "traffic volume on major road A from 22:00 to 22:30" in the app. Based on this condition, the device sends a request to the server and displays the retrieved data to the user.

[0708] Emotion recognition and customized alerts

[0709] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions, and then customizes traffic information displays and alerts accordingly. For example, if the user is nervous, the device will suggest a relaxing route, and if the user is happy, the device will suggest the shortest route.

[0710] Decision-making and action

[0711] Users can view the visualized data and make on-the-fly decisions as needed, such as choosing an alternative route to avoid heavy traffic on major road A. Urban planners can also use the insights gained to formulate transportation policies and infrastructure development plans.

[0712] Specific examples

[0713] For example, suppose a child looks sleepy while a family is on a road trip. The system recognizes the child's sleepy expression through the in-car camera and selects a quiet and comfortable route. If the parent looks excited, the system suggests the shortest route to arrive smoothly. In this way, customized information is provided that takes into account the user's emotions in real time.

[0714] Prompt Sentence Examples

[0715] "Create an application that suggests a quieter route with less traffic if the user is sleepy, and selects the shortest route if the user is excited. The data used will be real-time camera footage, the user's facial expressions, and audio data."

[0716] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0717] Step 1:

[0718] The server receives real-time video data from the camera. It takes the video stream as input and outputs video data (frame by frame). Specifically, the server monitors the camera stream and immediately captures the captured video data.

[0719] Step 2:

[0720] The server preprocesses the received video data. The input is the video data acquired in step 1, and the output is noise-removed, resolution-adjusted frames (preprocessed video data). Specifically, the server uses OpenCV to remove noise and adjust the resolution, and converts each frame into an appropriate format.

[0721] Step 3:

[0722] The server uses a generative AI model to identify vehicles from the preprocessed video data. The input is the preprocessed video data output in step 2, and the output is data on identified vehicles (vehicle type, number, and location information). Specifically, the server applies the generative AI model using Keras / TensorFlow to identify vehicles in the video with high accuracy.

[0723] Step 4:

[0724] The server stores the identified vehicle data in the traffic volume database. The input is the vehicle data output in step 3, and the updated traffic volume database is output. Specifically, the server converts the vehicle data into JSON format and automatically stores it in the database.

[0725] Step 5:

[0726] The server analyzes and visualizes traffic volume data. The input is data from the traffic volume database, and the output is analysis results and visualized data (graphs, heat maps). Specifically, analysis tools are used to generate statistical data and extract patterns, and visualization tools are used to visualize the results.

[0727] Step 6:

[0728] The terminal sends a request for traffic volume data to the server in response to a user request. The input is the request conditions from the user, and a request message is output. Specifically, the user sets conditions for specific time periods and road sections within the application, and the request is made according to those conditions.

[0729] Step 7:

[0730] The server acquires traffic volume data in response to a request from the terminal and sends it to the terminal. The input is the request message sent in step 6, and the corresponding traffic volume data is output. Specifically, the server acquires the relevant data from the traffic volume database and sends it back to the terminal.

[0731] Step 8:

[0732] The terminal analyzes the received traffic volume data and displays it in a format that is easy for the user to understand. The input is the traffic volume data acquired in step 7, and visualized data (statistical graphs, heat maps) is output. Specifically, the terminal processes the received data using analysis tools and displays it visually on the user interface.

[0733] Step 9:

[0734] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize emotions. The input is raw data from the in-car camera and microphone, and the output is the user's emotional state. Specifically, the device processes the data using an emotion recognition algorithm to determine the user's emotions in real time.

[0735] Step 10:

[0736] The device customizes the way traffic information and routes are presented based on the user's emotions. The input is the emotion data recognized in step 9 and the traffic volume data acquired in step 8, and customized route suggestions and display formats are output. Specifically, if the user is nervous, a relaxing route is suggested, and if the user is happy, the shortest route is displayed.

[0737] Step 11:

[0738] The user checks the visualized data and selects a route. The input is the visualized data and route proposals presented by the terminal, and the selected route is output. Specifically, the user looks at the application, selects the optimal route, and makes another request if necessary.

[0739] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0740] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0741] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0742] [Third embodiment]

[0743] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0744] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0745] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0746] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0747] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0748] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0749] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0750] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0751] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0752] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0753] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0754] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0755] This invention provides a system called "Traffic AI" that highly automates road traffic volume surveys. This system receives video data from cameras in real time, identifies vehicles using a generative AI model, and measures and analyzes road traffic volume. The specific roles of the server, terminal, and user, and the processing involved, are explained in detail below.

[0756] Server Roles and Operations

[0757] Video data reception and preprocessing

[0758] The server first receives real-time video data from the camera. This data is sent to the server with minimal delay, and processing begins immediately. The received video data is pre-processed into a format suitable for the generative AI model. This pre-processing includes cropping the image frame by frame, removing noise, and adjusting the resolution.

[0759] Vehicle identification and storage in a database

[0760] The preprocessed video data is then fed into a generative AI model. This model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. The identified vehicle types (cars, trucks, motorcycles, etc.) and their counts are output in JSON format and stored in a traffic database. This data also includes timestamps and vehicle location information.

[0761] Data analysis and visualization

[0762] The data stored in the traffic database is analyzed on an ongoing basis. This analysis involves generating statistical data and extracting patterns. The analyzed data is then visualized using visualization tools. Specifically, it is displayed as graphs or heat maps showing fluctuations in traffic volume.

[0763] Terminal roles and processing

[0764] Data Request and Receipt

[0765] In response to a user request, the terminal sends a request for traffic volume data to the server. The request includes a specification of a specific time period and road section. In response, the server retrieves the relevant data from the traffic volume database and sends it to the terminal.

[0766] Data analysis and presentation

[0767] The device analyzes the received traffic volume data and displays it in a user-friendly format, using statistical graphs and heat maps, allowing users to intuitively grasp the traffic situation.

[0768] User operations

[0769] Operating the application and checking the data

[0770] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section. For example, if the user wants to check traffic volume on a specific road at night, they set the condition "traffic volume on major road A from 22:00 to 22:30" in the app. Based on this condition, the device sends a request to the server and displays the retrieved data to the user.

[0771] Decision-making and action

[0772] Users can view the visualized data and make adaptive decisions as needed, such as choosing an alternative route to avoid heavy traffic on major road A. Urban planners can also use the insights gained to formulate transportation policies and infrastructure development plans.

[0773] The present invention makes it possible to carry out efficient and accurate surveys and analyses of road traffic volumes, which can be of great help in formulating urban plans and transportation policies.

[0774] The processing flow will be explained below.

[0775] Now, let me explain the program step by step.

[0776] Server Processing

[0777] Step 1:

[0778] The server receives real-time video data from the camera. The video data is transmitted at a constant frame rate, and the server receives it sequentially.

[0779] Specific operation: Receives video data from the camera at 10 fps (frames per second).

[0780] Step 2:

[0781] The server preprocesses the received video data, including cropping the image frame by frame, removing noise, and adjusting the resolution, converting it into a format suitable for the generative AI model.

[0782] Specific operation: Resize each frame to 256x256 pixels and remove noise from the image.

[0783] Step 3:

[0784] The server inputs the preprocessed data into a generative AI model to identify vehicles. The generative AI model then identifies vehicles in the video with high accuracy and determines their type (cars, trucks, motorcycles, etc.) and number.

[0785] Specific operation: In the first frame, two cars and one truck are identified, and in the second frame, one motorcycle is identified.

[0786] Step 4:

[0787] The server stores the identified vehicle data in a traffic database, including vehicle type, number, time stamp, and location information.

[0788] Specific operation: Save the data of two cars and one truck at 2023-10-01 22:05:10 in the database.

[0789] Terminal handling

[0790] Step 5:

[0791] The terminal sends a request for traffic data to the server based on the user's request, which includes a specification of a particular time period and road section.

[0792] Specific operation: Send a message to the server requesting traffic volume data for major road A from 22:00 to 22:30.

[0793] Step 6:

[0794] The terminal analyzes the traffic volume data received from the server, which includes formatting the data and statistical compilation.

[0795] Specific operation: Calculate the total number of vehicles and their breakdown on main road A from 22:00 to 22:30.

[0796] Step 7:

[0797] The device displays the analysis results in a user-friendly format, including generating graphs and heat maps showing traffic fluctuations.

[0798] Specific operation: Displays the traffic volume on major road A as a line graph, showing fluctuations by time period.

[0799] User operations

[0800] Step 8:

[0801] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section.

[0802] Specific operation: Set the condition "traffic volume on major road A from 22:00 to 22:30" on the application.

[0803] Step 9:

[0804] Users can view visualized data and make on-the-fly decisions as needed.

[0805] Specific operation: Select a route that avoids times when traffic on major road A is heavy.

[0806] These are the specific processing steps of the server, terminal, and user, which enable efficient and accurate survey and analysis of road traffic volume.

[0807] Example 1

[0808] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0809] Conventional road traffic volume surveys require manual analysis of large amounts of video data, which is time-consuming and labor-intensive. It is also difficult to accurately identify vehicles in nighttime video data. Furthermore, there is a lack of systems for analyzing and visualizing traffic volume data in real time, which hinders rapid decision-making in urban planning and transportation policies.

[0810] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0811] In this invention, the server includes a means for receiving real-time video data from the camera, a means for preprocessing the video data, and a means for identifying vehicles using a generative artificial intelligence model. This enables automatic processing of video data and highly accurate vehicle identification. It also includes a means for storing the identified vehicle data in an information storage device, a means for analyzing and visualizing traffic volume data, a means for requesting traffic volume data for a specified time period or road section and retrieving the corresponding data from the information storage device, and a means for analyzing the received data and displaying it as a statistical graph or heat map. This enables real-time analysis and visualization of traffic volume, enabling rapid decision-making in urban planning and transportation policies.

[0812] A "camera" is a device that captures video data and transmits it in real time.

[0813] "Video data" is a digital record of visual information captured by a camera.

[0814] "Preprocessing" is the process of converting video data into a format that can be used by a generative AI model, and includes processes such as noise removal and resolution adjustment.

[0815] A "generative artificial intelligence model" is a machine learning model that is trained using large amounts of data and can perform a specific task (e.g., vehicle identification) with high accuracy.

[0816] "Vehicle identification" is the process of identifying vehicles contained in video data and determining their type and location.

[0817] "Information storage device" means a digital storage system for long-term preservation of identified data.

[0818] "Traffic volume data" is a collection of information about vehicle traffic in a particular area or time period.

[0819] "Analysis" is the process of processing data using statistical methods and pattern recognition techniques to extract meaningful information.

[0820] "Visualization" means displaying the analyzed data in a visual format such as a graph or heat map.

[0821] A "statistical graph" is a diagram that visually represents numerical data and is used to show fluctuations and trends in traffic volume.

[0822] A "heat map" is a graphical tool that uses shades of color to represent data density, showing the concentration of traffic in a particular area or time period.

[0823] "User interface" refers to the screen and operating means by which a user interacts with a system and inputs and outputs information.

[0824] This invention relates to "Traffic AI," a system for highly automated road traffic volume surveys. This system receives real-time video data from cameras, identifies vehicles using generative AI models, and analyzes and visualizes traffic volume. For example, the hardware used includes cameras, servers, and terminals (e.g., mobile devices and PCs), and the software and libraries used include OpenCV, YOLOv5, Pandas, Grafana, HTTP protocol, and MongoDB.

[0825] Server Roles and Operations

[0826] The server first receives real-time video data from the camera. This data is then sent to the server via a streaming protocol. The received video data is immediately pre-processed using the OpenCV library. This pre-processing includes frame-by-frame image extraction, noise reduction, and resolution adjustment.

[0827] The preprocessed video data is input into a generative AI model (e.g., YOLOv5). This AI model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. The type and number of identified vehicles are output in JSON format along with timestamps and location information and stored in an information storage device (e.g., MongoDB).

[0828] The data stored in the traffic database is analyzed using the Pandas library. The analysis includes generating statistical data and extracting patterns, and the resulting data is visualized using visualization tools such as Grafana. Specifically, graphs and heat maps showing fluctuations in traffic volume are generated and displayed in a format that users can intuitively understand.

[0829] Terminal roles and processing

[0830] The device sends a data request to the server in response to a user request. This request includes information about a specific time period and road section. The server processes the request, retrieves the corresponding traffic data from the information storage device, and sends it to the device.

[0831] The received data is further analyzed on the device using Python libraries such as Matplotlib and Seaborn, and the data is displayed as statistical graphs and heat maps, allowing users to intuitively understand the traffic situation.

[0832] User operations

[0833] The user launches the application and sets the conditions for checking traffic volume data for a specific time period or road section. For example, the user may want to check "traffic volume on major road B from 07:00 to 09:00 on October 1, 2023," and specify this in the app. An example of this prompt is:

[0834] "Please display a graph showing the traffic volume and its fluctuations by vehicle type on major road B from 07:00 to 09:00 on October 1, 2023."

[0835] The device sends a request to the server based on these conditions and displays the retrieved data to the user. The user can then refer to the displayed graphs and heat maps and make appropriate decisions as needed. For example, they can change their departure time or choose an alternative route to avoid heavy traffic. Urban planners can also use the data to create transportation policies and infrastructure development plans.

[0836] As described above, "Traffic AI" is a system that realizes efficient and accurate surveys and analysis of road traffic volume and supports rapid decision-making in urban planning and transportation policy. By accurately performing each processing step based on the specific embodiment of this invention, advanced traffic volume management becomes possible.

[0837] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0838] Step 1:

[0839] The server receives real-time video data from the camera. The input is the video data from the camera, and the output is the video data transferred to the server. This video data is processed frame by frame and immediately moves to the next step. For example, we use the RTSP protocol to get the video stream and split it into frames using OpenCV.

[0840] Step 2:

[0841] The server preprocesses the received video data. The input is the video data received in step 1, and the output is the preprocessed video data. Preprocessing includes noise removal, resolution adjustment, and frame-by-frame image cropping. Specifically, the OpenCV library is used to remove noise with Gaussian blur and resize the video to the specified resolution.

[0842] Step 3:

[0843] The server inputs the preprocessed video data into a generative AI model. The input is the preprocessed video data, and the output is the identified vehicle data. The generative AI model (e.g., YOLOv5) identifies vehicles in the video with high accuracy and outputs the vehicle type and location information. For example, the YOLOv5 model can be used to identify cars, trucks, motorcycles, etc.

[0844] Step 4:

[0845] The server stores the identified vehicle data in an information storage device. The input is the identified vehicle data, and the output is the data stored in a database. The data is stored in JSON format along with timestamps and location information. Specifically, MongoDB is used to store the identification results.

[0846] Step 5:

[0847] The server analyzes the data stored in the traffic volume database. The input is the vehicle data stored in the database, and the output is the analysis results. The analysis includes generating statistical data and extracting patterns. Specifically, Pandas is used to aggregate the data and extract fluctuations and trends in traffic volume by time period.

[0848] Step 6:

[0849] The server visualizes the analysis results. The input is the analysis results, and the output is the visualized data. The visualization includes graphs and heat maps showing traffic volume fluctuations. Specifically, Grafana is used to create interactive dashboards.

[0850] Step 7:

[0851] The terminal sends a data request from the user to the server. The input is the user's request conditions (e.g., a specific time period or road section), and the output is the data request to the server. The request is sent using the HTTP protocol.

[0852] Step 8:

[0853] The terminal analyzes the traffic data received from the server. The input is the traffic data from the server, and the output is the analysis results. The analysis includes generating statistical graphs and heat maps. Specifically, the data is visualized using Python's Matplotlib and Seaborn.

[0854] Step 9:

[0855] The terminal displays the analysis results to the user. The input is the analysis results, and the output is visualization data displayed to the user. Dashboards, statistical graphs, and heat maps are used for display, allowing the user to intuitively understand the traffic situation.

[0856] Step 10:

[0857] The user launches the application and sets the conditions for checking traffic volume data. The input is the user's requested conditions, and the output is the condition settings to the device. For example, if the user wants to check "traffic volume on major road B from 07:00 to 09:00 on October 1, 2023," they specify this in the app.

[0858] Step 11:

[0859] Users make appropriate decisions based on the displayed traffic data. The input is the displayed visualization data, and the output is the user's decision. For example, they may change their departure time to avoid heavy traffic hours or choose an alternative route. Urban planners also use the data to create transportation policies and infrastructure development plans.

[0860] (Application example 1)

[0861] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0862] Conventional road traffic volume survey systems require a great deal of manpower and time to grasp traffic volume and identify vehicles, making real-time analysis difficult, especially when collecting data at night. Furthermore, they are inadequate in providing optimal detour routes based on the analyzed data, and in presenting information intuitively to drivers. New technologies are needed to solve these issues.

[0863] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0864] In this invention, the server includes means for receiving real-time video data from a camera, means for preprocessing the video data, means for identifying vehicles using a generative artificial intelligence model, means for storing the identified vehicle data in a traffic volume database, means for analyzing and visualizing the traffic volume data, and means for proposing an optimal route based on the analyzed traffic volume data, thereby enabling real-time traffic volume understanding, provision of optimal detour routes, and intuitive information presentation.

[0865] A "camera" is a device for acquiring real-time image data on a road.

[0866] "Real-time video data" refers to current video information continuously acquired through a camera.

[0867] "Video data preprocessing" refers to the process of converting raw data acquired from a camera into a format suitable for a generative artificial intelligence model.

[0868] A "generative artificial intelligence model" is an AI model trained using large amounts of data and capable of identifying vehicles in video footage.

[0869] "Vehicle identification" is the process of recognizing and classifying vehicles such as cars, trucks, and motorcycles from video data.

[0870] A "traffic database" is a data storage system for storing identified vehicle data for later analysis.

[0871] "Traffic data analysis" refers to the statistical processing of data stored in traffic databases to extract meaningful patterns and insights.

[0872] "Visualization" refers to the presentation of analyzed traffic data in a visual format such as a graph or heat map.

[0873] "Optimal route proposal" means selecting and presenting the most efficient and least congested route based on real-time traffic analysis data.

[0874] This invention provides a system called "Traffic Navigator" based on "Traffic AI," which highly automates road traffic volume surveys. This system receives video data from cameras in real time, identifies vehicles using a generative AI model, measures and analyzes traffic volume, and proposes optimal routes.

[0875] Server Roles and Operations

[0876] The server first receives real-time video data from the camera. This data is continuously acquired through the camera and processing begins immediately. The received video data is pre-processed into a format suitable for the generative AI model. This pre-processing includes cropping the image for each frame, removing noise, and adjusting the resolution. Libraries such as OpenCV are used for pre-processing.

[0877] The preprocessed video data is then fed into a generative AI model. This generative AI model is trained on a large amount of data and is able to identify vehicles in the video with high accuracy. The identified vehicle types (cars, trucks, motorcycles, etc.) and their counts are output in JSON format and stored in a traffic database. This data also includes timestamps and vehicle location information.

[0878] The data stored in the traffic database is analyzed on an ongoing basis. This analysis involves generating statistical data and extracting patterns. The analyzed data is then visualized using visualization tools. Specifically, it is displayed as graphs or heat maps showing fluctuations in traffic volume. Libraries such as Matplotlib are used at this stage.

[0879] The server then performs calculations to suggest optimal routes based on the analyzed traffic data, and presents the most efficient detour route to the driver based on the real-time analysis results.

[0880] Terminal roles and processing

[0881] The device sends a request for traffic volume data to the server in response to a user request. The request includes a specific time period and road section. The server then retrieves the relevant data from the traffic volume database and sends it to the device. The device then analyzes the received traffic volume data and displays it in an easy-to-understand format, such as a statistical graph or heat map.

[0882] User operations

[0883] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section. For example, if a user wants to check traffic volume on a specific road at night, they can set the condition "traffic volume on major road A from 22:00 to 22:30" on the app. Based on these conditions, the server provides analysis results and suggests the optimal detour route.

[0884] Specific examples

[0885] For example, it can be seen in real time that traffic volume on major road A is heavy at night. In this case, the system can immediately suggest optimal detour routes to help drivers avoid traffic jams. Urban planners can also use heat maps and statistical graphs to intuitively understand the traffic situation in specific areas and formulate appropriate traffic policies.

[0886] Prompt Sentence Examples

[0887] "Analyze the traffic volume on major road A from 10:00 PM to 10:30 PM at night and generate a heat map."

[0888] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0889] Step 1:

[0890] The server receives real-time video data from the camera. The video information continuously acquired through the camera is input and immediately processed. The received video data is used in subsequent processing steps.

[0891] Step 2:

[0892] The server preprocesses the received video data. This preprocessing involves cropping the image frame by frame, removing noise, and adjusting the resolution. Using an image processing library such as OpenCV, the quality of the input data is converted into a format suitable for the generative AI model. The preprocessed image data is then output.

[0893] Step 3:

[0894] The server inputs the preprocessed video data into a generative AI model to identify vehicles. The generative AI model uses a large amount of training data to perform highly accurate vehicle identification. The model outputs the identified vehicle type (car, truck, motorcycle, etc.) and its number, and this information is used in subsequent steps.

[0895] Step 4:

[0896] The server stores the identified vehicle data in a traffic database. The identification results are stored in JSON format, including timestamps and vehicle location information. The identified vehicle data is given as input and recorded in the traffic database as output.

[0897] Step 5:

[0898] The server analyzes the data stored in the traffic database. This analysis includes generating statistics and extracting patterns. The input data is the contents of the traffic database, and the output data is the analysis results. Statistical calculation methods are used.

[0899] Step 6:

[0900] The server visualizes the analyzed data. Using a visualization tool, the data is output as graphs and heat maps showing fluctuations in traffic volume. The Matplotlib library is used to visually display the analysis results.

[0901] Step 7:

[0902] The server proposes the optimal route based on the analysis results. Based on the analysis results in real time, it calculates the most efficient detour route for the driver. The input data is the analysis results, and the output data is the proposed optimal route.

[0903] Step 8:

[0904] The terminal receives the data sent from the server and requests specific traffic volume data according to the user's request. The request includes the specification of a specific time period or road section, and obtains the corresponding data from the server.

[0905] Step 9:

[0906] The device analyzes the traffic volume data it receives and displays it in statistical graphs and heat maps, allowing users to intuitively understand the data and assisting them in selecting the optimal route.

[0907] Step 10:

[0908] The user selects traffic volume data for a specific time period or road section through the application, and the device sends a request to the server. Based on the acquired data, the user can intuitively understand the traffic situation and act accordingly.

[0909] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0910] This invention provides a system called "Traffic AI Plus," which combines a highly automated system for road traffic volume surveys with an emotion engine that recognizes user emotions. This system receives video data from a camera in real time, identifies vehicles using a generative AI model, and measures and analyzes traffic volume. The emotion engine also recognizes user emotions and customizes the presentation of analysis results. The specific roles of the server, terminal, and user, as well as the processing involved, are described in detail below.

[0911] Server Roles and Operations

[0912] Video data reception and preprocessing

[0913] The server receives real-time video data from the camera, which is then processed immediately. The received video data is pre-processed into a format suitable for the generative AI model. Pre-processing includes cropping the image frame by frame, removing noise, and adjusting the resolution.

[0914] Vehicle identification and storage in a database

[0915] The preprocessed video data is then fed into a generative AI model. This model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. The identified vehicle types (cars, trucks, motorcycles, etc.) and their counts are output in JSON format and stored in a traffic database. The data also includes timestamps and vehicle location information.

[0916] Data analysis and visualization

[0917] The data stored in the traffic volume database is analyzed on an ongoing basis using analytical tools. This analysis includes generating statistical data and extracting patterns. The analyzed data is then visualized using visualization tools, and displayed as graphs and heat maps showing fluctuations in traffic volume.

[0918] Terminal roles and processing

[0919] Data Request and Receipt

[0920] The terminal sends a request for traffic volume data to the server in response to a user request. The request includes a specification of a specific time period and road section. The server then retrieves the relevant data from the traffic volume database and sends it to the terminal.

[0921] Data analysis and presentation

[0922] The device analyzes the received traffic data and displays it in a user-friendly format, using statistical graphs and heat maps. Furthermore, an emotion engine customizes the analysis results to reflect the user's emotions.

[0923] User operations

[0924] Operating the application and checking the data

[0925] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section. For example, if the user wants to check traffic volume on a specific road at night, they set the condition "traffic volume on major road A from 22:00 to 22:30" in the app. Based on this condition, the device sends a request to the server and displays the retrieved data to the user.

[0926] Emotion recognition and customized alerts

[0927] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions, and then customizes traffic information displays and alerts accordingly. For example, if the user is nervous, the device will suggest a relaxing route, and if the user is happy, the device will suggest the shortest route.

[0928] Decision-making and action

[0929] Users can view the visualized data and make on-the-fly decisions as needed, such as choosing an alternative route to avoid heavy traffic on major road A. Urban planners can also use the insights gained to formulate transportation policies and infrastructure development plans.

[0930] This invention not only enables efficient and accurate survey and analysis of road traffic volume, but also enables the provision of customized traffic information that takes user emotions into consideration, which will greatly support the formulation of urban planning and transportation policies and improve user satisfaction.

[0931] The processing flow will be explained below.

[0932] Server Processing

[0933] Step 1:

[0934] The server receives real-time video data from the camera. The video data is transmitted at a constant frame rate, and the server receives it sequentially.

[0935] Specific operation: Receives video data from the camera at 10 fps (frames per second).

[0936] Step 2:

[0937] The server pre-processes the received video data, including cutting out the image frame by frame, removing noise, and adjusting the resolution.

[0938] Specific operation: Resize each frame to 256x256 pixels and remove noise from the image.

[0939] Step 3:

[0940] The server inputs the preprocessed data into a generative AI model to identify vehicles. The generative AI model then identifies vehicles in the video with high accuracy and determines their type (cars, trucks, motorcycles, etc.) and number.

[0941] Specific operation: In the first frame, two cars and one truck are identified, and in the second frame, one motorcycle is identified.

[0942] Step 4:

[0943] The server stores the identified vehicle data in a traffic database, including vehicle type, number, time stamp, and location information.

[0944] Specific operation: Save the data of two cars and one truck at 2023-10-01 22:05:10 in the database.

[0945] Terminal handling

[0946] Step 5:

[0947] The terminal sends a request for traffic data to the server based on the user's request, which includes a specification of a particular time period and road section.

[0948] Specific operation: Send a message to the server requesting traffic volume data for major road A from 22:00 to 22:30.

[0949] Step 6:

[0950] The terminal analyzes the traffic volume data received from the server, which includes formatting the data and statistical compilation.

[0951] Specific operation: Calculate the total number of vehicles and their breakdown on main road A from 22:00 to 22:30.

[0952] Step 7:

[0953] The device displays the analysis results in a user-friendly format, including generating graphs and heat maps showing traffic fluctuations.

[0954] Specific operation: Displays the traffic volume on major road A as a line graph, showing fluctuations by time period.

[0955] Step 8:

[0956] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions, and displays and alerts are customized based on the recognized emotions.

[0957] What it does: If the user is nervous, provide them with directions and travel information to help them relax.

[0958] User operations

[0959] Step 9:

[0960] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section.

[0961] Specific operation: Set the condition "traffic volume on major road A from 22:00 to 22:30" on the application.

[0962] Step 10:

[0963] Users can view visualized data and make on-the-fly decisions as needed.

[0964] Specific operation: Select a route that avoids times when traffic on major road A is heavy.

[0965] These are the specific processing steps of the server, terminal, and user. This not only enables efficient and accurate survey and analysis of road traffic volume, but also provides customized traffic information that takes user emotions into consideration.

[0966] Example 2

[0967] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0968] While conventional road traffic volume survey systems can measure traffic volume using real-time video data, they have the problem of being unable to provide customized information that takes user emotions into account.Furthermore, they have a problem of not being able to identify vehicles with a high degree of accuracy even in nighttime video data.

[0969] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0970] In this invention, the server includes means for receiving real-time video data from a camera, means for preprocessing the video data, means for identifying vehicles using a generative artificial intelligence model, means for storing the identified vehicle data in a traffic volume database, means for analyzing and visualizing the traffic volume data, and means for recognizing a user's emotions and customizing the presentation method of the analysis results based on the emotions. This not only enables efficient and accurate road traffic volume survey and analysis, but also enables the provision of customized traffic information that takes the user's emotions into consideration.

[0971] A "camera" is a photographing device for capturing video data in real time.

[0972] "Real-time video data" refers to data that is obtained and transmitted immediately as video information at the current time.

[0973] "Preprocessing" refers to a series of operations that convert video data into a format suitable for subsequent analysis and recognition.

[0974] A "generative artificial intelligence model" is a machine learning model that learns from data and performs a specific task (e.g., vehicle identification).

[0975] "Means for identifying vehicles" refers to technologies or algorithms for detecting vehicles in video data and identifying their type and number.

[0976] "Vehicle data" refers to data including the type, number, timestamp, location information, etc. of identified vehicles.

[0977] A "traffic database" is a database system for storing identified vehicle data.

[0978] "Analysis" refers to the process of generating statistical data and extracting patterns based on traffic volume data.

[0979] "Visualization" refers to displaying analyzed data in a visual format such as a graph or heat map.

[0980] "Means for recognizing user emotions" refers to technology or devices that analyze emotions from the user's facial expressions and tone of voice.

[0981] "Customization means" refers to technologies and algorithms that change the way information is presented and the format of the display based on the user's emotions.

[0982] This invention relates to a system called "Traffic AI Plus," which combines a highly automated system for road traffic volume surveys with a function for recognizing user emotions. This system receives video data from a camera in real time, identifies vehicles using a generative AI model, and measures and analyzes traffic volume. It also has a function for recognizing user emotions using an emotion engine and customizing the way the analysis results are presented.

[0983] Server Roles

[0984] Video data reception and preprocessing

[0985] The server receives real-time video data from the camera. This video data is captured at 24 frames per second and temporarily stored in the server's buffer. The video data is then pre-processed. This includes cropping the image frame by frame using the OpenCV library, applying a noise reduction filter, and unifying the resolution.

[0986] Vehicle Identification

[0987] The preprocessed video data is input into a generative AI model. This generative AI model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. Specifically, it uses the prompt "How many vehicles are there in this frame?" to identify the type and number of vehicles.

[0988] Saving to a database

[0989] The identified vehicle data is converted to JSON format and stored in a traffic database. The data includes timestamps and vehicle location information. The server uses a NoSQL database (e.g., MongoDB) to store this data.

[0990] Data analysis and visualization

[0991] The data stored in the traffic volume database is analyzed using an analysis tool. For analysis, the Python pandas library is used to generate statistical data, and the matplotlib library is used to visualize the data, generating graphs and heat maps showing fluctuations in traffic volume.

[0992] Device Role

[0993] Sending and Receiving Data Requests

[0994] In response to a user request, the device requests traffic volume data for a specific time period or road section from the server. This request uses the HTTP protocol, and the corresponding traffic volume data is received from the server in JSON format.

[0995] Data analysis and presentation

[0996] The received traffic volume data is analyzed on the device, using the JavaScript D3.js library to generate interactive graphs and heat maps. Furthermore, the device's emotion engine customizes the display format based on the user's emotions.

[0997] User operations

[0998] Launching the application and checking the data

[0999] The user launches the application and sets conditions for checking specific traffic volume data, for example, "traffic volume on major road A from 22:00 to 22:30 on October 1, 2023."

[1000] Emotion Recognition and Customization

[1001] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions. Based on this, the display format of traffic information and alerts can be customized. For example, if the user is feeling stressed, the device will display information in a color scheme that has a relaxing effect.

[1002] Specific examples

[1003] It receives video data in real time, preprocesses it, and then inputs a prompt to the generative AI model: "How many vehicles are there in this frame?" The resulting data is then stored in a traffic database and analyzed using analytical tools, providing traffic information to users in an easy-to-understand format.

[1004] In this way, Traffic AI Plus is a system that realizes efficient and accurate surveys and analysis of road traffic volume, and also provides customized traffic information according to user sentiment. This system will greatly contribute to the formulation of urban planning and transportation policies, and will improve user satisfaction.

[1005] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1006] Step 1:

[1007] Receiving real-time video data

[1008] The server receives real-time video data from the camera. Specifically, the camera transmits video at 24 frames per second, and the server temporarily stores the data in a buffer. The input is the video data from the camera, and the output is the video data stored in the buffer. This data is used for subsequent processing steps.

[1009] Step 2:

[1010] Video data preprocessing

[1011] The server preprocesses the video data stored in the buffer. Specific preprocessing tasks include using the OpenCV library to crop the image frame by frame, applying a noise reduction filter, and unifying the resolution to 1080p. The input is the video data in the buffer, and the output is the preprocessed image data.

[1012] Step 3:

[1013] Vehicle Identification with Generative AI Models

[1014] The server inputs the preprocessed video data into a generative AI model, which then inputs the prompt "How many vehicles are there in this frame?" to identify the vehicles. The input is the preprocessed image data and the prompt, and the output is JSON-formatted data containing the type and number of identified vehicles.

[1015] Step 4:

[1016] Data storage

[1017] The server stores the identified vehicle data in a traffic volume database. Specifically, it uses a NoSQL database (e.g., MongoDB) to store JSON data including vehicle type, number, timestamp, and location information. The input is the identified vehicle data, and the output is the traffic volume data stored in the database.

[1018] Step 5:

[1019] Data analysis and visualization

[1020] The server analyzes and visualizes the data stored in the traffic volume database. For analysis, it uses Python's pandas library to generate statistical data and the matplotlib library to create visualizations such as heat maps and graphs. The input is the data in the traffic volume database, and the output is the visual data as the analysis results.

[1021] Step 6:

[1022] Sending a Data Request

[1023] In response to a user request, the terminal requests traffic volume data for a specific time period or road section from the server. Specific conditions are entered using an HTTP request, and the corresponding data is received from the server. The input is the conditions set by the user, and the output is the traffic volume data received from the server.

[1024] Step 7:

[1025] Data reception and analysis

[1026] The terminal analyzes the traffic volume data received from the server. It uses the D3.js library to create interactive graphs and heat maps and display them to the user. The input is the traffic volume data received from the server, and the output is the visualized data.

[1027] Step 8:

[1028] Emotion-aware customization

[1029] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions. Based on this, the display format is customized. For example, if the user is feeling stressed, the data is displayed in a color scheme that has a relaxing effect. The input is the user's facial expressions and voice data, and the output is a customized display format.

[1030] Step 9:

[1031] Application launch and condition settings

[1032] The user launches the application and sets a condition to check specific traffic volume data, for example, "traffic volume on major road A from 22:00 to 22:30 on October 1, 2023." The input is the condition setting, and the output is a data request based on the condition setting.

[1033] Step 10:

[1034] Data review and decision making

[1035] The user checks the visualized traffic data displayed on the device and decides on actions as necessary, for example, changing the route to avoid heavy traffic periods. The input is the visualized traffic data, and the output is the user's decision and actions.

[1036] In this way, Traffic AI Plus conducts efficient and accurate road traffic volume surveys and analysis through each step, and also provides information that responds to the user's emotions.

[1037] (Application example 2)

[1038] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1039] Conventional road traffic volume survey and analysis systems provide real-time traffic volume information, but they have not been able to provide customized information that takes user emotions into account. Furthermore, when proposing routes for autonomous vehicles, they lack the ability to respond to user emotions, making it difficult to ensure safe and comfortable driving at all times. Given this background, there is a growing need for a system that analyzes traffic volume in real time and customizes information based on user emotions.

[1040] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving real-time video data from a camera, means for preprocessing the video data, means for identifying vehicles using a generative artificial intelligence model, means for storing the identified vehicle data in a traffic volume database, means for analyzing and visualizing the traffic volume data, means for customizing the presentation method using an emotion engine that recognizes user emotions, and means for proposing a comfortable and safe route in an autonomously driven vehicle. This makes it possible to analyze traffic volume information in real time and provide customized information according to the user's emotions.

[1041] The "means for receiving real-time video data from a camera" refers to a system or device for receiving video data acquired from a camera in real-time and transmitting the data to another processing unit.

[1042] "Means for preprocessing video data" refers to a system or device that performs preprocessing such as frame cropping, noise removal, and resolution adjustment in order to convert the video data received from the camera into a format suitable for the generative AI model.

[1043] "Means for identifying vehicles using generative artificial intelligence models" refers to software or algorithms that use generative AI models trained on large amounts of data to identify vehicles in video footage with high accuracy.

[1044] "Means for storing identified vehicle data in a traffic volume database" refers to a system or device that automatically stores vehicle data (such as vehicle type, number, and location information) identified by the generative AI model in a traffic volume database in JSON format or similar.

[1045] "Means for analyzing and visualizing traffic volume data" refers to a system or device that statistically analyzes data stored in a traffic volume database and visualizes it using visualization tools as graphs or heat maps.

[1046] "Means for customizing presentation methods using an emotion engine that recognizes user emotions" refers to software or algorithms that analyze the user's facial expressions and tone of voice to recognize emotions, and customize the way traffic information and routes are presented based on that.

[1047] "Means for proposing comfortable and safe routes within an autonomous vehicle" refers to a navigation system or algorithm that proposes and executes optimal routes for an autonomous vehicle based on traffic volume data and user emotion data.

[1048] This invention, the "Traffic AI Plus" system, analyzes road traffic volume in a highly automated manner and also recognizes user emotions to provide customized information. It is particularly focused on operation in autonomous vehicles. The system has three main components: a server, a terminal, and a user.

[1049] Server Roles and Operations

[1050] The server performs the following functions:

[1051] Video data reception and preprocessing

[1052] The server receives real-time video data from the camera. This data begins processing immediately. The video data is pre-processed into a format suitable for the generative AI model. Pre-processing includes cropping the image frame by frame, removing noise, and adjusting the resolution. This allows the generative AI model to identify vehicles with high accuracy.

[1053] Vehicle identification and storage in a database

[1054] The preprocessed video data is then fed into a generative AI model. This model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. The identified vehicle types (cars, trucks, motorcycles, etc.) and their counts are output in JSON format and stored in a traffic database. The data also includes timestamps and vehicle location information.

[1055] Data analysis and visualization

[1056] The data stored in the traffic volume database is analyzed on an ongoing basis using analytical tools. This analysis includes generating statistical data and extracting patterns. The analyzed data is then visualized using visualization tools, and displayed as graphs and heat maps showing fluctuations in traffic volume.

[1057] Terminal roles and processing

[1058] The terminal implements the following functions:

[1059] Data Request and Receipt

[1060] The terminal sends a request for traffic volume data to the server in response to a user request. The request includes a specification of a specific time period and road section. The server then retrieves the relevant data from the traffic volume database and sends it to the terminal.

[1061] Data analysis and presentation

[1062] The device analyzes the received traffic data and displays it in a user-friendly format, using statistical graphs and heat maps. Furthermore, the analysis results are customized based on the user's emotions using an emotion engine.

[1063] User roles and operations

[1064] The user performs the following operations:

[1065] Operating the application and checking the data

[1066] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section. For example, if they want to check traffic volume on a specific road at night, they set the condition "traffic volume on major road A from 22:00 to 22:30" in the app. Based on this condition, the device sends a request to the server and displays the retrieved data to the user.

[1067] Emotion recognition and customized alerts

[1068] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions, and then customizes traffic information displays and alerts accordingly. For example, if the user is nervous, the device will suggest a relaxing route, and if the user is happy, the device will suggest the shortest route.

[1069] Decision-making and action

[1070] Users can view the visualized data and make on-the-fly decisions as needed, such as choosing an alternative route to avoid heavy traffic on major road A. Urban planners can also use the insights gained to formulate transportation policies and infrastructure development plans.

[1071] Specific examples

[1072] For example, suppose a child looks sleepy while a family is on a road trip. The system recognizes the child's sleepy expression through the in-car camera and selects a quiet and comfortable route. If the parent looks excited, the system suggests the shortest route to arrive smoothly. In this way, customized information is provided that takes into account the user's emotions in real time.

[1073] Prompt Sentence Examples

[1074] "Create an application that suggests a quieter route with less traffic if the user is sleepy, and selects the shortest route if the user is excited. The data used will be real-time camera footage, the user's facial expressions, and audio data."

[1075] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1076] Step 1:

[1077] The server receives real-time video data from the camera. It takes the video stream as input and outputs video data (frame by frame). Specifically, the server monitors the camera stream and immediately captures the captured video data.

[1078] Step 2:

[1079] The server preprocesses the received video data. The input is the video data acquired in step 1, and the output is noise-removed, resolution-adjusted frames (preprocessed video data). Specifically, the server uses OpenCV to remove noise and adjust the resolution, and converts each frame into an appropriate format.

[1080] Step 3:

[1081] The server uses a generative AI model to identify vehicles from the preprocessed video data. The input is the preprocessed video data output in step 2, and the output is data on identified vehicles (vehicle type, number, and location information). Specifically, the server applies the generative AI model using Keras / TensorFlow to identify vehicles in the video with high accuracy.

[1082] Step 4:

[1083] The server stores the identified vehicle data in the traffic volume database. The input is the vehicle data output in step 3, and the updated traffic volume database is output. Specifically, the server converts the vehicle data into JSON format and automatically stores it in the database.

[1084] Step 5:

[1085] The server analyzes and visualizes traffic volume data. The input is data from the traffic volume database, and the output is analysis results and visualized data (graphs, heat maps). Specifically, analysis tools are used to generate statistical data and extract patterns, and visualization tools are used to visualize the results.

[1086] Step 6:

[1087] The terminal sends a request for traffic volume data to the server in response to a user request. The input is the request conditions from the user, and a request message is output. Specifically, the user sets conditions for specific time periods and road sections within the application, and the request is made according to those conditions.

[1088] Step 7:

[1089] The server acquires traffic volume data in response to a request from the terminal and sends it to the terminal. The input is the request message sent in step 6, and the corresponding traffic volume data is output. Specifically, the server acquires the relevant data from the traffic volume database and sends it back to the terminal.

[1090] Step 8:

[1091] The terminal analyzes the received traffic volume data and displays it in a format that is easy for the user to understand. The input is the traffic volume data acquired in step 7, and visualized data (statistical graphs, heat maps) is output. Specifically, the terminal processes the received data using analysis tools and displays it visually on the user interface.

[1092] Step 9:

[1093] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize emotions. The input is raw data from the in-car camera and microphone, and the output is the user's emotional state. Specifically, the device processes the data using an emotion recognition algorithm to determine the user's emotions in real time.

[1094] Step 10:

[1095] The device customizes the way traffic information and routes are presented based on the user's emotions. The input is the emotion data recognized in step 9 and the traffic volume data acquired in step 8, and customized route suggestions and display formats are output. Specifically, if the user is nervous, a relaxing route is suggested, and if the user is happy, the shortest route is displayed.

[1096] Step 11:

[1097] The user checks the visualized data and selects a route. The input is the visualized data and route proposals presented by the terminal, and the selected route is output. Specifically, the user looks at the application, selects the optimal route, and makes another request if necessary.

[1098] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1099] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1100] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1101] [Fourth embodiment]

[1102] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1103] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1104] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1105] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1106] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1107] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1108] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1109] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1110] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1111] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1112] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1113] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1114] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1115] This invention provides a system called "Traffic AI" that highly automates road traffic volume surveys. This system receives video data from cameras in real time, identifies vehicles using a generative AI model, and measures and analyzes road traffic volume. The specific roles of the server, terminal, and user, and the processing involved, are explained in detail below.

[1116] Server Roles and Operations

[1117] Video data reception and preprocessing

[1118] The server first receives real-time video data from the camera. This data is sent to the server with minimal delay, and processing begins immediately. The received video data is pre-processed into a format suitable for the generative AI model. This pre-processing includes cropping the image frame by frame, removing noise, and adjusting the resolution.

[1119] Vehicle identification and storage in a database

[1120] The preprocessed video data is then fed into a generative AI model. This model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. The identified vehicle types (cars, trucks, motorcycles, etc.) and their counts are output in JSON format and stored in a traffic database. This data also includes timestamps and vehicle location information.

[1121] Data analysis and visualization

[1122] The data stored in the traffic database is analyzed on an ongoing basis. This analysis involves generating statistical data and extracting patterns. The analyzed data is then visualized using visualization tools. Specifically, it is displayed as graphs or heat maps showing fluctuations in traffic volume.

[1123] Terminal roles and processing

[1124] Data Request and Receipt

[1125] In response to a user request, the terminal sends a request for traffic volume data to the server. The request includes a specification of a specific time period and road section. In response, the server retrieves the relevant data from the traffic volume database and sends it to the terminal.

[1126] Data analysis and presentation

[1127] The device analyzes the received traffic volume data and displays it in a user-friendly format, using statistical graphs and heat maps, allowing users to intuitively grasp the traffic situation.

[1128] User operations

[1129] Operating the application and checking the data

[1130] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section. For example, if the user wants to check traffic volume on a specific road at night, they set the condition "traffic volume on major road A from 22:00 to 22:30" in the app. Based on this condition, the device sends a request to the server and displays the retrieved data to the user.

[1131] Decision-making and action

[1132] Users can view the visualized data and make adaptive decisions as needed, such as choosing an alternative route to avoid heavy traffic on major road A. Urban planners can also use the insights gained to formulate transportation policies and infrastructure development plans.

[1133] The present invention makes it possible to carry out efficient and accurate surveys and analyses of road traffic volumes, which can be of great help in formulating urban plans and transportation policies.

[1134] The processing flow will be explained below.

[1135] Now, let me explain the program step by step.

[1136] Server Processing

[1137] Step 1:

[1138] The server receives real-time video data from the camera. The video data is transmitted at a constant frame rate, and the server receives it sequentially.

[1139] Specific operation: Receives video data from the camera at 10 fps (frames per second).

[1140] Step 2:

[1141] The server preprocesses the received video data, including cropping the image frame by frame, removing noise, and adjusting the resolution, converting it into a format suitable for the generative AI model.

[1142] Specific operation: Resize each frame to 256x256 pixels and remove noise from the image.

[1143] Step 3:

[1144] The server inputs the preprocessed data into a generative AI model to identify vehicles. The generative AI model then identifies vehicles in the video with high accuracy and determines their type (cars, trucks, motorcycles, etc.) and number.

[1145] Specific operation: In the first frame, two cars and one truck are identified, and in the second frame, one motorcycle is identified.

[1146] Step 4:

[1147] The server stores the identified vehicle data in a traffic database, including vehicle type, number, time stamp, and location information.

[1148] Specific operation: Save the data of two cars and one truck at 2023-10-01 22:05:10 in the database.

[1149] Terminal handling

[1150] Step 5:

[1151] The terminal sends a request for traffic data to the server based on the user's request, which includes a specification of a particular time period and road section.

[1152] Specific operation: Send a message to the server requesting traffic volume data for major road A from 22:00 to 22:30.

[1153] Step 6:

[1154] The terminal analyzes the traffic volume data received from the server, which includes formatting the data and statistical compilation.

[1155] Specific operation: Calculate the total number of vehicles and their breakdown on main road A from 22:00 to 22:30.

[1156] Step 7:

[1157] The device displays the analysis results in a user-friendly format, including generating graphs and heat maps showing traffic fluctuations.

[1158] Specific operation: Displays the traffic volume on major road A as a line graph, showing fluctuations by time period.

[1159] User operations

[1160] Step 8:

[1161] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section.

[1162] Specific operation: Set the condition "traffic volume on major road A from 22:00 to 22:30" on the application.

[1163] Step 9:

[1164] Users can view visualized data and make on-the-fly decisions as needed.

[1165] Specific operation: Select a route that avoids times when traffic on major road A is heavy.

[1166] These are the specific processing steps of the server, terminal, and user, which enable efficient and accurate survey and analysis of road traffic volume.

[1167] Example 1

[1168] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1169] Conventional road traffic volume surveys require manual analysis of large amounts of video data, which is time-consuming and labor-intensive. It is also difficult to accurately identify vehicles in nighttime video data. Furthermore, there is a lack of systems for analyzing and visualizing traffic volume data in real time, which hinders rapid decision-making in urban planning and transportation policies.

[1170] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1171] In this invention, the server includes a means for receiving real-time video data from the camera, a means for preprocessing the video data, and a means for identifying vehicles using a generative artificial intelligence model. This enables automatic processing of video data and highly accurate vehicle identification. It also includes a means for storing the identified vehicle data in an information storage device, a means for analyzing and visualizing traffic volume data, a means for requesting traffic volume data for a specified time period or road section and retrieving the corresponding data from the information storage device, and a means for analyzing the received data and displaying it as a statistical graph or heat map. This enables real-time analysis and visualization of traffic volume, enabling rapid decision-making in urban planning and transportation policies.

[1172] A "camera" is a device that captures video data and transmits it in real time.

[1173] "Video data" is a digital record of visual information captured by a camera.

[1174] "Preprocessing" is the process of converting video data into a format that can be used by a generative AI model, and includes processes such as noise removal and resolution adjustment.

[1175] A "generative artificial intelligence model" is a machine learning model that is trained using large amounts of data and can perform a specific task (e.g., vehicle identification) with high accuracy.

[1176] "Vehicle identification" is the process of identifying vehicles contained in video data and determining their type and location.

[1177] "Information storage device" means a digital storage system for long-term preservation of identified data.

[1178] "Traffic volume data" is a collection of information about vehicle traffic in a particular area or time period.

[1179] "Analysis" is the process of processing data using statistical methods and pattern recognition techniques to extract meaningful information.

[1180] "Visualization" means displaying the analyzed data in a visual format such as a graph or heat map.

[1181] A "statistical graph" is a diagram that visually represents numerical data and is used to show fluctuations and trends in traffic volume.

[1182] A "heat map" is a graphical tool that uses shades of color to represent data density, showing the concentration of traffic in a particular area or time period.

[1183] "User interface" refers to the screen and operating means by which a user interacts with a system and inputs and outputs information.

[1184] This invention relates to "Traffic AI," a system for highly automated road traffic volume surveys. This system receives real-time video data from cameras, identifies vehicles using generative AI models, and analyzes and visualizes traffic volume. For example, the hardware used includes cameras, servers, and terminals (e.g., mobile devices and PCs), and the software and libraries used include OpenCV, YOLOv5, Pandas, Grafana, HTTP protocol, and MongoDB.

[1185] Server Roles and Operations

[1186] The server first receives real-time video data from the camera. This data is then sent to the server via a streaming protocol. The received video data is immediately pre-processed using the OpenCV library. This pre-processing includes frame-by-frame image extraction, noise reduction, and resolution adjustment.

[1187] The preprocessed video data is input into a generative AI model (e.g., YOLOv5). This AI model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. The type and number of identified vehicles are output in JSON format along with timestamps and location information and stored in an information storage device (e.g., MongoDB).

[1188] The data stored in the traffic database is analyzed using the Pandas library. The analysis includes generating statistical data and extracting patterns, and the resulting data is visualized using visualization tools such as Grafana. Specifically, graphs and heat maps showing fluctuations in traffic volume are generated and displayed in a format that users can intuitively understand.

[1189] Terminal roles and processing

[1190] The device sends a data request to the server in response to a user request. This request includes information about a specific time period and road section. The server processes the request, retrieves the corresponding traffic data from the information storage device, and sends it to the device.

[1191] The received data is further analyzed on the device using Python libraries such as Matplotlib and Seaborn, and the data is displayed as statistical graphs and heat maps, allowing users to intuitively understand the traffic situation.

[1192] User operations

[1193] The user launches the application and sets the conditions for checking traffic volume data for a specific time period or road section. For example, the user may want to check "traffic volume on major road B from 07:00 to 09:00 on October 1, 2023," and specify this in the app. An example of this prompt is:

[1194] "Please display a graph showing the traffic volume and its fluctuations by vehicle type on major road B from 07:00 to 09:00 on October 1, 2023."

[1195] The device sends a request to the server based on these conditions and displays the retrieved data to the user. The user can then refer to the displayed graphs and heat maps and make appropriate decisions as needed. For example, they can change their departure time or choose an alternative route to avoid heavy traffic. Urban planners can also use the data to create transportation policies and infrastructure development plans.

[1196] As described above, "Traffic AI" is a system that realizes efficient and accurate surveys and analysis of road traffic volume and supports rapid decision-making in urban planning and transportation policy. By accurately performing each processing step based on the specific embodiment of this invention, advanced traffic volume management becomes possible.

[1197] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1198] Step 1:

[1199] The server receives real-time video data from the camera. The input is the video data from the camera, and the output is the video data transferred to the server. This video data is processed frame by frame and immediately moves to the next step. For example, we use the RTSP protocol to get the video stream and split it into frames using OpenCV.

[1200] Step 2:

[1201] The server preprocesses the received video data. The input is the video data received in step 1, and the output is the preprocessed video data. Preprocessing includes noise removal, resolution adjustment, and frame-by-frame image cropping. Specifically, the OpenCV library is used to remove noise with Gaussian blur and resize the video to the specified resolution.

[1202] Step 3:

[1203] The server inputs the preprocessed video data into a generative AI model. The input is the preprocessed video data, and the output is the identified vehicle data. The generative AI model (e.g., YOLOv5) identifies vehicles in the video with high accuracy and outputs the vehicle type and location information. For example, the YOLOv5 model can be used to identify cars, trucks, motorcycles, etc.

[1204] Step 4:

[1205] The server stores the identified vehicle data in an information storage device. The input is the identified vehicle data, and the output is the data stored in a database. The data is stored in JSON format along with timestamps and location information. Specifically, MongoDB is used to store the identification results.

[1206] Step 5:

[1207] The server analyzes the data stored in the traffic volume database. The input is the vehicle data stored in the database, and the output is the analysis results. The analysis includes generating statistical data and extracting patterns. Specifically, Pandas is used to aggregate the data and extract fluctuations and trends in traffic volume by time period.

[1208] Step 6:

[1209] The server visualizes the analysis results. The input is the analysis results, and the output is the visualized data. The visualization includes graphs and heat maps showing traffic volume fluctuations. Specifically, Grafana is used to create interactive dashboards.

[1210] Step 7:

[1211] The terminal sends a data request from the user to the server. The input is the user's request conditions (e.g., a specific time period or road section), and the output is the data request to the server. The request is sent using the HTTP protocol.

[1212] Step 8:

[1213] The terminal analyzes the traffic data received from the server. The input is the traffic data from the server, and the output is the analysis results. The analysis includes generating statistical graphs and heat maps. Specifically, the data is visualized using Python's Matplotlib and Seaborn.

[1214] Step 9:

[1215] The terminal displays the analysis results to the user. The input is the analysis results, and the output is visualization data displayed to the user. Dashboards, statistical graphs, and heat maps are used for display, allowing the user to intuitively understand the traffic situation.

[1216] Step 10:

[1217] The user launches the application and sets the conditions for checking traffic volume data. The input is the user's requested conditions, and the output is the condition settings to the device. For example, if the user wants to check "traffic volume on major road B from 07:00 to 09:00 on October 1, 2023," they specify this in the app.

[1218] Step 11:

[1219] Users make appropriate decisions based on the displayed traffic data. The input is the displayed visualization data, and the output is the user's decision. For example, they may change their departure time to avoid heavy traffic hours or choose an alternative route. Urban planners also use the data to create transportation policies and infrastructure development plans.

[1220] (Application example 1)

[1221] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1222] Conventional road traffic volume survey systems require a great deal of manpower and time to grasp traffic volume and identify vehicles, making real-time analysis difficult, especially when collecting data at night. Furthermore, they are inadequate in providing optimal detour routes based on the analyzed data, and in presenting information intuitively to drivers. New technologies are needed to solve these issues.

[1223] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1224] In this invention, the server includes means for receiving real-time video data from a camera, means for preprocessing the video data, means for identifying vehicles using a generative artificial intelligence model, means for storing the identified vehicle data in a traffic volume database, means for analyzing and visualizing the traffic volume data, and means for proposing an optimal route based on the analyzed traffic volume data, thereby enabling real-time traffic volume understanding, provision of optimal detour routes, and intuitive information presentation.

[1225] A "camera" is a device for acquiring real-time image data on a road.

[1226] "Real-time video data" refers to current video information continuously acquired through a camera.

[1227] "Video data preprocessing" refers to the process of converting raw data acquired from a camera into a format suitable for a generative artificial intelligence model.

[1228] A "generative artificial intelligence model" is an AI model trained using large amounts of data and capable of identifying vehicles in video footage.

[1229] "Vehicle identification" is the process of recognizing and classifying vehicles such as cars, trucks, and motorcycles from video data.

[1230] A "traffic database" is a data storage system for storing identified vehicle data for later analysis.

[1231] "Traffic data analysis" refers to the statistical processing of data stored in traffic databases to extract meaningful patterns and insights.

[1232] "Visualization" refers to the presentation of analyzed traffic data in a visual format such as a graph or heat map.

[1233] "Optimal route proposal" means selecting and presenting the most efficient and least congested route based on real-time traffic analysis data.

[1234] This invention provides a system called "Traffic Navigator" based on "Traffic AI," which highly automates road traffic volume surveys. This system receives video data from cameras in real time, identifies vehicles using a generative AI model, measures and analyzes traffic volume, and proposes optimal routes.

[1235] Server Roles and Operations

[1236] The server first receives real-time video data from the camera. This data is continuously acquired through the camera and processing begins immediately. The received video data is pre-processed into a format suitable for the generative AI model. This pre-processing includes cropping the image for each frame, removing noise, and adjusting the resolution. Libraries such as OpenCV are used for pre-processing.

[1237] The preprocessed video data is then fed into a generative AI model. This generative AI model is trained on a large amount of data and is able to identify vehicles in the video with high accuracy. The identified vehicle types (cars, trucks, motorcycles, etc.) and their counts are output in JSON format and stored in a traffic database. This data also includes timestamps and vehicle location information.

[1238] The data stored in the traffic database is analyzed on an ongoing basis. This analysis involves generating statistical data and extracting patterns. The analyzed data is then visualized using visualization tools. Specifically, it is displayed as graphs or heat maps showing fluctuations in traffic volume. Libraries such as Matplotlib are used at this stage.

[1239] The server then performs calculations to suggest optimal routes based on the analyzed traffic data, and presents the most efficient detour route to the driver based on the real-time analysis results.

[1240] Terminal roles and processing

[1241] The device sends a request for traffic volume data to the server in response to a user request. The request includes a specific time period and road section. The server then retrieves the relevant data from the traffic volume database and sends it to the device. The device then analyzes the received traffic volume data and displays it in an easy-to-understand format, such as a statistical graph or heat map.

[1242] User operations

[1243] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section. For example, if a user wants to check traffic volume on a specific road at night, they can set the condition "traffic volume on major road A from 22:00 to 22:30" on the app. Based on these conditions, the server provides analysis results and suggests the optimal detour route.

[1244] Specific examples

[1245] For example, it can be seen in real time that traffic volume on major road A is heavy at night. In this case, the system can immediately suggest optimal detour routes to help drivers avoid traffic jams. Urban planners can also use heat maps and statistical graphs to intuitively understand the traffic situation in specific areas and formulate appropriate traffic policies.

[1246] Prompt Sentence Examples

[1247] "Analyze the traffic volume on major road A from 10:00 PM to 10:30 PM at night and generate a heat map."

[1248] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1249] Step 1:

[1250] The server receives real-time video data from the camera. The video information continuously acquired through the camera is input and immediately processed. The received video data is used in subsequent processing steps.

[1251] Step 2:

[1252] The server preprocesses the received video data. This preprocessing involves cropping the image frame by frame, removing noise, and adjusting the resolution. Using an image processing library such as OpenCV, the quality of the input data is converted into a format suitable for the generative AI model. The preprocessed image data is then output.

[1253] Step 3:

[1254] The server inputs the preprocessed video data into a generative AI model to identify vehicles. The generative AI model uses a large amount of training data to perform highly accurate vehicle identification. The model outputs the identified vehicle type (car, truck, motorcycle, etc.) and its number, and this information is used in subsequent steps.

[1255] Step 4:

[1256] The server stores the identified vehicle data in a traffic database. The identification results are stored in JSON format, including timestamps and vehicle location information. The identified vehicle data is given as input and recorded in the traffic database as output.

[1257] Step 5:

[1258] The server analyzes the data stored in the traffic database. This analysis includes generating statistics and extracting patterns. The input data is the contents of the traffic database, and the output data is the analysis results. Statistical calculation methods are used.

[1259] Step 6:

[1260] The server visualizes the analyzed data. Using a visualization tool, the data is output as graphs and heat maps showing fluctuations in traffic volume. The Matplotlib library is used to visually display the analysis results.

[1261] Step 7:

[1262] The server proposes the optimal route based on the analysis results. Based on the analysis results in real time, it calculates the most efficient detour route for the driver. The input data is the analysis results, and the output data is the proposed optimal route.

[1263] Step 8:

[1264] The terminal receives the data sent from the server and requests specific traffic volume data according to the user's request. The request includes the specification of a specific time period or road section, and obtains the corresponding data from the server.

[1265] Step 9:

[1266] The device analyzes the traffic volume data it receives and displays it in statistical graphs and heat maps, allowing users to intuitively understand the data and assisting them in selecting the optimal route.

[1267] Step 10:

[1268] The user selects traffic volume data for a specific time period or road section through the application, and the device sends a request to the server. Based on the acquired data, the user can intuitively understand the traffic situation and act accordingly.

[1269] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1270] This invention provides a system called "Traffic AI Plus," which combines a highly automated system for road traffic volume surveys with an emotion engine that recognizes user emotions. This system receives video data from a camera in real time, identifies vehicles using a generative AI model, and measures and analyzes traffic volume. The emotion engine also recognizes user emotions and customizes the presentation of analysis results. The specific roles of the server, terminal, and user, as well as the processing involved, are described in detail below.

[1271] Server Roles and Operations

[1272] Video data reception and preprocessing

[1273] The server receives real-time video data from the camera, which is then processed immediately. The received video data is pre-processed into a format suitable for the generative AI model. Pre-processing includes cropping the image frame by frame, removing noise, and adjusting the resolution.

[1274] Vehicle identification and storage in a database

[1275] The preprocessed video data is then fed into a generative AI model. This model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. The identified vehicle types (cars, trucks, motorcycles, etc.) and their counts are output in JSON format and stored in a traffic database. The data also includes timestamps and vehicle location information.

[1276] Data analysis and visualization

[1277] The data stored in the traffic volume database is analyzed on an ongoing basis using analytical tools. This analysis includes generating statistical data and extracting patterns. The analyzed data is then visualized using visualization tools, and displayed as graphs and heat maps showing fluctuations in traffic volume.

[1278] Terminal roles and processing

[1279] Data Request and Receipt

[1280] The terminal sends a request for traffic volume data to the server in response to a user request. The request includes a specification of a specific time period and road section. The server then retrieves the relevant data from the traffic volume database and sends it to the terminal.

[1281] Data analysis and presentation

[1282] The device analyzes the received traffic data and displays it in a user-friendly format, using statistical graphs and heat maps. Furthermore, an emotion engine customizes the analysis results to reflect the user's emotions.

[1283] User operations

[1284] Operating the application and checking the data

[1285] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section. For example, if the user wants to check traffic volume on a specific road at night, they set the condition "traffic volume on major road A from 22:00 to 22:30" in the app. Based on this condition, the device sends a request to the server and displays the retrieved data to the user.

[1286] Emotion recognition and customized alerts

[1287] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions, and then customizes traffic information displays and alerts accordingly. For example, if the user is nervous, the device will suggest a relaxing route, and if the user is happy, the device will suggest the shortest route.

[1288] Decision-making and action

[1289] Users can view the visualized data and make on-the-fly decisions as needed, such as choosing an alternative route to avoid heavy traffic on major road A. Urban planners can also use the insights gained to formulate transportation policies and infrastructure development plans.

[1290] This invention not only enables efficient and accurate survey and analysis of road traffic volume, but also enables the provision of customized traffic information that takes user emotions into consideration, which will greatly support the formulation of urban planning and transportation policies and improve user satisfaction.

[1291] The processing flow will be explained below.

[1292] Server Processing

[1293] Step 1:

[1294] The server receives real-time video data from the camera. The video data is transmitted at a constant frame rate, and the server receives it sequentially.

[1295] Specific operation: Receives video data from the camera at 10 fps (frames per second).

[1296] Step 2:

[1297] The server pre-processes the received video data, including cutting out the image frame by frame, removing noise, and adjusting the resolution.

[1298] Specific operation: Resize each frame to 256x256 pixels and remove noise from the image.

[1299] Step 3:

[1300] The server inputs the preprocessed data into a generative AI model to identify vehicles. The generative AI model then identifies vehicles in the video with high accuracy and determines their type (cars, trucks, motorcycles, etc.) and number.

[1301] Specific operation: In the first frame, two cars and one truck are identified, and in the second frame, one motorcycle is identified.

[1302] Step 4:

[1303] The server stores the identified vehicle data in a traffic database, including vehicle type, number, time stamp, and location information.

[1304] Specific operation: Save the data of two cars and one truck at 2023-10-01 22:05:10 in the database.

[1305] Terminal handling

[1306] Step 5:

[1307] The terminal sends a request for traffic data to the server based on the user's request, which includes a specification of a particular time period and road section.

[1308] Specific operation: Send a message to the server requesting traffic volume data for major road A from 22:00 to 22:30.

[1309] Step 6:

[1310] The terminal analyzes the traffic volume data received from the server, which includes formatting the data and statistical compilation.

[1311] Specific operation: Calculate the total number of vehicles and their breakdown on main road A from 22:00 to 22:30.

[1312] Step 7:

[1313] The device displays the analysis results in a user-friendly format, including generating graphs and heat maps showing traffic fluctuations.

[1314] Specific operation: Displays the traffic volume on major road A as a line graph, showing fluctuations by time period.

[1315] Step 8:

[1316] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions, and displays and alerts are customized based on the recognized emotions.

[1317] What it does: If the user is nervous, provide them with directions and travel information to help them relax.

[1318] User operations

[1319] Step 9:

[1320] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section.

[1321] Specific operation: Set the condition "traffic volume on major road A from 22:00 to 22:30" on the application.

[1322] Step 10:

[1323] Users can view visualized data and make on-the-fly decisions as needed.

[1324] Specific operation: Select a route that avoids times when traffic on major road A is heavy.

[1325] These are the specific processing steps of the server, terminal, and user. This not only enables efficient and accurate survey and analysis of road traffic volume, but also provides customized traffic information that takes user emotions into consideration.

[1326] Example 2

[1327] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1328] While conventional road traffic volume survey systems can measure traffic volume using real-time video data, they have the problem of being unable to provide customized information that takes user emotions into account.Furthermore, they have a problem of not being able to identify vehicles with a high degree of accuracy even in nighttime video data.

[1329] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1330] In this invention, the server includes means for receiving real-time video data from a camera, means for preprocessing the video data, means for identifying vehicles using a generative artificial intelligence model, means for storing the identified vehicle data in a traffic volume database, means for analyzing and visualizing the traffic volume data, and means for recognizing a user's emotions and customizing the presentation method of the analysis results based on the emotions. This not only enables efficient and accurate road traffic volume survey and analysis, but also enables the provision of customized traffic information that takes the user's emotions into consideration.

[1331] A "camera" is a photographing device for capturing video data in real time.

[1332] "Real-time video data" refers to data that is obtained and transmitted immediately as video information at the current time.

[1333] "Preprocessing" refers to a series of operations that convert video data into a format suitable for subsequent analysis and recognition.

[1334] A "generative artificial intelligence model" is a machine learning model that learns from data and performs a specific task (e.g., vehicle identification).

[1335] "Means for identifying vehicles" refers to technologies or algorithms for detecting vehicles in video data and identifying their type and number.

[1336] "Vehicle data" refers to data including the type, number, timestamp, location information, etc. of identified vehicles.

[1337] A "traffic database" is a database system for storing identified vehicle data.

[1338] "Analysis" refers to the process of generating statistical data and extracting patterns based on traffic volume data.

[1339] "Visualization" refers to displaying analyzed data in a visual format such as a graph or heat map.

[1340] "Means for recognizing user emotions" refers to technology or devices that analyze emotions from the user's facial expressions and tone of voice.

[1341] "Customization means" refers to technologies and algorithms that change the way information is presented and the format of the display based on the user's emotions.

[1342] This invention relates to a system called "Traffic AI Plus," which combines a highly automated system for road traffic volume surveys with a function for recognizing user emotions. This system receives video data from a camera in real time, identifies vehicles using a generative AI model, and measures and analyzes traffic volume. It also has a function for recognizing user emotions using an emotion engine and customizing the way the analysis results are presented.

[1343] Server Roles

[1344] Video data reception and preprocessing

[1345] The server receives real-time video data from the camera. This video data is captured at 24 frames per second and temporarily stored in the server's buffer. The video data is then pre-processed. This includes cropping the image frame by frame using the OpenCV library, applying a noise reduction filter, and unifying the resolution.

[1346] Vehicle Identification

[1347] The preprocessed video data is input into a generative AI model. This generative AI model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. Specifically, it uses the prompt "How many vehicles are there in this frame?" to identify the type and number of vehicles.

[1348] Saving to a database

[1349] The identified vehicle data is converted to JSON format and stored in a traffic database. The data includes timestamps and vehicle location information. The server uses a NoSQL database (e.g., MongoDB) to store this data.

[1350] Data analysis and visualization

[1351] The data stored in the traffic volume database is analyzed using an analysis tool. For analysis, the Python pandas library is used to generate statistical data, and the matplotlib library is used to visualize the data, generating graphs and heat maps showing fluctuations in traffic volume.

[1352] Device Role

[1353] Sending and Receiving Data Requests

[1354] In response to a user request, the device requests traffic volume data for a specific time period or road section from the server. This request uses the HTTP protocol, and the corresponding traffic volume data is received from the server in JSON format.

[1355] Data analysis and presentation

[1356] The received traffic volume data is analyzed on the device, using the JavaScript D3.js library to generate interactive graphs and heat maps. Furthermore, the device's emotion engine customizes the display format based on the user's emotions.

[1357] User operations

[1358] Launching the application and checking the data

[1359] The user launches the application and sets conditions for checking specific traffic volume data, for example, "traffic volume on major road A from 22:00 to 22:30 on October 1, 2023."

[1360] Emotion Recognition and Customization

[1361] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions. Based on this, the display format of traffic information and alerts can be customized. For example, if the user is feeling stressed, the device will display information in a color scheme that has a relaxing effect.

[1362] Specific examples

[1363] It receives video data in real time, preprocesses it, and then inputs a prompt to the generative AI model: "How many vehicles are there in this frame?" The resulting data is then stored in a traffic database and analyzed using analytical tools, providing traffic information to users in an easy-to-understand format.

[1364] In this way, Traffic AI Plus is a system that realizes efficient and accurate surveys and analysis of road traffic volume, and also provides customized traffic information according to user sentiment. This system will greatly contribute to the formulation of urban planning and transportation policies, and will improve user satisfaction.

[1365] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1366] Step 1:

[1367] Receiving real-time video data

[1368] The server receives real-time video data from the camera. Specifically, the camera transmits video at 24 frames per second, and the server temporarily stores the data in a buffer. The input is the video data from the camera, and the output is the video data stored in the buffer. This data is used for subsequent processing steps.

[1369] Step 2:

[1370] Video data preprocessing

[1371] The server preprocesses the video data stored in the buffer. Specific preprocessing tasks include using the OpenCV library to crop the image frame by frame, applying a noise reduction filter, and unifying the resolution to 1080p. The input is the video data in the buffer, and the output is the preprocessed image data.

[1372] Step 3:

[1373] Vehicle Identification with Generative AI Models

[1374] The server inputs the preprocessed video data into a generative AI model, which then inputs the prompt "How many vehicles are there in this frame?" to identify the vehicles. The input is the preprocessed image data and the prompt, and the output is JSON-formatted data containing the type and number of identified vehicles.

[1375] Step 4:

[1376] Data storage

[1377] The server stores the identified vehicle data in a traffic volume database. Specifically, it uses a NoSQL database (e.g., MongoDB) to store JSON data including vehicle type, number, timestamp, and location information. The input is the identified vehicle data, and the output is the traffic volume data stored in the database.

[1378] Step 5:

[1379] Data analysis and visualization

[1380] The server analyzes and visualizes the data stored in the traffic volume database. For analysis, it uses Python's pandas library to generate statistical data and the matplotlib library to create visualizations such as heat maps and graphs. The input is the data in the traffic volume database, and the output is the visual data as the analysis results.

[1381] Step 6:

[1382] Sending a Data Request

[1383] In response to a user request, the terminal requests traffic volume data for a specific time period or road section from the server. Specific conditions are entered using an HTTP request, and the corresponding data is received from the server. The input is the conditions set by the user, and the output is the traffic volume data received from the server.

[1384] Step 7:

[1385] Data reception and analysis

[1386] The terminal analyzes the traffic volume data received from the server. It uses the D3.js library to create interactive graphs and heat maps and display them to the user. The input is the traffic volume data received from the server, and the output is the visualized data.

[1387] Step 8:

[1388] Emotion-aware customization

[1389] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions. Based on this, the display format is customized. For example, if the user is feeling stressed, the data is displayed in a color scheme that has a relaxing effect. The input is the user's facial expressions and voice data, and the output is a customized display format.

[1390] Step 9:

[1391] Application launch and condition settings

[1392] The user launches the application and sets a condition to check specific traffic volume data, for example, "traffic volume on major road A from 22:00 to 22:30 on October 1, 2023." The input is the condition setting, and the output is a data request based on the condition setting.

[1393] Step 10:

[1394] Data review and decision making

[1395] The user checks the visualized traffic data displayed on the device and decides on actions as necessary, for example, changing the route to avoid heavy traffic periods. The input is the visualized traffic data, and the output is the user's decision and actions.

[1396] In this way, Traffic AI Plus conducts efficient and accurate road traffic volume surveys and analysis through each step, and also provides information that responds to the user's emotions.

[1397] (Application example 2)

[1398] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1399] Conventional road traffic volume survey and analysis systems provide real-time traffic volume information, but they have not been able to provide customized information that takes user emotions into account. Furthermore, when proposing routes for autonomous vehicles, they lack the ability to respond to user emotions, making it difficult to ensure safe and comfortable driving at all times. Given this background, there is a growing need for a system that analyzes traffic volume in real time and customizes information based on user emotions.

[1400] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving real-time video data from a camera, means for preprocessing the video data, means for identifying vehicles using a generative artificial intelligence model, means for storing the identified vehicle data in a traffic volume database, means for analyzing and visualizing the traffic volume data, means for customizing the presentation method using an emotion engine that recognizes user emotions, and means for proposing a comfortable and safe route in an autonomously driven vehicle. This makes it possible to analyze traffic volume information in real time and provide customized information according to the user's emotions.

[1401] The "means for receiving real-time video data from a camera" refers to a system or device for receiving video data acquired from a camera in real-time and transmitting the data to another processing unit.

[1402] "Means for preprocessing video data" refers to a system or device that performs preprocessing such as frame cropping, noise removal, and resolution adjustment in order to convert the video data received from the camera into a format suitable for the generative AI model.

[1403] "Means for identifying vehicles using generative artificial intelligence models" refers to software or algorithms that use generative AI models trained on large amounts of data to identify vehicles in video footage with high accuracy.

[1404] "Means for storing identified vehicle data in a traffic volume database" refers to a system or device that automatically stores vehicle data (such as vehicle type, number, and location information) identified by the generative AI model in a traffic volume database in JSON format or similar.

[1405] "Means for analyzing and visualizing traffic volume data" refers to a system or device that statistically analyzes data stored in a traffic volume database and visualizes it using visualization tools as graphs or heat maps.

[1406] "Means for customizing presentation methods using an emotion engine that recognizes user emotions" refers to software or algorithms that analyze the user's facial expressions and tone of voice to recognize emotions, and customize the way traffic information and routes are presented based on that.

[1407] "Means for proposing comfortable and safe routes within an autonomous vehicle" refers to a navigation system or algorithm that proposes and executes optimal routes for an autonomous vehicle based on traffic volume data and user emotion data.

[1408] This invention, the "Traffic AI Plus" system, analyzes road traffic volume in a highly automated manner and also recognizes user emotions to provide customized information. It is particularly focused on operation in autonomous vehicles. The system has three main components: a server, a terminal, and a user.

[1409] Server Roles and Operations

[1410] The server performs the following functions:

[1411] Video data reception and preprocessing

[1412] The server receives real-time video data from the camera. This data begins processing immediately. The video data is pre-processed into a format suitable for the generative AI model. Pre-processing includes cropping the image frame by frame, removing noise, and adjusting the resolution. This allows the generative AI model to identify vehicles with high accuracy.

[1413] Vehicle identification and storage in a database

[1414] The preprocessed video data is then fed into a generative AI model. This model is trained using a large amount of data and is able to identify vehicles in the video with high accuracy. The identified vehicle types (cars, trucks, motorcycles, etc.) and their counts are output in JSON format and stored in a traffic database. The data also includes timestamps and vehicle location information.

[1415] Data analysis and visualization

[1416] The data stored in the traffic volume database is analyzed on an ongoing basis using analytical tools. This analysis includes generating statistical data and extracting patterns. The analyzed data is then visualized using visualization tools, and displayed as graphs and heat maps showing fluctuations in traffic volume.

[1417] Terminal roles and processing

[1418] The terminal implements the following functions:

[1419] Data Request and Receipt

[1420] The terminal sends a request for traffic volume data to the server in response to a user request. The request includes a specification of a specific time period and road section. The server then retrieves the relevant data from the traffic volume database and sends it to the terminal.

[1421] Data analysis and presentation

[1422] The device analyzes the received traffic data and displays it in a user-friendly format, using statistical graphs and heat maps. Furthermore, the analysis results are customized based on the user's emotions using an emotion engine.

[1423] User roles and operations

[1424] The user performs the following operations:

[1425] Operating the application and checking the data

[1426] The user launches the application and sets conditions for checking traffic volume data for a specific time period or road section. For example, if they want to check traffic volume on a specific road at night, they set the condition "traffic volume on major road A from 22:00 to 22:30" in the app. Based on this condition, the device sends a request to the server and displays the retrieved data to the user.

[1427] Emotion recognition and customized alerts

[1428] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions, and then customizes traffic information displays and alerts accordingly. For example, if the user is nervous, the device will suggest a relaxing route, and if the user is happy, the device will suggest the shortest route.

[1429] Decision-making and action

[1430] Users can view the visualized data and make on-the-fly decisions as needed, such as choosing an alternative route to avoid heavy traffic on major road A. Urban planners can also use the insights gained to formulate transportation policies and infrastructure development plans.

[1431] Specific examples

[1432] For example, suppose a child looks sleepy while a family is on a road trip. The system recognizes the child's sleepy expression through the in-car camera and selects a quiet and comfortable route. If the parent looks excited, the system suggests the shortest route to arrive smoothly. In this way, customized information is provided that takes into account the user's emotions in real time.

[1433] Prompt Sentence Examples

[1434] "Create an application that suggests a quieter route with less traffic if the user is sleepy, and selects the shortest route if the user is excited. The data used will be real-time camera footage, the user's facial expressions, and audio data."

[1435] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1436] Step 1:

[1437] The server receives real-time video data from the camera. It takes the video stream as input and outputs video data (frame by frame). Specifically, the server monitors the camera stream and immediately captures the captured video data.

[1438] Step 2:

[1439] The server preprocesses the received video data. The input is the video data acquired in step 1, and the output is noise-removed, resolution-adjusted frames (preprocessed video data). Specifically, the server uses OpenCV to remove noise and adjust the resolution, and converts each frame into an appropriate format.

[1440] Step 3:

[1441] The server uses a generative AI model to identify vehicles from the preprocessed video data. The input is the preprocessed video data output in step 2, and the output is data on identified vehicles (vehicle type, number, and location information). Specifically, the server applies the generative AI model using Keras / TensorFlow to identify vehicles in the video with high accuracy.

[1442] Step 4:

[1443] The server stores the identified vehicle data in the traffic volume database. The input is the vehicle data output in step 3, and the updated traffic volume database is output. Specifically, the server converts the vehicle data into JSON format and automatically stores it in the database.

[1444] Step 5:

[1445] The server analyzes and visualizes traffic volume data. The input is data from the traffic volume database, and the output is analysis results and visualized data (graphs, heat maps). Specifically, analysis tools are used to generate statistical data and extract patterns, and visualization tools are used to visualize the results.

[1446] Step 6:

[1447] The terminal sends a request for traffic volume data to the server in response to a user request. The input is the request conditions from the user, and a request message is output. Specifically, the user sets conditions for specific time periods and road sections within the application, and the request is made according to those conditions.

[1448] Step 7:

[1449] The server acquires traffic volume data in response to a request from the terminal and sends it to the terminal. The input is the request message sent in step 6, and the corresponding traffic volume data is output. Specifically, the server acquires the relevant data from the traffic volume database and sends it back to the terminal.

[1450] Step 8:

[1451] The terminal analyzes the received traffic volume data and displays it in a format that is easy for the user to understand. The input is the traffic volume data acquired in step 7, and visualized data (statistical graphs, heat maps) is output. Specifically, the terminal processes the received data using analysis tools and displays it visually on the user interface.

[1452] Step 9:

[1453] The device's emotion engine analyzes the user's facial expressions and tone of voice to recognize emotions. The input is raw data from the in-car camera and microphone, and the output is the user's emotional state. Specifically, the device processes the data using an emotion recognition algorithm to determine the user's emotions in real time.

[1454] Step 10:

[1455] The device customizes the way traffic information and routes are presented based on the user's emotions. The input is the emotion data recognized in step 9 and the traffic volume data acquired in step 8, and customized route suggestions and display formats are output. Specifically, if the user is nervous, a relaxing route is suggested, and if the user is happy, the shortest route is displayed.

[1456] Step 11:

[1457] The user checks the visualized data and selects a route. The input is the visualized data and route proposals presented by the terminal, and the selected route is output. Specifically, the user looks at the application, selects the optimal route, and makes another request if necessary.

[1458] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1459] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1460] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1461] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1462] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1463] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1464] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1465] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1466] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1467] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1468] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1469] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1470] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1471] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1472] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1473] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1474] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1475] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1476] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1477] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1478] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1479] The following is further disclosed regarding the above embodiment.

[1480] (Claim 1)

[1481] means for receiving real-time video data from the camera;

[1482] means for preprocessing the video data;

[1483] means for identifying a vehicle using a generative artificial intelligence model;

[1484] means for storing the identified vehicle data in a traffic database;

[1485] A means of analyzing and visualizing traffic data;

[1486] A system including:

[1487] (Claim 2)

[1488] The system according to claim 1, characterized in that the generative artificial intelligence model identifies vehicles with high accuracy even in nighttime video data.

[1489] (Claim 3)

[1490] The system of claim 1, further comprising: providing insights based on the analyzed traffic data.

[1491] "Example 1"

[1492] (Claim 1)

[1493] means for receiving real-time video data from the camera;

[1494] means for preprocessing the video data;

[1495] means for identifying a vehicle using a generative artificial intelligence model;

[1496] means for storing the identified vehicle data in an information storage device;

[1497] A means of analyzing and visualizing traffic data;

[1498] A system including:

[1499] (Claim 2)

[1500] A means for the generative artificial intelligence model to identify vehicles with high accuracy even in nighttime video data;

[1501] a means for requesting traffic volume data for a specified time period or road section, and retrieving the corresponding data from an information storage device in response;

[1502] A means to analyze the received data and display it as statistical graphs or heat maps,

[1503] 10. The system of claim 1, comprising:

[1504] (Claim 3)

[1505] A means for providing an interface that displays the analyzed traffic volume data in a form that can be checked by users and supports decision-making in transportation planning and route selection;

[1506] 10. The system of claim 1, comprising:

[1507] "Application Example 1"

[1508] (Claim 1)

[1509] means for receiving real-time video data from the camera;

[1510] means for preprocessing the video data;

[1511] means for identifying a vehicle using a generative artificial intelligence model;

[1512] means for storing the identified vehicle data in a traffic database;

[1513] A means of analyzing and visualizing traffic data;

[1514] A means of suggesting optimal routes based on the analyzed traffic data;

[1515] A system including:

[1516] (Claim 2)

[1517] The system described in claim 1, characterized in that the generative artificial intelligence model can identify vehicles with high accuracy even in nighttime video data and provide optimal detour routes in real time based on analyzed traffic volume data.

[1518] (Claim 3)

[1519] The system according to claim 1, characterized in that it presents traffic conditions as heat maps and statistical graphs based on the analyzed traffic volume data, allowing drivers to intuitively grasp the information.

[1520] "Example 2: Combining Emotion Engines"

[1521] (Claim 1)

[1522] means for receiving real-time video data from the camera;

[1523] means for preprocessing the video data;

[1524] means for identifying a vehicle using a generative artificial intelligence model;

[1525] means for storing the identified vehicle data in a traffic database;

[1526] A means of analyzing and visualizing traffic data;

[1527] a means for recognizing a user's emotions and customizing the presentation of analysis results based on the emotions;

[1528] A system including:

[1529] (Claim 2)

[1530] The system according to claim 1, characterized in that the generative artificial intelligence model identifies vehicles with high accuracy even in nighttime video data.

[1531] (Claim 3)

[1532] The system according to claim 1, characterized in that it provides insights based on the analyzed traffic data, and the way in which the insights are presented is customized based on the user's emotions.

[1533] "Application example 2 when combining emotion engines"

[1534] (Claim 1)

[1535] means for receiving real-time video data from the camera;

[1536] means for preprocessing the video data;

[1537] means for identifying a vehicle using a generative artificial intelligence model;

[1538] means for storing the identified vehicle data in a traffic database;

[1539] A means of analyzing and visualizing traffic data;

[1540] a means for customizing the presentation using an emotion engine that recognizes user emotions;

[1541] A means of proposing a comfortable and safe route in an autonomous vehicle;

[1542] A system including:

[1543] (Claim 2)

[1544] The system according to claim 1, characterized in that the generative artificial intelligence model identifies vehicles with high accuracy even in nighttime video data.

[1545] (Claim 3)

[1546] The system of claim 1, further comprising: providing insights based on the analyzed traffic data. [Explanation of symbols]

[1547] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving real-time video data from the camera; means for preprocessing the video data; means for identifying a vehicle using a generative artificial intelligence model; means for storing the identified vehicle data in a traffic database; A means of analyzing and visualizing traffic data; A system including:

2. 2. The system according to claim 1, wherein the generative artificial intelligence model identifies vehicles with high accuracy even in nighttime video data.

3. The system of claim 1, further comprising: providing insights based on the analyzed traffic volume data.

Images