System

Abstract

To provide a system capable of effectively optimizing the flow of traffic by suppressing the occurrence of temporary and local congestion.SOLUTION: The server includes a unit for collecting video data obtained by a camera, a unit for preprocessing the video data obtained by the camera, a unit for analyzing the collected video data to evaluate traffic, a unit for executing a signal control algorithm based on an evaluation result, a unit for generating a signal control instruction and sending the signal control instruction to a signal light, a unit for adjusting a lighting time of the signal light based on the signal control instruction, and a unit for collecting and analyzing feedback data. This makes it possible to monitor the traffic situation in real time and perform appropriate traffic signal control.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traffic congestion is one of the major problems in urban areas, and it is widely known that it results in increased economic losses and environmental impacts. Conventional traffic signal systems often operate at fixed timings, making it difficult to flexibly respond to real-time traffic conditions. As a result, they are unable to respond appropriately to temporary traffic concentrations or localized congestion, hindering efficient traffic flow. Furthermore, conventional systems have limitations in traffic volume analysis and signal control optimization, and no satisfactory solution has yet been provided. [Means for solving the problem]

[0005] In order to solve the above problems, the present invention provides the following means. First, a means for collecting video data acquired by a camera is provided. Next, a means for analyzing the collected video data and evaluating traffic volume is provided. A means for dynamically adjusting the illumination time of a traffic light based on the analysis results is provided. Furthermore, a means for transmitting signal control instructions to the traffic light is provided, enabling real-time signal control according to traffic conditions. Furthermore, by adding a means for pre-processing the video data acquired by the camera and a means for executing a signal control algorithm based on the evaluation results, more accurate optimization of traffic flow is achieved. This makes it possible to suppress the occurrence of temporary, localized congestion and effectively optimize traffic flow.

[0006] A "camera" is a photographic device installed at a traffic light to capture video data of traffic.

[0007] "Video data" refers to real-time traffic video information captured by a camera.

[0008] The "collection means" is a mechanism or function for acquiring video data obtained from a camera and transferring it to another device.

[0009] "Means for analyzing" refers to a mechanism or function that uses image recognition algorithms and data processing techniques to assess traffic volume from collected video data.

[0010] "Traffic volume" is the number of vehicles passing through a specific point or section per unit of time.

[0011] "Means of evaluation" refers to a mechanism or function that judges and determines the current traffic situation as a numerical value or pattern based on the analysis results.

[0012] "On time" is the length of time a traffic light stays lit in a particular color (red, green, or yellow).

[0013] "Dynamic adjustment means" refers to a mechanism or function that changes and optimizes the lighting time of traffic lights in real time according to fluctuations in traffic volume.

[0014] A "signal control instruction" is a specific control command for changing the lighting time or sequence of a traffic light.

[0015] A "traffic light" is a flashing indicator installed at an intersection or on a road to control traffic.

[0016] "Means for transmitting to traffic lights" refers to a mechanism or function for transmitting control instructions from a server or control center to traffic lights via communication.

[0017] An "algorithm" is a set of steps or computational techniques designed to solve a particular problem.

[0018] "Real-time" means processing and responding immediately to the situation occurring in the present moment.

[0019] "Preprocessing" refers to the initial data conversion and adjustment work performed on the original video data to make it easier to analyze and process. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6]FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] The present invention relates to a system for mitigating traffic congestion by utilizing cameras installed at traffic lights. Specific embodiments of the system are described below.

[0042] (Data collection and preprocessing)

[0043] Terminal

[0044] The terminal is responsible for acquiring video data in real time from cameras installed at traffic lights. The acquired video data undergoes pre-processing to analyze traffic volume and vehicle congestion. Specifically, the device performs noise removal and contrast adjustment on the video data.

[0045] (Data analysis and traffic volume evaluation)

[0046] server

[0047] The server analyzes pre-processed video data sent from the devices to assess traffic volume. The analysis is performed using image recognition algorithms to detect and count the number of vehicles and their movements. This data is used to assess the traffic situation at specific intersections and road sections in real time.

[0048] (Execution of signal control algorithm)

[0049] server

[0050] The server then executes a traffic light control algorithm based on the results of its assessment of traffic volume and congestion. This algorithm calculates the optimal signal lighting time for each intersection. For example, dynamic adjustments are made, such as extending the green light time at intersections where vehicles are congested.

[0051] (Generation and transmission of signal control instructions)

[0052] server

[0053] The server generates signal control instructions based on the calculated optimal lighting time. These instructions are instantly sent to each traffic light, and the lighting time of the traffic light is adjusted in real time. For example, an instruction to extend the green light at intersection A by 30 seconds is sent from the server to the terminal.

[0054] (Traffic signal adjustment)

[0055] Terminal

[0056] The terminal receives the signal control instruction sent from the server and automatically adjusts the lighting time of the traffic light, for example, by extending the green light time by 30 seconds.

[0057] (Feedback collection and analysis)

[0058] Terminal

[0059] After adjusting the traffic lights, the terminal again collects traffic data based on the new signal pattern and transmits it to the server.

[0060] server

[0061] The server analyzes this feedback data and continuously improves the signal control algorithm, allowing it to flexibly respond to changes in traffic conditions and optimize the entire system.

[0062] (Example)

[0063] Example 1: Reducing congestion at intersection A

[0064] 1. Data Collection

[0065] The terminal acquires video data from a camera installed at intersection A.

[0066] Preprocess the video data and send it to the server.

[0067] 2. Data Analysis

[0068] The server analyzes the received video data using an image recognition algorithm and counts the number of vehicles at intersection A.

[0069] Based on the analysis results, the traffic volume and congestion situation at intersection A are evaluated.

[0070] 3. Execution of signal control algorithm

[0071] The server executes a traffic light control algorithm to extend the green light duration at intersection A by 30 seconds.

[0072] 4. Transmission of signal control instructions

[0073] The server transmits the generated signal control instruction to the terminal.

[0074] 5. Traffic light adjustments

[0075] The device adjusts and controls the traffic light to extend the green light duration by 30 seconds.

[0076] 6. Feedback Collection and Analysis

[0077] The terminal again collects traffic data based on the new signal pattern and transmits it to the server.

[0078] The server analyzes the feedback data and improves the traffic light control algorithm.

[0079] In this way, traffic flow can be optimized and congestion can be alleviated by dynamically adjusting the illumination duration of traffic lights based on the analysis of video data acquired by the camera.

[0080] The processing flow will be explained below.

[0081] Step 1:

[0082] Terminal

[0083] The device acquires video data in real time from cameras installed at traffic lights. Specifically, it acquires video within the camera's field of view and extracts frames at regular intervals. This video data is used as basic data for analyzing traffic volume and vehicle congestion.

[0084] Step 2:

[0085] Terminal

[0086] The device performs preprocessing on the captured video data, including noise reduction, contrast adjustment, and resolution optimization, to improve analysis accuracy and reduce the risk of misidentification.

[0087] Step 3:

[0088] Terminal

[0089] The pre-processed video data is sent to the server, which converts the data packets into an appropriate format and transfers them to the server in real time over the network.

[0090] Step 4:

[0091] server

[0092] The server receives pre-processed video data sent from the device, ready for immediate analysis.

[0093] Step 5:

[0094] server

[0095] The server analyzes the received video data using an image recognition algorithm. Specifically, it counts the number of vehicles in each frame and measures their movement speed and stationary time. Based on the results of this analysis, it evaluates traffic volume and stationary conditions.

[0096] Step 6:

[0097] server

[0098] Based on the results of the traffic volume and congestion assessment, the system executes a traffic light control algorithm that calculates the optimal lighting times (red, green, and yellow) for each intersection. For example, this may include extending the green light time on roads with heavy traffic.

[0099] Step 7:

[0100] server

[0101] Based on the calculated optimal lighting time, specific traffic light control instructions are generated, and the server converts these instructions into data packets for transmission to the target traffic light.

[0102] Step 8:

[0103] server

[0104] The server transmits the signal control instructions, including the traffic light control parameters (lighting time, sequence, etc.), to the terminal via the network.

[0105] Step 9:

[0106] Terminal

[0107] The terminal receives the signal control instructions sent from the server. This is applied to the internal control system to adjust the lighting time of the traffic light. Specifically, it controls the time the green light stays on by 30 seconds, for example.

[0108] Step 10:

[0109] Terminal

[0110] The operation of the adjusted traffic lights is monitored and traffic data is collected again based on the new signal pattern, which is then sent as feedback to the server.

[0111] Step 11:

[0112] server

[0113] The server receives feedback data from the devices and evaluates the effectiveness of the signal control algorithm. Based on the analysis results, the algorithm is improved and used for future adjustments.

[0114] Step 12:

[0115] User

[0116] Users can check current traffic conditions and traffic light change information through a dedicated app or website, and user feedback will also be incorporated into the system for further optimization.

[0117] Example 1

[0118] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0119] Conventional traffic systems face the challenge of effectively alleviating traffic congestion in real time. Specifically, they lack the ability to properly grasp ongoing traffic conditions and dynamically adjust traffic light illumination times accordingly. As a result, traffic flow becomes unsmooth and congestion occurs.

[0120] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0121] In this invention, the server includes means for collecting video data acquired by a camera, means for pre-processing the video data acquired by the camera, means for analyzing the collected video data to evaluate traffic volume, means for executing a signal control algorithm based on the evaluation results, means for generating and transmitting signal control instructions to the traffic lights, means for adjusting the illumination duration of the traffic lights based on the signal control instructions, and means for collecting and analyzing feedback data. This makes it possible to monitor traffic conditions in real time and perform appropriate traffic light control. Furthermore, the collected feedback data can be used to continuously improve the signal control algorithm, thereby effectively alleviating traffic congestion.

[0122] "Video data acquired by a camera" refers to video information captured in real time by a camera installed at an intersection or road section.

[0123] "Preprocessing" refers to a data cleaning process, such as noise removal and contrast adjustment, that is carried out before analyzing the video data acquired by the camera.

[0124] "Analysis" is a method of evaluating traffic volume and vehicle congestion based on preprocessed video data, and is carried out using image recognition algorithms.

[0125] "Evaluating traffic volume" means detecting the number and movement of vehicles at a particular intersection or road section from the analyzed data and determining the traffic situation.

[0126] A "signal control algorithm" refers to the calculation procedures and rules for calculating the optimal lighting time of a traffic light based on the results of traffic volume evaluation.

[0127] A "signal control instruction" is a command that indicates the lighting time of a specific traffic light, generated by a signal control algorithm.

[0128] "Adjusting the lighting time of a traffic light" means changing the display time of a green light, red light, etc. of a traffic light based on a signal control instruction.

[0129] "Feedback data" refers to new traffic data that is re-collected after adjusting traffic light durations and is used to continuously improve the algorithm.

[0130] A "traffic light" is a display device installed at an intersection or road section to control the flow of traffic.

[0131] MODE FOR CARRYING OUT THE INVENTION

[0132] The present invention is a system that uses cameras installed at traffic lights to alleviate traffic congestion. A specific embodiment of this system is described below.

[0133] Data collection and preprocessing

[0134] Terminal

[0135] The device acquires video data in real time from cameras installed at traffic lights. The hardware used includes commercially available surveillance cameras and a small computer for data collection. This video data undergoes preprocessing before analyzing traffic volume and vehicle congestion. Specifically, libraries such as OpenCV are used to remove noise and adjust contrast.

[0136] Data analysis and traffic volume evaluation

[0137] server

[0138] The server receives and analyzes the preprocessed video data sent from the device. This analysis uses image recognition algorithms such as YOLO (You Only Look Once) to detect and count vehicles in the video in real time. Based on the analysis results, the server evaluates the traffic volume and congestion status at specific intersections and road sections. The evaluation results are temporarily stored in a database.

[0139] Execution of signal control algorithms

[0140] server

[0141] The server then executes a traffic light control algorithm based on the results of its assessment of traffic volume and congestion. This algorithm calculates the optimal signal lighting time for each intersection. For example, at intersections where traffic volume has increased and congestion has occurred, dynamic adjustments are made, such as extending the green light time by 30 seconds.

[0142] Generation and transmission of signal control instructions

[0143] server

[0144] The server generates traffic light control instructions based on the calculated optimal lighting time. These instructions are instantly sent to each traffic light. Communication protocols such as MQTT are used to send specific instructions, such as "Extend the green light at intersection A by 30 seconds," to the terminal in JSON format.

[0145] Traffic light adjustments

[0146] Terminal

[0147] The terminal receives the signal control instructions sent from the server and automatically adjusts the traffic lights based on the instructions. A PLC (Programmable Logic Controller) is used to control the traffic lights, and the green light duration is extended as specified.

[0148] Collecting and analyzing feedback

[0149] Terminal

[0150] After adjusting the traffic lights, the device collects traffic data again based on the new signal patterns and sends it to the server. This data, also in the form of video data, is again pre-processed and analyzed.

[0151] server

[0152] The server analyzes this feedback data and adjusts parameters to continuously improve the signal control algorithm, allowing it to flexibly respond to changing traffic conditions.

[0153] Specific examples

[0154] As a specific example, the case of easing traffic congestion at intersection A will be shown below.

[0155] 1. Data Collection

[0156] The terminal acquires video data from a camera installed at intersection A, preprocesses it, and sends it to the server.

[0157] 2. Data Analysis

[0158] The server analyzes the received video data using an image recognition algorithm and counts the number of vehicles at intersection A.

[0159] Based on the analysis results, the traffic volume and congestion situation at intersection A are evaluated.

[0160] 3. Execution of signal control algorithm

[0161] The server executes a traffic light control algorithm to extend the green light duration at intersection A by 30 seconds.

[0162] 4. Transmission of signal control instructions

[0163] The server transmits the generated signal control instruction to the terminal.

[0164] 5. Traffic light adjustments

[0165] The device adjusts and controls the traffic light to extend the green light duration by 30 seconds.

[0166] 6. Feedback Collection and Analysis

[0167] The terminal again collects traffic data based on the new signal pattern and transmits it to the server.

[0168] The server analyzes the feedback data and improves the traffic light control algorithm.

[0169] Prompt Sentence Examples

[0170] "How will this system alleviate traffic congestion? What are the specific steps and technologies used?"

[0171] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0172] Step 1:

[0173] Data collection

[0174] Terminal

[0175] The terminal acquires video data in real time from cameras installed at traffic lights. Specifically, commercially available surveillance cameras capture video at a set frame rate. The input data is the captured raw video data. This data is preprocessed in the next step.

[0176] Step 2:

[0177] Video data preprocessing

[0178] Terminal

[0179] The device preprocesses the acquired video data. Specifically, it uses the OpenCV library to remove noise and adjust the contrast of the video data. The input data is raw video data. The output data is preprocessed video data with noise removed and contrast adjusted. This enables subsequent image analysis to be performed with high accuracy.

[0180] Step 3:

[0181] Data analysis

[0182] server

[0183] The server receives preprocessed video data sent from the device. The server analyzes this data using image recognition algorithms such as YOLO. The input data is the preprocessed video data, and the output data is the number and location of detected vehicles. Based on the analyzed data, traffic volume and congestion at specific intersections and road sections are evaluated.

[0184] Step 4:

[0185] Traffic volume evaluation

[0186] server

[0187] The server evaluates traffic volume based on the analysis results. Specifically, it calculates the actual traffic volume and congestion status at specific intersections and road sections based on the number of vehicles and location information obtained from the analysis. The input data is the number of vehicles and location information from the analysis results, and the output data is the evaluated traffic volume and congestion status. This evaluation result is temporarily stored in a database.

[0188] Step 5:

[0189] Execution of signal control algorithms

[0190] server

[0191] The server executes a signal control algorithm based on the evaluation results of traffic volume and congestion status. The input data is the traffic volume and congestion status of the evaluation results, and the output data is the optimal signal lighting time for each intersection. Specifically, at intersections where vehicles are congested, dynamic adjustments are made, such as extending the green light time.

[0192] Step 6:

[0193] Generation and transmission of signal control instructions

[0194] server

[0195] The server generates traffic light control instructions based on the calculated optimal lighting time. These instructions are instantly sent to each traffic light using a communication protocol such as MQTT. The input data is the optimized lighting time, and the output data is the specific control instructions for the traffic light. For example, this could include an instruction such as "Extend the green light at intersection A by 30 seconds."

[0196] Step 7:

[0197] Traffic light adjustments

[0198] Terminal

[0199] The terminal receives signal control instructions sent from the server. Based on the received instructions, the terminal controls the traffic light. The input data is the signal control instructions, and the output operation is to adjust the traffic light's lighting time. Specifically, the green light's lighting time is extended as specified. A PLC (Programmable Logic Controller) is used to control the traffic light.

[0200] Step 8:

[0201] Feedback data collection

[0202] Terminal

[0203] After the traffic light is adjusted, the terminal collects traffic data again according to the new signal pattern, and the input data is the video data under the new signal pattern, and the output data is the preprocessed video data.

[0204] Step 9:

[0205] Analysis of feedback data and algorithm improvement

[0206] server

[0207] The server receives and analyzes feedback data sent from the terminal. The input data is preprocessed feedback video data, and the output data is the analysis results of the number of vehicles and their location information. Based on these results, the signal control algorithm is continuously improved to achieve adaptive signal control.

[0208] (Application example 1)

[0209] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0210] Conventional traffic systems have difficulty in controlling signals in a fast and flexible manner to respond to fluctuations in traffic volume, and congestion has not been sufficiently alleviated. Furthermore, conventional navigation systems for autonomous vehicles are unable to reflect real-time traffic information and may not be able to provide optimal driving routes.

[0211] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0212] In this invention, the server includes means for collecting video data acquired by cameras, means for analyzing the collected video data to evaluate traffic volume, means for dynamically adjusting the illumination time of traffic signals based on the evaluation results, and means for providing traffic information in real time and optimizing vehicle routes, thereby enabling dynamic adjustment of traffic volume and optimal navigation.

[0213] A "camera" is a device installed at a traffic light that captures images of road traffic conditions in real time.

[0214] "Video data" refers to digital video recordings of traffic conditions captured by a camera.

[0215] "Collection means" is a function that collects video data acquired by a camera in real time into a database or server.

[0216] The "analysis means" is a function that uses collected video data to evaluate traffic volume and identify the number of vehicles and congestion conditions.

[0217] "Evaluation results" refer to data indicating traffic volume and congestion conditions obtained by the analysis means.

[0218] The "signal lighting time adjustment means" is a function for dynamically changing the lighting time of the green or red signal of a traffic light based on the evaluation result.

[0219] "Signal control instructions" are control information for instructing the lighting time and operating state of a traffic light.

[0220] A "traffic light" is an optical signal device installed at road intersections and other locations to control traffic flow.

[0221] "Real-time traffic information provision means" is a function that provides the current optimal driving route based on traffic data.

[0222] The "driving route optimization means" is a function that navigates the vehicle along the optimal route based on real-time traffic information.

[0223] A system for carrying out the present invention comprises a camera, a server, and a terminal. Specific embodiments will be described in detail below.

[0224] Terminal

[0225] First, cameras installed at traffic lights capture real-time video data of road traffic conditions. The captured video data undergoes preprocessing such as noise removal and contrast adjustment on the device. After preprocessing, the data is sent to the server.

[0226] server

[0227] The server uses an image recognition algorithm to evaluate traffic volume based on preprocessed video data sent from the device. Specifically, it counts the number of vehicles and detects their movements. Based on the analysis results, it evaluates traffic volume and congestion conditions.

[0228] By running the signal control algorithm on the server, the optimal signal lighting time for each intersection is calculated. For example, dynamic adjustments are made, such as extending the green light time at intersections where congestion is occurring. The calculated optimal signal control instructions are sent to each traffic light in real time.

[0229] The server also provides real-time navigation information based on traffic information, allowing it to present optimal driving routes to autonomous vehicles.

[0230] Autonomous vehicles

[0231] The autonomous vehicle receives real-time traffic information sent from the server and optimizes its route, for example by changing its route to coincide with the time when the green light remains green at a particular intersection for a long period of time.

[0232] Specific examples

[0233] Video data obtained by the camera at intersection A is preprocessed on the terminal and sent to the server. The server uses an analysis algorithm to count the number of vehicles and evaluate traffic volume. If congestion occurs, a signal control algorithm is executed to generate signal control instructions that dynamically extend the green light duration and send them to the traffic lights. Real-time traffic information is also sent to the navigation system, which displays the optimal driving route.

[0234] Prompt Sentence Examples

[0235] "I want to build an application that analyzes video data from a traffic light camera at intersection A and suggests the optimal driving route based on traffic congestion. Please tell me the code to evaluate the current traffic volume, obtain signal control information based on that data, and calculate the optimal route."

[0236] In this way, traffic lights can be dynamically adjusted based on the analysis of video data acquired by cameras, providing autonomous vehicles with optimal routes to optimize traffic flow and alleviate congestion.

[0237] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0238] Step 1: Acquire video data with the camera

[0239] Cameras installed at traffic lights capture traffic conditions in real time. The video data obtained from the cameras is digital footage that captures the movement of vehicles on the road. Specifically, the video captured by the cameras is sent to a terminal in streaming format. The input of this step is the camera image itself, and the output is the unprocessed video data.

[0240] Step 2: Preprocess the video data

[0241] The device receives video data captured by the camera and performs preprocessing such as noise reduction and contrast adjustment. Specifically, the video data is converted to grayscale using a library such as OpenCV, and noise is removed by applying a Gaussian filter. The input of this step is the raw video data, and the output is the preprocessed video data.

[0242] Step 3: Send the preprocessed data to the server

[0243] The terminal sends the preprocessed video data to the server. In this step, data is transferred to the server using a communication protocol such as an HTTP request. The input is the preprocessed video data, and the output is the data reception on the server.

[0244] Step 4: Analyze video data to assess traffic volume

[0245] The server analyzes the received preprocessed video data and evaluates traffic volume. This process uses an image recognition algorithm to count the number of vehicles and identify their movements. For example, it uses YOLO or OpenCV vehicle detection models. The input is the preprocessed video data, and the output is an evaluation result showing the number of vehicles and congestion status.

[0246] Step 5: Run the signal control algorithm

[0247] The server executes a signal control algorithm based on the evaluation results. Specifically, it dynamically adjusts signals, such as extending the green light duration at intersections with congestion. The input is evaluation data based on vehicle detection results, and the output is optimal signal control instructions.

[0248] Step 6: Send signal control instructions to the traffic light

[0249] The signal control instructions generated by the server are sent to the terminal to control the traffic lights. Here, signal control information is sent to the traffic lights using a communication protocol. The input is the signal control instructions, and the output is the transfer of instructions to the traffic lights.

[0250] Step 7: Providing real-time traffic information

[0251] The server provides real-time traffic information to autonomous vehicles based on traffic volume evaluation results and signal control information. Specifically, it sends route optimization information via API. The input is the latest traffic data, and the output is route optimization information.

[0252] Step 8: Optimize the autonomous vehicle's route

[0253] An autonomous vehicle optimizes its driving route based on traffic information received from a server. Here, the vehicle's navigation system analyzes the information from the server and presents the optimal route. The input is the optimized driving route information, and the output is the optimal route the vehicle should follow.

[0254] By going through the above processing steps, it is possible to optimize traffic flow and alleviate congestion.

[0255] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0256] This invention relates to a system that alleviates traffic congestion by utilizing cameras installed at traffic lights and combining them with an emotion engine that recognizes the emotions of users. Specific embodiments of this system are described below.

[0257] (Data collection and preprocessing)

[0258] Terminal

[0259] The device is responsible for acquiring video data in real time from cameras installed at traffic lights. The acquired video data undergoes preprocessing to analyze traffic volume and vehicle congestion. Specifically, noise removal and contrast adjustment are performed on the video data. The camera also captures the facial expressions of the vehicle's users, which are used as emotional data.

[0260] (Data analysis and traffic volume evaluation)

[0261] server

[0262] The server receives preprocessed video data and emotion data sent from the device. Using the received data, it analyzes traffic volume and then analyzes the user's emotions using an emotion engine. Traffic volume analysis uses an image recognition algorithm to detect and count the number of vehicles and their movements. Meanwhile, the emotion engine measures the user's stress level and irritation from their facial expressions.

[0263] (traffic volume and user sentiment evaluation)

[0264] server

[0265] The server comprehensively evaluates traffic volume and user emotion data, and determines that traffic conditions are deteriorating or the user's stress level is high, and prioritizes traffic light adjustment.

[0266] (Execution of signal control algorithm)

[0267] server

[0268] The server then executes a traffic light control algorithm based on the evaluation results. The algorithm calculates the optimal traffic light duration for each intersection. For example, if traffic volume is heavy and the user is feeling stressed, the algorithm dynamically adjusts the duration of the green light, such as extending it.

[0269] (Generation and transmission of signal control instructions)

[0270] server

[0271] The server generates signal control instructions based on the calculated optimal lighting time. These instructions are instantly sent to each traffic light, and the lighting time of the traffic light is adjusted in real time. For example, an instruction to extend the green light at intersection A by 30 seconds is sent from the server to the terminal.

[0272] (Traffic signal adjustment)

[0273] Terminal

[0274] The device receives the signal control instructions sent from the server and automatically adjusts the traffic light duration, for example, extending the green light duration by 30 seconds.

[0275] (Feedback collection and analysis)

[0276] Terminal

[0277] After the traffic light is adjusted, the terminal collects traffic data and user emotion data according to the new traffic light pattern again and transmits them to the server.

[0278] server

[0279] The server analyzes this feedback data and evaluates the effectiveness of the signal control algorithm. Based on the analysis results, the algorithm is improved and used for future adjustments.

[0280] (Example)

[0281] Example 1: Reducing congestion at intersection A

[0282] 1. Data Collection

[0283] The terminal acquires video data and emotion data from a camera installed at intersection A.

[0284] Preprocess the video data and send it to the server.

[0285] 2. Data Analysis

[0286] The server analyzes the received video data using an image recognition algorithm and counts the number of vehicles at intersection A.

[0287] An emotion engine is used to analyze the user's emotion data and assess their stress level.

[0288] 3. Evaluation of traffic volume and emotion data

[0289] The server comprehensively evaluates the traffic volume at intersection A and the user's emotional data to determine the traffic situation and the user's stress level.

[0290] 4. Execution of signal control algorithm

[0291] The server executes a traffic light control algorithm to extend the green light duration at intersection A by 30 seconds.

[0292] 5. Transmission of signal control instructions

[0293] The server transmits the generated signal control instruction to the terminal.

[0294] 6. Traffic light adjustment

[0295] The device adjusts and controls the traffic light to extend the green light duration by 30 seconds.

[0296] 7. Feedback Collection and Analysis

[0297] The terminal again collects traffic data and user emotion data based on the new signal pattern and transmits them to the server.

[0298] The server analyzes the feedback data and improves the traffic light control algorithm.

[0299] In this way, traffic lights can be dynamically adjusted based on the video data acquired by the camera and the user's emotional data obtained from the emotion engine, thereby optimizing traffic flow, easing congestion, and reducing user stress.

[0300] The processing flow will be explained below.

[0301] Step 1:

[0302] Terminal

[0303] The device acquires real-time video data from cameras installed at traffic lights. The cameras are high-resolution video cameras that capture the entire intersection and extract frames at regular intervals, providing data for monitoring temporary congestion and fluctuations in traffic volume.

[0304] Step 2:

[0305] Terminal

[0306] Preprocessing is performed on the video data captured by the camera. Specifically, processes such as noise removal, contrast adjustment, and resolution optimization are performed. This improves the quality of the video data and increases the accuracy of analysis. In addition, if the user is in the vehicle, their facial expressions are captured and extracted as emotion data.

[0307] Step 3:

[0308] Terminal

[0309] The preprocessed video data and emotion data are sent to the server. The data is set to be sent in real time via the network, allowing the server to constantly monitor the latest traffic conditions and the user's emotional state.

[0310] Step 4:

[0311] server

[0312] The server receives the preprocessed video data and emotion data sent from the device. It then applies an image recognition algorithm to analyze the received video data. Specifically, the analysis counts the number of vehicles in each frame and measures their movement and dwell time.

[0313] Step 5:

[0314] server

[0315] The server uses an emotion engine to analyze the user's emotional data. Specifically, it uses facial expression recognition technology to evaluate the user's stress level and irritation. This evaluation result is combined with traffic volume data to help understand the overall traffic situation.

[0316] Step 6:

[0317] server

[0318] The system evaluates traffic volume and user emotional data in an integrated manner. For example, when traffic volume is heavy and vehicles are backed up, and when a high stress level is detected from the user's emotional data, it determines that it is necessary to raise the priority of traffic signal control.

[0319] Step 7:

[0320] server

[0321] Based on the evaluation results, a traffic light control algorithm is executed. The algorithm calculates the optimal traffic light duration for each intersection. Dynamic adjustments are made, such as extending the green light duration at intersection A by 30 seconds.

[0322] Step 8:

[0323] server

[0324] It generates signal control instructions, including the specific time to extend the green light and the priority of roads, and converts these instructions into data packets for transmission to the traffic light controller.

[0325] Step 9:

[0326] server

[0327] A signal control instruction is sent to the terminal. The instruction is sent immediately via the network, and the server receives a response confirming receipt, thereby confirming that the instruction was transmitted correctly.

[0328] Step 10:

[0329] Terminal

[0330] The terminal receives the signal control instructions sent from the server. It applies these to the internal control system and sets the lighting time and sequence of the traffic lights. Specifically, it controls the signal by extending the green light time by 30 seconds, for example.

[0331] Step 11:

[0332] Terminal

[0333] The operation of the traffic lights after adjustment is monitored. Traffic data and user emotion data based on the new traffic light patterns are collected again and sent to the server.

[0334] Step 12:

[0335] server

[0336] The server receives feedback data from the devices and evaluates the effectiveness of the signal control algorithm. Based on the analysis results, the algorithm is improved and used for future adjustments.

[0337] Step 13:

[0338] User

[0339] Users can check current traffic conditions and traffic light change information through a dedicated app or website, and receive advice and notifications based on their stress and irritation levels to help them drive with peace of mind.

[0340] Example 2

[0341] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0342] Conventional traffic signal control systems are unable to respond quickly to fluctuations in traffic volume, making it difficult to effectively alleviate traffic congestion. Furthermore, because they do not take into account the user's stress or emotional state, drivers experience increased stress and the risk of traffic accidents increases as they spend longer at intersections. The present invention aims to solve these problems by analyzing traffic volume and the user's emotional state in real time and dynamically adjusting signal control.

[0343] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0344] In this invention, the server includes means for collecting video data captured by cameras installed at traffic lights in real time, means for performing noise reduction and contrast adjustment on the collected video data, means for capturing a user's facial expressions to collect emotion data, means for analyzing the preprocessed video data and emotion data to evaluate traffic volume and the user's emotional state, means for executing a signal control algorithm based on the evaluation results, means for generating and transmitting signal control instructions to the traffic lights, means for re-collecting new traffic data and emotion data based on the adjusted signal pattern, and means for analyzing the feedback data to improve the signal control algorithm, thereby enabling fast and effective signal control based on fluctuations in traffic volume and the user's emotional state.

[0345] A "traffic light" is an optical signaling device installed to control the flow of road traffic.

[0346] A "camera" is a photographing device for acquiring video data.

[0347] "Video Data" refers to visual information captured by a camera.

[0348] "Real-time" means that data acquisition and processing occur almost simultaneously.

[0349] "Noise reduction" is the process of removing unnecessary information and errors from collected video data.

[0350] "Contrast adjustment" is a process for appropriately adjusting the contrast between light and dark in the video data.

[0351] "Facial expressions" express emotions based on the shape and movement of the user's face.

[0352] "Emotion data" is information that represents the user's emotional state.

[0353] "Analysis" is the process of examining data in detail to clarify its content and meaning.

[0354] "Traffic volume" refers to the number and flow of vehicles in a particular time unit.

[0355] "Evaluation" is the act of judging a situation or condition based on data.

[0356] A "traffic light control algorithm" is a calculation method for determining the lighting time and control pattern of traffic lights.

[0357] A "signal control instruction" is a specific command for controlling the operation of a traffic light.

[0358] "Feedback data" is new data collected after traffic light adjustments.

[0359] "User" refers to the driver or passenger of a vehicle that uses this system.

[0360] This invention relates to a system that utilizes cameras installed at traffic lights and combines them with an emotion engine that recognizes the emotions of users to alleviate traffic congestion. Specific embodiments of this system are described below.

[0361] (System Configuration)

[0362] Hardware

[0363] Cameras installed at traffic lights:

[0364] The camera captures the traffic conditions at the intersection in real time, collecting video data, and also captures the user's facial expressions to obtain emotional data.

[0365] Device:

[0366] The terminal is installed at the traffic light, receives the video data from the camera, performs noise reduction and contrast adjustment, and sends the pre-processed data to the server.

[0367] server:

[0368] The server receives and analyzes the video and emotion data sent from the device. Based on the results of the analysis, it executes a traffic light control algorithm and generates instructions for the traffic light.

[0369] (software)

[0370] Image Recognition Algorithm

[0371] The server uses image recognition algorithms to analyze the number and movement of vehicles from pre-processed video data, allowing for accurate assessment of traffic volume.

[0372] Emotion Engine

[0373] The server uses an emotion engine to analyze the user's facial expression data and assess their stress level and irritation, thereby taking into account not only the traffic situation but also the user's emotional state.

[0374] Signal Control Algorithm

[0375] The server runs a traffic light control algorithm based on the results of traffic volume analysis and emotion analysis, which calculates the optimal signal lighting time for each intersection and dynamically adjusts the traffic lights.

[0376] Feedback Mechanism

[0377] After adjusting the traffic lights, the device will again collect traffic and emotion data based on the new signal patterns and send it to the server, which will analyze this feedback data, evaluate the effectiveness of the signal control algorithm, and make improvements.

[0378] (Example)

[0379] Example 1: Reducing congestion at intersection A

[0380] 1. Data Collection:

[0381] The terminal acquires video data and emotion data from a camera installed at intersection A. The acquired data is preprocessed and sent to the server.

[0382] 2. Data Analysis:

[0383] The server analyzes the received video data using an image recognition algorithm and counts the number of vehicles at intersection A. It uses an emotion engine to analyze the user's emotion data and evaluate their stress level.

[0384] 3. Traffic and Sentiment Data Evaluation:

[0385] The server comprehensively evaluates the traffic volume at intersection A and the user's emotional data to determine the traffic situation and the user's stress level.

[0386] 4. Execution of signal control algorithm:

[0387] The server executes a traffic light control algorithm to extend the green light duration at intersection A by 30 seconds.

[0388] 5. Sending signal control instructions:

[0389] The server transmits the generated signal control instruction to the terminal.

[0390] 6. Traffic light adjustment:

[0391] The device adjusts and controls the traffic light to extend the green light duration by 30 seconds.

[0392] 7. Feedback Collection and Analysis:

[0393] The terminal again collects traffic data and user emotion data based on the new signal pattern and transmits them to the server, which analyzes the feedback data and improves the signal control algorithm.

[0394] Prompt Sentence Examples

[0395] "Please explain a system that uses cameras and an emotion engine to collect and analyze traffic volume and user emotion data, and dynamically adjusts traffic light durations to alleviate congestion at intersections."

[0396] In this way, traffic lights can be dynamically adjusted based on the video data acquired by the camera and the user's emotional data obtained from the emotion engine, thereby optimizing traffic flow, easing congestion, and reducing user stress.

[0397] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0398] Step 1:

[0399] Data collection

[0400] The terminal acquires video data in real time from cameras installed at traffic lights.

[0401] Input: Video data from a camera installed at a traffic light.

[0402] Data processing and calculation: Capture of video data.

[0403] Output: Captured video data.

[0404] Step 2:

[0405] Video data preprocessing

[0406] The device performs noise removal and contrast adjustment on the acquired video data.

[0407] Input: Captured video data.

[0408] Data processing and calculation: noise removal and contrast adjustment.

[0409] Output: Pre-processed video data. Specific operations include applying noise filters and histogram equalization.

[0410] Step 3:

[0411] Collecting Emotional Data

[0412] The device's camera captures the facial expressions of the user riding in the vehicle and uses them as emotional data.

[0413] Input: Video data containing the user's facial expressions.

[0414] Data processing and calculation: Image analysis for facial expression recognition.

[0415] Output: User's emotional data. Specifically, the stress level and irritation level are measured from the user's facial expressions.

[0416] Step 4:

[0417] Video data analysis

[0418] The server receives the pre-processed video data sent from the device and analyzes it using image recognition algorithms.

[0419] Input: Preprocessed video data.

[0420] Data processing and calculation: The number and movement of vehicles are detected and counted using an image recognition algorithm.

[0421] Output: Parsed traffic data. Specific actions include vehicle identification and counting.

[0422] Step 5:

[0423] Emotional Data Analysis

[0424] The server uses an emotion engine to analyze the user's facial expression data and assess their stress level and irritability.

[0425] Input: User emotion data.

[0426] Data processing and calculation: Emotion analysis using an emotion engine.

[0427] Output: Analyzed emotional state data. Specifically, facial expression recognition technology is used to detect the user's micro-expressions and estimate their emotional state.

[0428] Step 6:

[0429] Integrated evaluation of traffic volume and emotion data

[0430] The server performs an integrated evaluation based on the analyzed traffic volume data and emotion data.

[0431] Input: Traffic data and emotional state data.

[0432] Data processing and calculation: Integrated evaluation of data.

[0433] Output: Integrated evaluation results. Specific actions include determining whether traffic conditions are worsening or the user's stress level is high.

[0434] Step 7:

[0435] Execution of signal control algorithms

[0436] The server executes the signal control algorithm based on the results of the integrated evaluation.

[0437] Input: Integrated evaluation results.

[0438] Data processing and calculation: Application of signal control algorithms.

[0439] Output: Optimal signal control instructions. Specifically, the optimal green and red light durations at each intersection are calculated.

[0440] Step 8:

[0441] Generation of signal control instructions

[0442] The server generates signal control instructions based on the results of the signal control algorithm.

[0443] Input: The result of the signal control algorithm.

[0444] Data processing and calculation: Generation of signal control instructions.

[0445] Output: Specific signal control instructions. Specifically, instructions such as "Extend the green light at intersection A by 30 seconds" are generated.

[0446] Step 9:

[0447] Sending signal control instructions

[0448] The server transmits the generated signal control instruction to the terminal.

[0449] Input: Signal control instructions.

[0450] Data processing and calculation: Sending signal control instructions.

[0451] Output: Signal control instructions sent to the terminal.

[0452] Step 10:

[0453] Traffic light adjustments

[0454] The terminal adjusts the lighting time of the traffic light based on the signal control instruction received from the server.

[0455] Input: Signal control instruction from the server.

[0456] Data processing and calculation: Traffic light control.

[0457] Output: Adjusted traffic lights. Specifically, the green light duration is extended by 30 seconds.

[0458] Step 11:

[0459] Collecting new data

[0460] After the traffic light is adjusted, the terminal collects traffic data and emotion data again according to the new traffic light pattern.

[0461] Input: Adjusted traffic data and sentiment data.

[0462] Data processing and calculation: Data collection.

[0463] Output: Recollected traffic data and emotion data.

[0464] Step 12:

[0465] Analysis of feedback data

[0466] The server again analyzes the received feedback data and evaluates the effectiveness of the signal control algorithm.

[0467] Input: Recollected traffic data and emotion data.

[0468] Data processing and calculation: Analysis of feedback data.

[0469] Output: Analysis results, specifically evaluating the effectiveness of signal conditioning and modifying algorithm parameters if necessary.

[0470] Through the above steps, traffic flow can be optimized, congestion can be alleviated, and user stress can be reduced by dynamically adjusting the illumination times of traffic lights based on the video data acquired by the camera and the user's emotional data obtained from the emotion engine.

[0471] (Application example 2)

[0472] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0473] Conventional traffic signal control systems rely solely on traffic volume and do not take into account passenger emotions or stress levels, which limits their effectiveness in easing traffic congestion and reducing passenger stress. This often leads to poor traffic efficiency and lower passenger satisfaction. Furthermore, conventional systems have fixed signal control, making real-time adjustments difficult and preventing flexible responses to changing situations.

[0474] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0475] In this invention, the server includes means for collecting video data acquired by a camera, means for analyzing the collected video data to evaluate traffic volume, and emotion recognition means for evaluating passenger emotions, thereby enabling traffic signal control based on traffic conditions and passenger emotion data in real time.

[0476] "Camera" refers to an image capture device installed on traffic lights and vehicles to capture traffic and passenger expressions.

[0477] "Video data" refers to real-time video information captured by a camera.

[0478] "Traffic volume" is a number or data that indicates the number or flow of vehicles on a road in a particular area or period of time.

[0479] "Signal duration" refers to the duration that each color of light remains on at a traffic light, and is an important parameter for traffic control.

[0480] "Signal control instructions" refer to commands to set specific lighting patterns and times for traffic lights.

[0481] "Emotion recognition means" is a technology that analyzes passengers' facial expressions from video data acquired by a camera and evaluates their emotions and stress levels.

[0482] "Emotion Data" refers to data indicative of passenger emotions and stress levels analyzed by the emotion recognition means.

[0483] The "evaluation result" is information that indicates a comprehensive assessment of the situation obtained from the analysis of traffic volume and emotion data.

[0484] A "traffic light control algorithm" is a method or program for calculating optimal traffic light lighting patterns based on traffic volume and emotion data.

[0485] The "information processing device" is a device that includes a computer system for analyzing acquired data and performing signal control, etc.

[0486] This invention is a system that aims to alleviate traffic congestion and reduce passenger stress by analyzing traffic conditions and passenger emotions and controlling traffic signals in real time. A specific embodiment of this system is described below.

[0487] 1. Data Collection and Preprocessing

[0488] Terminal

[0489] The device captures real-time video data from cameras installed on traffic lights and autonomous vehicles. The video data is preprocessed to analyze traffic volume and vehicle congestion. Preprocessing includes noise reduction and contrast adjustment. The camera also captures facial expressions of passengers in the vehicle and uses this data as emotion data.

[0490] 2. Data analysis and traffic volume evaluation

[0491] server

[0492] The server receives preprocessed video data and emotion data sent from the terminals. Using this data, it analyzes traffic volume and further analyzes passenger emotions using an emotion recognition engine. Traffic volume analysis uses an image recognition algorithm to detect and count the number of vehicles and their movements. Meanwhile, the emotion recognition engine measures passenger stress levels and irritation from their facial expressions.

[0493] 3. Integrated evaluation of traffic volume and user sentiment

[0494] server

[0495] The server comprehensively evaluates traffic volume and emotion data, and determines that traffic conditions are deteriorating or passenger stress levels are high, and prioritizes traffic signal adjustments.

[0496] 4. Execution of signal control algorithm

[0497] server

[0498] Based on the evaluation results, the server executes a traffic light control algorithm, which calculates the optimal signal duration for each intersection. For example, if traffic volume is high and passengers are feeling stressed, dynamic adjustments are made, such as extending the green light duration.

[0499] 5. Generation and transmission of signal control instructions

[0500] server

[0501] The server generates signal control instructions based on the calculated optimal lighting time and instantly sends them to each traffic light device. For example, an instruction to extend the green light at a certain intersection by 30 seconds is sent from the server to the terminal.

[0502] 6. Traffic light adjustment

[0503] Terminal

[0504] The terminal receives the signal control instruction sent from the server and automatically adjusts the lighting time of the signal device, for example, extending the green light lighting time by 30 seconds.

[0505] 7. Feedback Collection and Analysis

[0506] Terminal and Server

[0507] After adjusting the traffic lights, the device again collects traffic data and passenger sentiment data based on the new signal patterns and sends them to the server. The server analyzes this feedback data and evaluates the effectiveness of the signal control algorithm. Based on the analysis results, the algorithm is improved and used for future adjustments.

[0508] Specific examples

[0509] As a concrete example 1, consider alleviating congestion at intersection A. A device acquires video data and emotion data from a camera installed at intersection A and sends it to a server. The server analyzes the received data, generates instructions to extend the green light duration by 30 seconds as needed, and sends them to the device. The traffic light follows this instruction and actually extends the green light duration by 30 seconds. Feedback data is collected again, and the server analyzes it to improve the algorithm.

[0510] Prompt Sentence Examples

[0511] The following prompt sentences are input into the generative AI model to analyze passenger emotional data:

[0512] Recognize the facial expressions of passengers in your car to assess their stress levels.

[0513] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0514] Step 1:

[0515] Data collection

[0516] The device acquires video data in real time from cameras installed on traffic lights and autonomous vehicles.

[0517] Input: Raw video data from the camera.

[0518] Output: Captured video data.

[0519] Specifically, the device captures video from a designated camera at specific time intervals and stores or transmits it as a data stream.

[0520] Step 2:

[0521] Video data preprocessing

[0522] Pre-processing such as noise removal and contrast adjustment is performed on the video data acquired by the device.

[0523] Input: Acquired raw video data.

[0524] Output: Preprocessed video data.

[0525] Specifically, it uses an image processing library (e.g., OpenCV) to apply filters to reduce noise in the image and adjust the contrast and brightness appropriately.

[0526] Step 3:

[0527] Sending data

[0528] The terminal transmits the preprocessed video data to the server.

[0529] Input: Preprocessed video data.

[0530] Output: The data sent to the server.

[0531] Specifically, the device sends preprocessed data to the server in real time through a specific API endpoint.

[0532] Step 4:

[0533] Traffic volume assessment and emotion recognition

[0534] The server receives the preprocessed video data and performs traffic analysis and emotion recognition.

[0535] Input: Preprocessed video data.

[0536] Output: Traffic data and sentiment data.

[0537] Specifically, the server uses image recognition algorithms to detect and count the number and movement of vehicles, and an emotion recognition engine (e.g., facial expression analysis software) to analyze passengers' emotions and stress levels from their facial expressions in the video data.

[0538] Step 5:

[0539] Integrated evaluation

[0540] The server comprehensively evaluates the traffic volume data and emotion data.

[0541] Input: Traffic data and sentiment data.

[0542] Output: Overall evaluation result.

[0543] Specifically, the server uses the analysis results, predefined rules or generative AI models to evaluate the relationship between traffic conditions and stress levels, and determines the necessary adjustments.

[0544] Step 6:

[0545] Execution of signal control algorithms

[0546] The server executes the signal control algorithm based on the comprehensive evaluation result.

[0547] Input: Overall evaluation result.

[0548] Output: Signal control instructions.

[0549] Specifically, the server runs an algorithm to calculate the optimal signal duration and determine, for example, the green light extension time.

[0550] Step 7:

[0551] Sending signal control instructions

[0552] The server transmits the calculated signal control instructions to the terminal.

[0553] Input: Signal control instructions.

[0554] Output: Signal control instructions sent to the terminal.

[0555] As a specific operation, the server transmits instruction data including a command to adjust the signal timing to the terminal.

[0556] Step 8:

[0557] Traffic light adjustments

[0558] The terminal adjusts the signaling device based on the received signal control instructions.

[0559] Input: Signal control instructions.

[0560] Output: Traffic light operating status after adjustment.

[0561] Specifically, the terminal sends instructions to the traffic light control unit to change its timing.

[0562] Step 9:

[0563] Collecting and sending feedback

[0564] The terminal again collects traffic data and emotion data for the new signal pattern and transmits them to the server.

[0565] Input: Traffic data and emotion data with adjusted signal patterns.

[0566] Output: Feedback data sent to the server.

[0567] As a specific operation, the terminal collects data in the same manner as in the first step and transmits it to the server as feedback data.

[0568] Step 10:

[0569] Feedback analysis and algorithm improvement

[0570] The server analyzes the feedback data and evaluates the effectiveness of the signal control algorithm.

[0571] Input: Feedback data.

[0572] Output: Algorithm improvements.

[0573] Specifically, the server compares the collected data with the results of the algorithm and adjusts and improves the control algorithm as needed.

[0574] The above are the processing steps of the system for implementing the present invention.

[0575] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0576] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0577] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0578] [Second embodiment]

[0579] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0580] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0581] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0582] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0583] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0584] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0585] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0586] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0587] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0588] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0589] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0590] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0591] The present invention relates to a system for mitigating traffic congestion by utilizing cameras installed at traffic lights. Specific embodiments of the system are described below.

[0592] (Data collection and preprocessing)

[0593] Terminal

[0594] The terminal is responsible for acquiring video data in real time from cameras installed at traffic lights. The acquired video data undergoes pre-processing to analyze traffic volume and vehicle congestion. Specifically, the device performs noise removal and contrast adjustment on the video data.

[0595] (Data analysis and traffic volume evaluation)

[0596] server

[0597] The server analyzes pre-processed video data sent from the devices to assess traffic volume. The analysis is performed using image recognition algorithms to detect and count the number of vehicles and their movements. This data is used to assess the traffic situation at specific intersections and road sections in real time.

[0598] (Execution of signal control algorithm)

[0599] server

[0600] The server then executes a traffic light control algorithm based on the results of its assessment of traffic volume and congestion. This algorithm calculates the optimal signal lighting time for each intersection. For example, dynamic adjustments are made, such as extending the green light time at intersections where vehicles are congested.

[0601] (Generation and transmission of signal control instructions)

[0602] server

[0603] The server generates signal control instructions based on the calculated optimal lighting time. These instructions are instantly sent to each traffic light, and the lighting time of the traffic light is adjusted in real time. For example, an instruction to extend the green light at intersection A by 30 seconds is sent from the server to the terminal.

[0604] (Traffic signal adjustment)

[0605] Terminal

[0606] The terminal receives the signal control instruction sent from the server and automatically adjusts the lighting time of the traffic light, for example, by extending the green light time by 30 seconds.

[0607] (Feedback collection and analysis)

[0608] Terminal

[0609] After adjusting the traffic lights, the terminal again collects traffic data based on the new signal pattern and transmits it to the server.

[0610] server

[0611] The server analyzes this feedback data and continuously improves the signal control algorithm, allowing it to flexibly respond to changes in traffic conditions and optimize the entire system.

[0612] (Example)

[0613] Example 1: Reducing congestion at intersection A

[0614] 1. Data Collection

[0615] The terminal acquires video data from a camera installed at intersection A.

[0616] Preprocess the video data and send it to the server.

[0617] 2. Data Analysis

[0618] The server analyzes the received video data using an image recognition algorithm and counts the number of vehicles at intersection A.

[0619] Based on the analysis results, the traffic volume and congestion situation at intersection A are evaluated.

[0620] 3. Execution of signal control algorithm

[0621] The server executes a traffic light control algorithm to extend the green light duration at intersection A by 30 seconds.

[0622] 4. Transmission of signal control instructions

[0623] The server transmits the generated signal control instruction to the terminal.

[0624] 5. Traffic light adjustments

[0625] The device adjusts and controls the traffic light to extend the green light duration by 30 seconds.

[0626] 6. Feedback Collection and Analysis

[0627] The terminal again collects traffic data based on the new signal pattern and transmits it to the server.

[0628] The server analyzes the feedback data and improves the traffic light control algorithm.

[0629] In this way, traffic flow can be optimized and congestion can be alleviated by dynamically adjusting the illumination duration of traffic lights based on the analysis of video data acquired by the camera.

[0630] The processing flow will be explained below.

[0631] Step 1:

[0632] Terminal

[0633] The device acquires video data in real time from cameras installed at traffic lights. Specifically, it acquires video within the camera's field of view and extracts frames at regular intervals. This video data is used as basic data for analyzing traffic volume and vehicle congestion.

[0634] Step 2:

[0635] Terminal

[0636] The device performs preprocessing on the captured video data, including noise reduction, contrast adjustment, and resolution optimization, to improve analysis accuracy and reduce the risk of misidentification.

[0637] Step 3:

[0638] Terminal

[0639] The pre-processed video data is sent to the server, which converts the data packets into an appropriate format and transfers them to the server in real time over the network.

[0640] Step 4:

[0641] server

[0642] The server receives pre-processed video data sent from the device, ready for immediate analysis.

[0643] Step 5:

[0644] server

[0645] The server analyzes the received video data using an image recognition algorithm. Specifically, it counts the number of vehicles in each frame and measures their movement speed and stationary time. Based on the results of this analysis, it evaluates traffic volume and stationary conditions.

[0646] Step 6:

[0647] server

[0648] Based on the results of the traffic volume and congestion assessment, the system executes a traffic light control algorithm that calculates the optimal lighting times (red, green, and yellow) for each intersection. For example, this may include extending the green light time on roads with heavy traffic.

[0649] Step 7:

[0650] server

[0651] Based on the calculated optimal lighting time, specific traffic light control instructions are generated, and the server converts these instructions into data packets for transmission to the target traffic light.

[0652] Step 8:

[0653] server

[0654] The server transmits the signal control instructions, including the traffic light control parameters (lighting time, sequence, etc.), to the terminal via the network.

[0655] Step 9:

[0656] Terminal

[0657] The terminal receives the signal control instructions sent from the server. This is applied to the internal control system to adjust the lighting time of the traffic light. Specifically, it controls the time the green light stays on by 30 seconds, for example.

[0658] Step 10:

[0659] Terminal

[0660] The operation of the adjusted traffic lights is monitored and traffic data is collected again based on the new signal pattern, which is then sent as feedback to the server.

[0661] Step 11:

[0662] server

[0663] The server receives feedback data from the devices and evaluates the effectiveness of the signal control algorithm. Based on the analysis results, the algorithm is improved and used for future adjustments.

[0664] Step 12:

[0665] User

[0666] Users can check current traffic conditions and traffic light change information through a dedicated app or website, and user feedback will also be incorporated into the system for further optimization.

[0667] Example 1

[0668] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0669] Conventional traffic systems face the challenge of effectively alleviating traffic congestion in real time. Specifically, they lack the ability to properly grasp ongoing traffic conditions and dynamically adjust traffic light illumination times accordingly. As a result, traffic flow becomes unsmooth and congestion occurs.

[0670] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0671] In this invention, the server includes means for collecting video data acquired by a camera, means for pre-processing the video data acquired by the camera, means for analyzing the collected video data to evaluate traffic volume, means for executing a signal control algorithm based on the evaluation results, means for generating and transmitting signal control instructions to the traffic lights, means for adjusting the illumination duration of the traffic lights based on the signal control instructions, and means for collecting and analyzing feedback data. This makes it possible to monitor traffic conditions in real time and perform appropriate traffic light control. Furthermore, the collected feedback data can be used to continuously improve the signal control algorithm, thereby effectively alleviating traffic congestion.

[0672] "Video data acquired by a camera" refers to video information captured in real time by a camera installed at an intersection or road section.

[0673] "Preprocessing" refers to a data cleaning process, such as noise removal and contrast adjustment, that is carried out before analyzing the video data acquired by the camera.

[0674] "Analysis" is a method of evaluating traffic volume and vehicle congestion based on preprocessed video data, and is carried out using image recognition algorithms.

[0675] "Evaluating traffic volume" means detecting the number and movement of vehicles at a particular intersection or road section from the analyzed data and determining the traffic situation.

[0676] A "signal control algorithm" refers to the calculation procedures and rules for calculating the optimal lighting time of a traffic light based on the results of traffic volume evaluation.

[0677] A "signal control instruction" is a command that indicates the lighting time of a specific traffic light, generated by a signal control algorithm.

[0678] "Adjusting the lighting time of a traffic light" means changing the display time of a green light, red light, etc. of a traffic light based on a signal control instruction.

[0679] "Feedback data" refers to new traffic data that is re-collected after adjusting traffic light durations and is used to continuously improve the algorithm.

[0680] A "traffic light" is a display device installed at an intersection or road section to control the flow of traffic.

[0681] MODE FOR CARRYING OUT THE INVENTION

[0682] The present invention is a system that uses cameras installed at traffic lights to alleviate traffic congestion. A specific embodiment of this system is described below.

[0683] Data collection and preprocessing

[0684] Terminal

[0685] The device acquires video data in real time from cameras installed at traffic lights. The hardware used includes commercially available surveillance cameras and a small computer for data collection. This video data undergoes preprocessing before analyzing traffic volume and vehicle congestion. Specifically, libraries such as OpenCV are used to remove noise and adjust contrast.

[0686] Data analysis and traffic volume evaluation

[0687] server

[0688] The server receives and analyzes the preprocessed video data sent from the device. This analysis uses image recognition algorithms such as YOLO (You Only Look Once) to detect and count vehicles in the video in real time. Based on the analysis results, the server evaluates the traffic volume and congestion status at specific intersections and road sections. The evaluation results are temporarily stored in a database.

[0689] Execution of signal control algorithms

[0690] server

[0691] The server then executes a traffic light control algorithm based on the results of its assessment of traffic volume and congestion. This algorithm calculates the optimal signal lighting time for each intersection. For example, at intersections where traffic volume has increased and congestion has occurred, dynamic adjustments are made, such as extending the green light time by 30 seconds.

[0692] Generation and transmission of signal control instructions

[0693] server

[0694] The server generates traffic light control instructions based on the calculated optimal lighting time. These instructions are instantly sent to each traffic light. Communication protocols such as MQTT are used to send specific instructions, such as "Extend the green light at intersection A by 30 seconds," to the terminal in JSON format.

[0695] Traffic light adjustments

[0696] Terminal

[0697] The terminal receives the signal control instructions sent from the server and automatically adjusts the traffic lights based on the instructions. A PLC (Programmable Logic Controller) is used to control the traffic lights, and the green light duration is extended as specified.

[0698] Collecting and analyzing feedback

[0699] Terminal

[0700] After adjusting the traffic lights, the device collects traffic data again based on the new signal patterns and sends it to the server. This data, also in the form of video data, is again pre-processed and analyzed.

[0701] server

[0702] The server analyzes this feedback data and adjusts parameters to continuously improve the signal control algorithm, allowing it to flexibly respond to changing traffic conditions.

[0703] Specific examples

[0704] As a specific example, the case of easing traffic congestion at intersection A will be shown below.

[0705] 1. Data Collection

[0706] The terminal acquires video data from a camera installed at intersection A, preprocesses it, and sends it to the server.

[0707] 2. Data Analysis

[0708] The server analyzes the received video data using an image recognition algorithm and counts the number of vehicles at intersection A.

[0709] Based on the analysis results, the traffic volume and congestion situation at intersection A are evaluated.

[0710] 3. Execution of signal control algorithm

[0711] The server executes a traffic light control algorithm to extend the green light duration at intersection A by 30 seconds.

[0712] 4. Transmission of signal control instructions

[0713] The server transmits the generated signal control instruction to the terminal.

[0714] 5. Traffic light adjustments

[0715] The device adjusts and controls the traffic light to extend the green light duration by 30 seconds.

[0716] 6. Feedback Collection and Analysis

[0717] The terminal again collects traffic data based on the new signal pattern and transmits it to the server.

[0718] The server analyzes the feedback data and improves the traffic light control algorithm.

[0719] Prompt Sentence Examples

[0720] "How will this system alleviate traffic congestion? What are the specific steps and technologies used?"

[0721] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0722] Step 1:

[0723] Data collection

[0724] Terminal

[0725] The terminal acquires video data in real time from cameras installed at traffic lights. Specifically, commercially available surveillance cameras capture video at a set frame rate. The input data is the captured raw video data. This data is preprocessed in the next step.

[0726] Step 2:

[0727] Video data preprocessing

[0728] Terminal

[0729] The device preprocesses the acquired video data. Specifically, it uses the OpenCV library to remove noise and adjust the contrast of the video data. The input data is raw video data. The output data is preprocessed video data with noise removed and contrast adjusted. This enables subsequent image analysis to be performed with high accuracy.

[0730] Step 3:

[0731] Data analysis

[0732] server

[0733] The server receives preprocessed video data sent from the device. The server analyzes this data using image recognition algorithms such as YOLO. The input data is the preprocessed video data, and the output data is the number and location of detected vehicles. Based on the analyzed data, traffic volume and congestion at specific intersections and road sections are evaluated.

[0734] Step 4:

[0735] Traffic volume evaluation

[0736] server

[0737] The server evaluates traffic volume based on the analysis results. Specifically, it calculates the actual traffic volume and congestion status at specific intersections and road sections based on the number of vehicles and location information obtained from the analysis. The input data is the number of vehicles and location information from the analysis results, and the output data is the evaluated traffic volume and congestion status. This evaluation result is temporarily stored in a database.

[0738] Step 5:

[0739] Execution of signal control algorithms

[0740] server

[0741] The server executes a signal control algorithm based on the evaluation results of traffic volume and congestion status. The input data is the traffic volume and congestion status of the evaluation results, and the output data is the optimal signal lighting time for each intersection. Specifically, at intersections where vehicles are congested, dynamic adjustments are made, such as extending the green light time.

[0742] Step 6:

[0743] Generation and transmission of signal control instructions

[0744] server

[0745] The server generates traffic light control instructions based on the calculated optimal lighting time. These instructions are instantly sent to each traffic light using a communication protocol such as MQTT. The input data is the optimized lighting time, and the output data is the specific control instructions for the traffic light. For example, this could include an instruction such as "Extend the green light at intersection A by 30 seconds."

[0746] Step 7:

[0747] Traffic light adjustments

[0748] Terminal

[0749] The terminal receives signal control instructions sent from the server. Based on the received instructions, the terminal controls the traffic light. The input data is the signal control instructions, and the output operation is to adjust the traffic light's lighting time. Specifically, the green light's lighting time is extended as specified. A PLC (Programmable Logic Controller) is used to control the traffic light.

[0750] Step 8:

[0751] Feedback data collection

[0752] Terminal

[0753] After the traffic light is adjusted, the terminal collects traffic data again according to the new signal pattern. The input data is the video data under the new signal pattern, and the output data is the preprocessed video data.

[0754] Step 9:

[0755] Analysis of feedback data and algorithm improvement

[0756] server

[0757] The server receives and analyzes feedback data sent from the terminal. The input data is preprocessed feedback video data, and the output data is the analysis results of the number of vehicles and their location information. Based on these results, the signal control algorithm is continuously improved to achieve adaptive signal control.

[0758] (Application example 1)

[0759] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0760] Conventional traffic systems have difficulty in controlling signals in a fast and flexible manner to respond to fluctuations in traffic volume, and congestion has not been sufficiently alleviated. Furthermore, conventional navigation systems for autonomous vehicles are unable to reflect real-time traffic information and may not be able to provide optimal driving routes.

[0761] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0762] In this invention, the server includes means for collecting video data acquired by cameras, means for analyzing the collected video data to evaluate traffic volume, means for dynamically adjusting the illumination time of traffic signals based on the evaluation results, and means for providing traffic information in real time and optimizing vehicle routes, thereby enabling dynamic adjustment of traffic volume and optimal navigation.

[0763] A "camera" is a device installed at a traffic light that captures images of road traffic conditions in real time.

[0764] "Video data" refers to digital video recordings of traffic conditions captured by a camera.

[0765] "Collection means" is a function that collects video data acquired by a camera in real time into a database or server.

[0766] The "analysis means" is a function that uses collected video data to evaluate traffic volume and identify the number of vehicles and congestion conditions.

[0767] "Evaluation results" refer to data indicating traffic volume and congestion conditions obtained by the analysis means.

[0768] The "signal lighting time adjustment means" is a function for dynamically changing the lighting time of the green or red signal of a traffic light based on the evaluation result.

[0769] "Signal control instructions" are control information for instructing the lighting time and operating state of a traffic light.

[0770] A "traffic light" is an optical signal device installed at road intersections and other locations to control traffic flow.

[0771] "Real-time traffic information provision means" is a function that provides the current optimal driving route based on traffic data.

[0772] The "driving route optimization means" is a function that navigates the vehicle along the optimal route based on real-time traffic information.

[0773] A system for carrying out the present invention comprises a camera, a server, and a terminal. Specific embodiments will be described in detail below.

[0774] Terminal

[0775] First, cameras installed at traffic lights capture real-time video data of road traffic conditions. The captured video data undergoes preprocessing such as noise removal and contrast adjustment on the device. After preprocessing, the data is sent to the server.

[0776] server

[0777] The server uses an image recognition algorithm to evaluate traffic volume based on preprocessed video data sent from the device. Specifically, it counts the number of vehicles and detects their movements. Based on the analysis results, it evaluates traffic volume and congestion conditions.

[0778] By running the signal control algorithm on the server, the optimal signal lighting time for each intersection is calculated. For example, dynamic adjustments are made, such as extending the green light time at intersections where congestion is occurring. The calculated optimal signal control instructions are sent to each traffic light in real time.

[0779] The server also provides real-time navigation information based on traffic information, allowing it to present optimal driving routes to autonomous vehicles.

[0780] Autonomous vehicles

[0781] The autonomous vehicle receives real-time traffic information sent from the server and optimizes its route, for example by changing its route to coincide with the time when the green light remains green at a particular intersection for a long period of time.

[0782] Specific examples

[0783] Video data obtained by the camera at intersection A is preprocessed on the terminal and sent to the server. The server uses an analysis algorithm to count the number of vehicles and evaluate traffic volume. If congestion occurs, a signal control algorithm is executed to generate signal control instructions that dynamically extend the green light duration and send them to the traffic lights. Real-time traffic information is also sent to the navigation system, which displays the optimal driving route.

[0784] Prompt Sentence Examples

[0785] "I want to build an application that analyzes video data from a traffic light camera at intersection A and suggests the optimal driving route based on traffic congestion. Please tell me the code to evaluate the current traffic volume, obtain signal control information based on that data, and calculate the optimal route."

[0786] In this way, traffic lights can be dynamically adjusted based on the analysis of video data acquired by cameras, providing autonomous vehicles with optimal routes to optimize traffic flow and alleviate congestion.

[0787] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0788] Step 1: Acquire video data with the camera

[0789] Cameras installed at traffic lights capture traffic conditions in real time. The video data obtained from the cameras is digital footage that captures the movement of vehicles on the road. Specifically, the video captured by the cameras is sent to a terminal in streaming format. The input of this step is the camera image itself, and the output is the unprocessed video data.

[0790] Step 2: Preprocess the video data

[0791] The device receives video data captured by the camera and performs preprocessing such as noise reduction and contrast adjustment. Specifically, the video data is converted to grayscale using a library such as OpenCV, and noise is removed by applying a Gaussian filter. The input of this step is the raw video data, and the output is the preprocessed video data.

[0792] Step 3: Send the preprocessed data to the server

[0793] The terminal sends the preprocessed video data to the server. In this step, data is transferred to the server using a communication protocol such as an HTTP request. The input is the preprocessed video data, and the output is the data reception on the server.

[0794] Step 4: Analyze video data to assess traffic volume

[0795] The server analyzes the received preprocessed video data and evaluates traffic volume. This process uses an image recognition algorithm to count the number of vehicles and identify their movements. For example, it uses YOLO or OpenCV vehicle detection models. The input is the preprocessed video data, and the output is an evaluation result showing the number of vehicles and congestion status.

[0796] Step 5: Run the signal control algorithm

[0797] The server executes a signal control algorithm based on the evaluation results. Specifically, it dynamically adjusts signals, such as extending the green light duration at intersections with congestion. The input is evaluation data based on vehicle detection results, and the output is optimal signal control instructions.

[0798] Step 6: Send signal control instructions to the traffic light

[0799] The signal control instructions generated by the server are sent to the terminal to control the traffic lights. Here, signal control information is sent to the traffic lights using a communication protocol. The input is the signal control instructions, and the output is the transfer of instructions to the traffic lights.

[0800] Step 7: Providing real-time traffic information

[0801] The server provides real-time traffic information to autonomous vehicles based on traffic volume evaluation results and signal control information. Specifically, it sends route optimization information via API. The input is the latest traffic data, and the output is route optimization information.

[0802] Step 8: Optimize the autonomous vehicle's route

[0803] An autonomous vehicle optimizes its driving route based on traffic information received from a server. Here, the vehicle's navigation system analyzes the information from the server and presents the optimal route. The input is the optimized driving route information, and the output is the optimal route the vehicle should follow.

[0804] By going through the above processing steps, it is possible to optimize traffic flow and alleviate congestion.

[0805] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0806] This invention relates to a system that alleviates traffic congestion by utilizing cameras installed at traffic lights and combining them with an emotion engine that recognizes the emotions of users. Specific embodiments of this system are described below.

[0807] (Data collection and preprocessing)

[0808] Terminal

[0809] The device is responsible for acquiring video data in real time from cameras installed at traffic lights. The acquired video data undergoes preprocessing to analyze traffic volume and vehicle congestion. Specifically, noise removal and contrast adjustment are performed on the video data. The camera also captures the facial expressions of the vehicle's users, which are used as emotional data.

[0810] (Data analysis and traffic volume evaluation)

[0811] server

[0812] The server receives preprocessed video data and emotion data sent from the device. Using the received data, it analyzes traffic volume and then analyzes the user's emotions using an emotion engine. Traffic volume analysis uses an image recognition algorithm to detect and count the number of vehicles and their movements. Meanwhile, the emotion engine measures the user's stress level and irritation from their facial expressions.

[0813] (traffic volume and user sentiment evaluation)

[0814] server

[0815] The server comprehensively evaluates traffic volume and user emotion data, and determines that traffic conditions are deteriorating or the user's stress level is high, and prioritizes traffic light adjustment.

[0816] (Execution of signal control algorithm)

[0817] server

[0818] The server then executes a traffic light control algorithm based on the evaluation results. The algorithm calculates the optimal traffic light duration for each intersection. For example, if traffic volume is heavy and the user is feeling stressed, the algorithm dynamically adjusts the duration of the green light, such as extending it.

[0819] (Generation and transmission of signal control instructions)

[0820] server

[0821] The server generates signal control instructions based on the calculated optimal lighting time. These instructions are instantly sent to each traffic light, and the lighting time of the traffic light is adjusted in real time. For example, an instruction to extend the green light at intersection A by 30 seconds is sent from the server to the terminal.

[0822] (Traffic signal adjustment)

[0823] Terminal

[0824] The device receives the signal control instructions sent from the server and automatically adjusts the traffic light duration, for example, extending the green light duration by 30 seconds.

[0825] (Feedback collection and analysis)

[0826] Terminal

[0827] After the traffic light is adjusted, the terminal collects traffic data and user emotion data according to the new traffic light pattern again and transmits them to the server.

[0828] server

[0829] The server analyzes this feedback data and evaluates the effectiveness of the signal control algorithm. Based on the analysis results, the algorithm is improved and used for future adjustments.

[0830] (Example)

[0831] Example 1: Reducing congestion at intersection A

[0832] 1. Data Collection

[0833] The terminal acquires video data and emotion data from a camera installed at intersection A.

[0834] Preprocess the video data and send it to the server.

[0835] 2. Data Analysis

[0836] The server analyzes the received video data using an image recognition algorithm and counts the number of vehicles at intersection A.

[0837] An emotion engine is used to analyze the user's emotion data and assess their stress level.

[0838] 3. Evaluation of traffic volume and emotion data

[0839] The server comprehensively evaluates the traffic volume at intersection A and the user's emotional data to determine the traffic situation and the user's stress level.

[0840] 4. Execution of signal control algorithm

[0841] The server executes a traffic light control algorithm to extend the green light duration at intersection A by 30 seconds.

[0842] 5. Transmission of signal control instructions

[0843] The server transmits the generated signal control instruction to the terminal.

[0844] 6. Traffic light adjustment

[0845] The device adjusts and controls the traffic light to extend the green light duration by 30 seconds.

[0846] 7. Feedback Collection and Analysis

[0847] The terminal again collects traffic data and user emotion data based on the new signal pattern and transmits them to the server.

[0848] The server analyzes the feedback data and improves the traffic light control algorithm.

[0849] In this way, traffic lights can be dynamically adjusted based on the video data acquired by the camera and the user's emotional data obtained from the emotion engine, thereby optimizing traffic flow, easing congestion, and reducing user stress.

[0850] The processing flow will be explained below.

[0851] Step 1:

[0852] Terminal

[0853] The device acquires real-time video data from cameras installed at traffic lights. The cameras are high-resolution video cameras that capture the entire intersection and extract frames at regular intervals, providing data for monitoring temporary congestion and fluctuations in traffic volume.

[0854] Step 2:

[0855] Terminal

[0856] Preprocessing is performed on the video data captured by the camera. Specifically, processes such as noise removal, contrast adjustment, and resolution optimization are performed. This improves the quality of the video data and increases the accuracy of analysis. In addition, if the user is in the vehicle, their facial expressions are captured and extracted as emotion data.

[0857] Step 3:

[0858] Terminal

[0859] The preprocessed video data and emotion data are sent to the server. The data is set to be sent in real time via the network, allowing the server to constantly monitor the latest traffic conditions and the user's emotional state.

[0860] Step 4:

[0861] server

[0862] The server receives the preprocessed video data and emotion data sent from the device. It then applies an image recognition algorithm to analyze the received video data. Specifically, the analysis counts the number of vehicles in each frame and measures their movement and dwell time.

[0863] Step 5:

[0864] server

[0865] The server uses an emotion engine to analyze the user's emotional data. Specifically, it uses facial expression recognition technology to evaluate the user's stress level and irritation. This evaluation result is combined with traffic volume data to help understand the overall traffic situation.

[0866] Step 6:

[0867] server

[0868] The system evaluates traffic volume and user emotional data in an integrated manner. For example, when traffic volume is heavy and vehicles are backed up, and when a high stress level is detected from the user's emotional data, it determines that it is necessary to raise the priority of traffic signal control.

[0869] Step 7:

[0870] server

[0871] Based on the evaluation results, a traffic light control algorithm is executed. The algorithm calculates the optimal traffic light duration for each intersection. Dynamic adjustments are made, such as extending the green light duration at intersection A by 30 seconds.

[0872] Step 8:

[0873] server

[0874] It generates signal control instructions, including the specific time to extend the green light and the priority of roads, and converts these instructions into data packets for transmission to the traffic light controller.

[0875] Step 9:

[0876] server

[0877] A signal control instruction is sent to the terminal. The instruction is sent immediately via the network, and the server receives a response confirming receipt, thereby confirming that the instruction was transmitted correctly.

[0878] Step 10:

[0879] Terminal

[0880] The terminal receives the signal control instructions sent from the server. It applies these to the internal control system and sets the lighting time and sequence of the traffic lights. Specifically, it controls the signal by extending the green light time by 30 seconds, for example.

[0881] Step 11:

[0882] Terminal

[0883] The operation of the traffic lights after adjustment is monitored. Traffic data and user emotion data based on the new traffic light patterns are collected again and sent to the server.

[0884] Step 12:

[0885] server

[0886] The server receives feedback data from the devices and evaluates the effectiveness of the signal control algorithm. Based on the analysis results, the algorithm is improved and used for future adjustments.

[0887] Step 13:

[0888] User

[0889] Users can check current traffic conditions and traffic light change information through a dedicated app or website, and receive advice and notifications based on their stress and irritation levels to help them drive with peace of mind.

[0890] Example 2

[0891] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0892] Conventional traffic signal control systems are unable to respond quickly to fluctuations in traffic volume, making it difficult to effectively alleviate traffic congestion. Furthermore, because they do not take into account the user's stress or emotional state, drivers experience increased stress and the risk of traffic accidents increases as they spend longer at intersections. The present invention aims to solve these problems by analyzing traffic volume and the user's emotional state in real time and dynamically adjusting signal control.

[0893] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0894] In this invention, the server includes means for collecting video data captured by cameras installed at traffic lights in real time, means for performing noise reduction and contrast adjustment on the collected video data, means for capturing a user's facial expressions to collect emotion data, means for analyzing the preprocessed video data and emotion data to evaluate traffic volume and the user's emotional state, means for executing a signal control algorithm based on the evaluation results, means for generating and transmitting signal control instructions to the traffic lights, means for re-collecting new traffic data and emotion data based on the adjusted signal pattern, and means for analyzing the feedback data to improve the signal control algorithm, thereby enabling fast and effective signal control based on fluctuations in traffic volume and the user's emotional state.

[0895] A "traffic light" is an optical signaling device installed to control the flow of road traffic.

[0896] A "camera" is a photographing device for acquiring video data.

[0897] "Video Data" refers to visual information captured by a camera.

[0898] "Real-time" means that data acquisition and processing occur almost simultaneously.

[0899] "Noise reduction" is the process of removing unnecessary information and errors from collected video data.

[0900] "Contrast adjustment" is a process for appropriately adjusting the contrast between light and dark in the video data.

[0901] "Facial expressions" express emotions based on the shape and movement of the user's face.

[0902] "Emotion data" is information that represents the user's emotional state.

[0903] "Analysis" is the process of examining data in detail to clarify its content and meaning.

[0904] "Traffic volume" refers to the number and flow of vehicles in a particular time unit.

[0905] "Evaluation" is the act of judging a situation or condition based on data.

[0906] A "traffic light control algorithm" is a calculation method for determining the lighting time and control pattern of traffic lights.

[0907] A "signal control instruction" is a specific command for controlling the operation of a traffic light.

[0908] "Feedback data" is new data collected after traffic light adjustments.

[0909] "User" refers to the driver or passenger of a vehicle that uses this system.

[0910] This invention relates to a system that utilizes cameras installed at traffic lights and combines them with an emotion engine that recognizes the emotions of users to alleviate traffic congestion. Specific embodiments of this system are described below.

[0911] (System Configuration)

[0912] Hardware

[0913] Cameras installed at traffic lights:

[0914] The camera captures the traffic conditions at the intersection in real time, collecting video data, and also captures the user's facial expressions to obtain emotional data.

[0915] Device:

[0916] The terminal is installed at the traffic light, receives the video data from the camera, performs noise reduction and contrast adjustment, and sends the pre-processed data to the server.

[0917] server:

[0918] The server receives and analyzes the video and emotion data sent from the device. Based on the results of the analysis, it executes a traffic light control algorithm and generates instructions for the traffic light.

[0919] (software)

[0920] Image Recognition Algorithm

[0921] The server uses image recognition algorithms to analyze the number and movement of vehicles from pre-processed video data, allowing for accurate assessment of traffic volume.

[0922] Emotion Engine

[0923] The server uses an emotion engine to analyze the user's facial expression data and assess their stress level and irritation, thereby taking into account not only the traffic situation but also the user's emotional state.

[0924] Signal Control Algorithm

[0925] The server runs a traffic light control algorithm based on the results of traffic volume analysis and emotion analysis, which calculates the optimal signal lighting time for each intersection and dynamically adjusts the traffic lights.

[0926] Feedback Mechanism

[0927] After adjusting the traffic lights, the device will again collect traffic and emotion data based on the new signal patterns and send it to the server, which will analyze this feedback data, evaluate the effectiveness of the signal control algorithm, and make improvements.

[0928] (Example)

[0929] Example 1: Reducing congestion at intersection A

[0930] 1. Data Collection:

[0931] The terminal acquires video data and emotion data from a camera installed at intersection A. The acquired data is preprocessed and sent to the server.

[0932] 2. Data Analysis:

[0933] The server analyzes the received video data using an image recognition algorithm and counts the number of vehicles at intersection A. It uses an emotion engine to analyze the user's emotion data and evaluate their stress level.

[0934] 3. Traffic and Sentiment Data Evaluation:

[0935] The server comprehensively evaluates the traffic volume at intersection A and the user's emotional data to determine the traffic situation and the user's stress level.

[0936] 4. Execution of signal control algorithm:

[0937] The server executes a traffic light control algorithm to extend the green light duration at intersection A by 30 seconds.

[0938] 5. Sending signal control instructions:

[0939] The server transmits the generated signal control instruction to the terminal.

[0940] 6. Traffic light adjustment:

[0941] The device adjusts and controls the traffic light to extend the green light duration by 30 seconds.

[0942] 7. Feedback Collection and Analysis:

[0943] The terminal again collects traffic data and user emotion data based on the new signal pattern and transmits them to the server, which analyzes the feedback data and improves the signal control algorithm.

[0944] Prompt Sentence Examples

[0945] "Please explain a system that uses cameras and an emotion engine to collect and analyze traffic volume and user emotion data, and dynamically adjusts traffic light durations to alleviate congestion at intersections."

[0946] In this way, traffic lights can be dynamically adjusted based on the video data acquired by the camera and the user's emotional data obtained from the emotion engine, thereby optimizing traffic flow, easing congestion, and reducing user stress.

[0947] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0948] Step 1:

[0949] Data collection

[0950] The terminal acquires video data in real time from cameras installed at traffic lights.

[0951] Input: Video data from a camera installed at a traffic light.

[0952] Data processing and calculation: Capture of video data.

[0953] Output: Captured video data.

[0954] Step 2:

[0955] Video data preprocessing

[0956] The device performs noise removal and contrast adjustment on the acquired video data.

[0957] Input: Captured video data.

[0958] Data processing and calculation: noise removal and contrast adjustment.

[0959] Output: Pre-processed video data. Specific operations include applying noise filters and histogram equalization.

[0960] Step 3:

[0961] Collecting Emotional Data

[0962] The device's camera captures the facial expressions of the user riding in the vehicle and uses them as emotional data.

[0963] Input: Video data containing the user's facial expressions.

[0964] Data processing and calculation: Image analysis for facial expression recognition.

[0965] Output: User's emotional data. Specifically, the stress level and irritation level are measured from the user's facial expressions.

[0966] Step 4:

[0967] Video data analysis

[0968] The server receives the pre-processed video data sent from the device and analyzes it using image recognition algorithms.

[0969] Input: Preprocessed video data.

[0970] Data processing and calculation: The number and movement of vehicles are detected and counted using an image recognition algorithm.

[0971] Output: Parsed traffic data. Specific actions include vehicle identification and counting.

[0972] Step 5:

[0973] Emotional Data Analysis

[0974] The server uses an emotion engine to analyze the user's facial expression data and assess their stress level and irritability.

[0975] Input: User emotion data.

[0976] Data processing and calculation: Emotion analysis using an emotion engine.

[0977] Output: Analyzed emotional state data. Specifically, facial expression recognition technology is used to detect the user's micro-expressions and estimate their emotional state.

[0978] Step 6:

[0979] Integrated evaluation of traffic volume and emotion data

[0980] The server performs an integrated evaluation based on the analyzed traffic volume data and emotion data.

[0981] Input: Traffic data and emotional state data.

[0982] Data processing and calculation: Integrated evaluation of data.

[0983] Output: Integrated evaluation results. Specific actions include determining whether traffic conditions are worsening or the user's stress level is high.

[0984] Step 7:

[0985] Execution of signal control algorithms

[0986] The server executes the signal control algorithm based on the results of the integrated evaluation.

[0987] Input: Integrated evaluation results.

[0988] Data processing and calculation: Application of signal control algorithms.

[0989] Output: Optimal signal control instructions. Specifically, the optimal green and red light durations at each intersection are calculated.

[0990] Step 8:

[0991] Generation of signal control instructions

[0992] The server generates signal control instructions based on the results of the signal control algorithm.

[0993] Input: The result of the signal control algorithm.

[0994] Data processing and calculation: Generation of signal control instructions.

[0995] Output: Specific signal control instructions. Specifically, instructions such as "Extend the green light at intersection A by 30 seconds" are generated.

[0996] Step 9:

[0997] Sending signal control instructions

[0998] The server transmits the generated signal control instruction to the terminal.

[0999] Input: Signal control instructions.

[1000] Data processing and calculation: Sending signal control instructions.

[1001] Output: Signal control instructions sent to the terminal.

[1002] Step 10:

[1003] Traffic light adjustments

[1004] The terminal adjusts the lighting time of the traffic light based on the signal control instruction received from the server.

[1005] Input: Signal control instruction from the server.

[1006] Data processing and calculation: Traffic light control.

[1007] Output: Adjusted traffic lights. Specifically, the green light duration is extended by 30 seconds.

[1008] Step 11:

[1009] Collecting new data

[1010] After the traffic light is adjusted, the terminal collects traffic data and emotion data again according to the new traffic light pattern.

[1011] Input: Adjusted traffic data and sentiment data.

[1012] Data processing and calculation: Data collection.

[1013] Output: Recollected traffic data and emotion data.

[1014] Step 12:

[1015] Analysis of feedback data

[1016] The server again analyzes the received feedback data and evaluates the effectiveness of the signal control algorithm.

[1017] Input: Recollected traffic data and emotion data.

[1018] Data processing and calculation: Analysis of feedback data.

[1019] Output: Analysis results, specifically evaluating the effectiveness of signal conditioning and modifying algorithm parameters if necessary.

[1020] Through the above steps, traffic flow can be optimized, congestion can be alleviated, and user stress can be reduced by dynamically adjusting the illumination times of traffic lights based on the video data acquired by the camera and the user's emotional data obtained from the emotion engine.

[1021] (Application example 2)

[1022] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1023] Conventional traffic signal control systems rely solely on traffic volume and do not take into account passenger emotions or stress levels, which limits their effectiveness in easing traffic congestion and reducing passenger stress. This often leads to poor traffic efficiency and lower passenger satisfaction. Furthermore, conventional systems have fixed signal control, making real-time adjustments difficult and preventing flexible responses to changing situations.

[1024] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1025] In this invention, the server includes means for collecting video data acquired by a camera, means for analyzing the collected video data to evaluate traffic volume, and emotion recognition means for evaluating passenger emotions, thereby enabling traffic signal control based on traffic conditions and passenger emotion data in real time.

[1026] "Camera" refers to an image capture device installed on traffic lights and vehicles to capture traffic and passenger expressions.

[1027] "Video data" refers to real-time video information captured by a camera.

[1028] "Traffic volume" is a number or data that indicates the number or flow of vehicles on a road in a particular area or period of time.

[1029] "Signal duration" refers to the duration that each color of light remains on at a traffic light, and is an important parameter for traffic control.

[1030] "Signal control instructions" refer to commands to set specific lighting patterns and times for traffic lights.

[1031] "Emotion recognition means" is a technology that analyzes passengers' facial expressions from video data acquired by a camera and evaluates their emotions and stress levels.

[1032] "Emotion Data" refers to data indicative of passenger emotions and stress levels analyzed by the emotion recognition means.

[1033] The "evaluation result" is information that indicates a comprehensive assessment of the situation obtained from the analysis of traffic volume and emotion data.

[1034] A "traffic light control algorithm" is a method or program for calculating optimal traffic light lighting patterns based on traffic volume and emotion data.

[1035] The "information processing device" is a device that includes a computer system for analyzing acquired data and performing signal control, etc.

[1036] This invention is a system that aims to alleviate traffic congestion and reduce passenger stress by analyzing traffic conditions and passenger emotions and controlling traffic signals in real time. A specific embodiment of this system is described below.

[1037] 1. Data Collection and Preprocessing

[1038] Terminal

[1039] The device captures real-time video data from cameras installed on traffic lights and autonomous vehicles. The video data is preprocessed to analyze traffic volume and vehicle congestion. Preprocessing includes noise reduction and contrast adjustment. The camera also captures facial expressions of passengers in the vehicle and uses this data as emotion data.

[1040] 2. Data analysis and traffic volume evaluation

[1041] server

[1042] The server receives preprocessed video data and emotion data sent from the terminals. Using this data, it analyzes traffic volume and further analyzes passenger emotions using an emotion recognition engine. Traffic volume analysis uses an image recognition algorithm to detect and count the number of vehicles and their movements. Meanwhile, the emotion recognition engine measures passenger stress levels and irritation from their facial expressions.

[1043] 3. Integrated evaluation of traffic volume and user sentiment

[1044] server

[1045] The server comprehensively evaluates traffic volume and emotion data, and determines that traffic conditions are deteriorating or passenger stress levels are high, and prioritizes traffic signal adjustments.

[1046] 4. Execution of signal control algorithm

[1047] server

[1048] Based on the evaluation results, the server executes a traffic light control algorithm, which calculates the optimal signal duration for each intersection. For example, if traffic volume is high and passengers are feeling stressed, dynamic adjustments are made, such as extending the green light duration.

[1049] 5. Generation and transmission of signal control instructions

[1050] server

[1051] The server generates signal control instructions based on the calculated optimal lighting time and instantly sends them to each traffic light device. For example, an instruction to extend the green light at a certain intersection by 30 seconds is sent from the server to the terminal.

[1052] 6. Traffic light adjustment

[1053] Terminal

[1054] The terminal receives the signal control instruction sent from the server and automatically adjusts the lighting time of the signal device, for example, extending the green light lighting time by 30 seconds.

[1055] 7. Feedback Collection and Analysis

[1056] Terminal and Server

[1057] After adjusting the traffic lights, the device again collects traffic data and passenger sentiment data based on the new signal patterns and sends them to the server. The server analyzes this feedback data and evaluates the effectiveness of the signal control algorithm. Based on the analysis results, the algorithm is improved and used for future adjustments.

[1058] Specific examples

[1059] As a concrete example 1, consider alleviating congestion at intersection A. A device acquires video data and emotion data from a camera installed at intersection A and sends it to a server. The server analyzes the received data, generates instructions to extend the green light duration by 30 seconds as needed, and sends them to the device. The traffic light follows this instruction and actually extends the green light duration by 30 seconds. Feedback data is collected again, and the server analyzes it to improve the algorithm.

[1060] Prompt Sentence Examples

[1061] The following prompt sentences are input into the generative AI model to analyze passenger emotional data:

[1062] Recognize the facial expressions of passengers in your car to assess their stress levels.

[1063] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1064] Step 1:

[1065] Data collection

[1066] The device acquires video data in real time from cameras installed on traffic lights and autonomous vehicles.

[1067] Input: Raw video data from the camera.

[1068] Output: Captured video data.

[1069] Specifically, the device captures video from a designated camera at specific time intervals and stores or transmits it as a data stream.

[1070] Step 2:

[1071] Video data preprocessing

[1072] Pre-processing such as noise removal and contrast adjustment is performed on the video data acquired by the device.

[1073] Input: Acquired raw video data.

[1074] Output: Preprocessed video data.

[1075] Specifically, it uses an image processing library (e.g., OpenCV) to apply filters to reduce noise in the image and adjust the contrast and brightness appropriately.

[1076] Step 3:

[1077] Sending data

[1078] The terminal transmits the preprocessed video data to the server.

[1079] Input: Preprocessed video data.

[1080] Output: The data sent to the server.

[1081] Specifically, the device sends preprocessed data to the server in real time through a specific API endpoint.

[1082] Step 4:

[1083] Traffic volume assessment and emotion recognition

[1084] The server receives the preprocessed video data and performs traffic analysis and emotion recognition.

[1085] Input: Preprocessed video data.

[1086] Output: Traffic data and sentiment data.

[1087] Specifically, the server uses image recognition algorithms to detect and count the number and movement of vehicles, and an emotion recognition engine (e.g., facial expression analysis software) to analyze passengers' emotions and stress levels from their facial expressions in the video data.

[1088] Step 5:

[1089] Integrated evaluation

[1090] The server comprehensively evaluates the traffic volume data and emotion data.

[1091] Input: Traffic data and sentiment data.

[1092] Output: Overall evaluation result.

[1093] Specifically, the server uses the analysis results, predefined rules or generative AI models to evaluate the relationship between traffic conditions and stress levels, and determines the necessary adjustments.

[1094] Step 6:

[1095] Execution of signal control algorithms

[1096] The server executes the signal control algorithm based on the comprehensive evaluation result.

[1097] Input: Overall evaluation result.

[1098] Output: Signal control instructions.

[1099] Specifically, the server runs an algorithm to calculate the optimal signal duration and determine, for example, the green light extension time.

[1100] Step 7:

[1101] Sending signal control instructions

[1102] The server transmits the calculated signal control instructions to the terminal.

[1103] Input: Signal control instructions.

[1104] Output: Signal control instructions sent to the terminal.

[1105] As a specific operation, the server transmits instruction data including a command to adjust the signal timing to the terminal.

[1106] Step 8:

[1107] Traffic light adjustments

[1108] The terminal adjusts the signaling device based on the received signal control instructions.

[1109] Input: Signal control instructions.

[1110] Output: Traffic light operating status after adjustment.

[1111] Specifically, the terminal sends instructions to the traffic light control unit to change its timing.

[1112] Step 9:

[1113] Collecting and sending feedback

[1114] The terminal again collects traffic data and emotion data for the new signal pattern and transmits them to the server.

[1115] Input: Traffic data and emotion data with adjusted signal patterns.

[1116] Output: Feedback data sent to the server.

[1117] As a specific operation, the terminal collects data in the same manner as in the first step and transmits it to the server as feedback data.

[1118] Step 10:

[1119] Feedback analysis and algorithm improvement

[1120] The server analyzes the feedback data and evaluates the effectiveness of the signal control algorithm.

[1121] Input: Feedback data.

[1122] Output: Algorithm improvements.

[1123] Specifically, the server compares the collected data with the results of the algorithm and adjusts and improves the control algorithm as needed.

[1124] The above are the processing steps of the system for implementing the present invention.

[1125] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1126] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1127] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1128] [Third embodiment]

[1129] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1130] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1131] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1132] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1133] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1134] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1135] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1136] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1137] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1138] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1139] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1140] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1141] The present invention relates to a system for mitigating traffic congestion by utilizing cameras installed at traffic lights. Specific embodiments of the system are described below.

[1142] (Data collection and preprocessing)

[1143] Terminal

[1144] The terminal is responsible for acquiring video data in real time from cameras installed at traffic lights. The acquired video data undergoes pre-processing to analyze traffic volume and vehicle congestion. Specifically, the device performs noise removal and contrast adjustment on the video data.

[1145] (Data analysis and traffic volume evaluation)

[1146] server

[1147] The server analyzes pre-processed video data sent from the devices to assess traffic volume. The analysis is performed using image recognition algorithms to detect and count the number of vehicles and their movements. This data is used to assess the traffic situation at specific intersections and road sections in real time.

[1148] (Execution of signal control algorithm)

[1149] server

[1150] The server then executes a traffic light control algorithm based on the results of its assessment of traffic volume and congestion. This algorithm calculates the optimal signal lighting time for each intersection. For example, dynamic adjustments are made, such as extending the green light time at intersections where vehicles are congested.

[1151] (Generation and transmission of signal control instructions)

[1152] server

[1153] The server generates signal control instructions based on the calculated optimal lighting time. These instructions are instantly sent to each traffic light, and the lighting time of the traffic light is adjusted in real time. For example, an instruction to extend the green light at intersection A by 30 seconds is sent from the server to the terminal.

[1154] (Traffic signal adjustment)

[1155] Terminal

[1156] The terminal receives the signal control instruction sent from the server and automatically adjusts the lighting time of the traffic light, for example, by extending the green light time by 30 seconds.

[1157] (Feedback collection and analysis)

[1158] Terminal

[1159] After adjusting the traffic lights, the terminal again collects traffic data based on the new signal pattern and transmits it to the server.

[1160] server

[1161] The server analyzes this feedback data and continuously improves the signal control algorithm, allowing it to flexibly respond to changes in traffic conditions and optimize the entire system.

[1162] (Example)

[1163] Example 1: Reducing congestion at intersection A

[1164] 1. Data Collection

[1165] The terminal acquires video data from a camera installed at intersection A.

[1166] Preprocess the video data and send it to the server.

[1167] 2. Data Analysis

[1168] The server analyzes the received video data using an image recognition algorithm and counts the number of vehicles at intersection A.

[1169] Based on the analysis results, the traffic volume and congestion situation at intersection A are evaluated.

[1170] 3. Execution of signal control algorithm

[1171] The server executes a traffic light control algorithm to extend the green light duration at intersection A by 30 seconds.

[1172] 4. Transmission of signal control instructions

[1173] The server transmits the generated signal control instruction to the terminal.

[1174] 5. Traffic light adjustments

[1175] The device adjusts and controls the traffic light to extend the green light duration by 30 seconds.

[1176] 6. Feedback Collection and Analysis

[1177] The terminal again collects traffic data based on the new signal pattern and transmits it to the server.

[1178] The server analyzes the feedback data and improves the traffic light control algorithm.

[1179] In this way, traffic flow can be optimized and congestion can be alleviated by dynamically adjusting the illumination duration of traffic lights based on the analysis of video data acquired by the camera.

[1180] The processing flow will be explained below.

[1181] Step 1:

[1182] Terminal

[1183] The device acquires video data in real time from cameras installed at traffic lights. Specifically, it acquires video within the camera's field of view and extracts frames at regular intervals. This video data is used as basic data for analyzing traffic volume and vehicle congestion.

[1184] Step 2:

[1185] Terminal

[1186] The device performs preprocessing on the captured video data, including noise reduction, contrast adjustment, and resolution optimization, to improve analysis accuracy and reduce the risk of misidentification.

[1187] Step 3:

[1188] Terminal

[1189] The pre-processed video data is sent to the server, which converts the data packets into an appropriate format and transfers them to the server in real time over the network.

[1190] Step 4:

[1191] server

[1192] The server receives pre-processed video data sent from the device, ready for immediate analysis.

[1193] Step 5:

[1194] server

[1195] The server analyzes the received video data using an image recognition algorithm. Specifically, it counts the number of vehicles in each frame and measures their movement speed and stationary time. Based on the results of this analysis, it evaluates traffic volume and stationary conditions.

[1196] Step 6:

[1197] server

[1198] Based on the results of the traffic volume and congestion assessment, the system executes a traffic light control algorithm that calculates the optimal lighting times (red, green, and yellow) for each intersection. For example, this may include extending the green light time on roads with heavy traffic.

[1199] Step 7:

[1200] server

[1201] Based on the calculated optimal lighting time, specific traffic light control instructions are generated, and the server converts these instructions into data packets for transmission to the target traffic light.

[1202] Step 8:

[1203] server

[1204] The server transmits the signal control instructions, including the traffic light control parameters (lighting time, sequence, etc.), to the terminal via the network.

[1205] Step 9:

[1206] Terminal

[1207] The terminal receives the signal control instructions sent from the server. This is applied to the internal control system to adjust the lighting time of the traffic light. Specifically, it controls the time the green light stays on by 30 seconds, for example.

[1208] Step 10:

[1209] Terminal

[1210] The operation of the adjusted traffic lights is monitored and traffic data is collected again based on the new signal pattern, which is then sent as feedback to the server.

[1211] Step 11:

[1212] server

[1213] The server receives feedback data from the devices and evaluates the effectiveness of the signal control algorithm. Based on the analysis results, the algorithm is improved and used for future adjustments.

[1214] Step 12:

[1215] User

[1216] Users can check current traffic conditions and traffic light change information through a dedicated app or website, and user feedback will also be incorporated into the system for further optimization.

[1217] Example 1

[1218] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1219] Conventional traffic systems face the challenge of effectively alleviating traffic congestion in real time. Specifically, they lack the ability to properly grasp ongoing traffic conditions and dynamically adjust traffic light illumination times accordingly. As a result, traffic flow becomes unsmooth and congestion occurs.

[1220] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1221] In this invention, the server includes means for collecting video data acquired by a camera, means for pre-processing the video data acquired by the camera, means for analyzing the collected video data to evaluate traffic volume, means for executing a signal control algorithm based on the evaluation results, means for generating and transmitting signal control instructions to the traffic lights, means for adjusting the illumination duration of the traffic lights based on the signal control instructions, and means for collecting and analyzing feedback data. This makes it possible to monitor traffic conditions in real time and perform appropriate traffic light control. Furthermore, the collected feedback data can be used to continuously improve the signal control algorithm, thereby effectively alleviating traffic congestion.

[1222] "Video data acquired by a camera" refers to video information captured in real time by a camera installed at an intersection or road section.

[1223] "Preprocessing" refers to a data cleaning process, such as noise removal and contrast adjustment, that is carried out before analyzing the video data acquired by the camera.

[1224] "Analysis" is a method of evaluating traffic volume and vehicle congestion based on preprocessed video data, and is carried out using image recognition algorithms.

[1225] "Evaluating traffic volume" means detecting the number and movement of vehicles at a particular intersection or road section from the analyzed data and determining the traffic situation.

[1226] A "signal control algorithm" refers to the calculation procedures and rules for calculating the optimal lighting time of a traffic light based on the results of traffic volume evaluation.

[1227] A "signal control instruction" is a command that indicates the lighting time of a specific traffic light, generated by a signal control algorithm.

[1228] "Adjusting the lighting time of a traffic light" means changing the display time of a green light, red light, etc. of a traffic light based on a signal control instruction.

[1229] "Feedback data" refers to new traffic data that is re-collected after adjusting traffic light durations and is used to continuously improve the algorithm.

[1230] A "traffic light" is a display device installed at an intersection or road section to control the flow of traffic.

[1231] MODE FOR CARRYING OUT THE INVENTION

[1232] The present invention is a system that uses cameras installed at traffic lights to alleviate traffic congestion. A specific embodiment of this system is described below.

[1233] Data collection and preprocessing

[1234] Terminal

[1235] The device acquires video data in real time from cameras installed at traffic lights. The hardware used includes commercially available surveillance cameras and a small computer for data collection. This video data undergoes preprocessing before analyzing traffic volume and vehicle congestion. Specifically, libraries such as OpenCV are used to remove noise and adjust contrast.

[1236] Data analysis and traffic volume evaluation

[1237] server

[1238] The server receives and analyzes the preprocessed video data sent from the device. This analysis uses image recognition algorithms such as YOLO (You Only Look Once) to detect and count vehicles in the video in real time. Based on the analysis results, the server evaluates the traffic volume and congestion status at specific intersections and road sections. The evaluation results are temporarily stored in a database.

[1239] Execution of signal control algorithms

[1240] server

[1241] The server then executes a traffic light control algorithm based on the results of its assessment of traffic volume and congestion. This algorithm calculates the optimal signal lighting time for each intersection. For example, at intersections where traffic volume has increased and congestion has occurred, dynamic adjustments are made, such as extending the green light time by 30 seconds.

[1242] Generation and transmission of signal control instructions

[1243] server

[1244] The server generates traffic light control instructions based on the calculated optimal lighting time. These instructions are instantly sent to each traffic light. Communication protocols such as MQTT are used to send specific instructions, such as "Extend the green light at intersection A by 30 seconds," to the terminal in JSON format.

[1245] Traffic light adjustments

[1246] Terminal

[1247] The terminal receives the signal control instructions sent from the server and automatically adjusts the traffic lights based on the instructions. A PLC (Programmable Logic Controller) is used to control the traffic lights, and the green light duration is extended as specified.

[1248] Collecting and analyzing feedback

[1249] Terminal

[1250] After adjusting the traffic lights, the device collects traffic data again based on the new signal patterns and sends it to the server. This data, also in the form of video data, is again pre-processed and analyzed.

[1251] server

[1252] The server analyzes this feedback data and adjusts parameters to continuously improve the signal control algorithm, allowing it to flexibly respond to changing traffic conditions.

[1253] Specific examples

[1254] As a specific example, the case of easing traffic congestion at intersection A will be shown below.

[1255] 1. Data Collection

[1256] The terminal acquires video data from a camera installed at intersection A, preprocesses it, and sends it to the server.

[1257] 2. Data Analysis

[1258] The server analyzes the received video data using an image recognition algorithm and counts the number of vehicles at intersection A.

[1259] Based on the analysis results, the traffic volume and congestion situation at intersection A are evaluated.

[1260] 3. Execution of signal control algorithm

[1261] The server executes a traffic light control algorithm to extend the green light duration at intersection A by 30 seconds.

[1262] 4. Transmission of signal control instructions

[1263] The server transmits the generated signal control instruction to the terminal.

[1264] 5. Traffic light adjustments

[1265] The device adjusts and controls the traffic light to extend the green light duration by 30 seconds.

[1266] 6. Feedback Collection and Analysis

[1267] The terminal again collects traffic data based on the new signal pattern and transmits it to the server.

[1268] The server analyzes the feedback data and improves the traffic light control algorithm.

[1269] Prompt Sentence Examples

[1270] "How will this system alleviate traffic congestion? What are the specific steps and technologies used?"

[1271] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1272] Step 1:

[1273] Data collection

[1274] Terminal

[1275] The terminal acquires video data in real time from cameras installed at traffic lights. Specifically, commercially available surveillance cameras capture video at a set frame rate. The input data is the captured raw video data. This data is preprocessed in the next step.

[1276] Step 2:

[1277] Video data preprocessing

[1278] Terminal

[1279] The device preprocesses the acquired video data. Specifically, it uses the OpenCV library to remove noise and adjust the contrast of the video data. The input data is raw video data. The output data is preprocessed video data with noise removed and contrast adjusted. This enables subsequent image analysis to be performed with high accuracy.

[1280] Step 3:

[1281] Data analysis

[1282] server

[1283] The server receives preprocessed video data sent from the device. The server analyzes this data using image recognition algorithms such as YOLO. The input data is the preprocessed video data, and the output data is the number and location of detected vehicles. Based on the analyzed data, traffic volume and congestion at specific intersections and road sections are evaluated.

[1284] Step 4:

[1285] Traffic volume evaluation

[1286] server

[1287] The server evaluates traffic volume based on the analysis results. Specifically, it calculates the actual traffic volume and congestion status at specific intersections and road sections based on the number of vehicles and location information obtained from the analysis. The input data is the number of vehicles and location information from the analysis results, and the output data is the evaluated traffic volume and congestion status. This evaluation result is temporarily stored in a database.

[1288] Step 5:

[1289] Execution of signal control algorithms

[1290] server

[1291] The server executes a signal control algorithm based on the evaluation results of traffic volume and congestion status. The input data is the traffic volume and congestion status of the evaluation results, and the output data is the optimal signal lighting time for each intersection. Specifically, at intersections where vehicles are congested, dynamic adjustments are made, such as extending the green light time.

[1292] Step 6:

[1293] Generation and transmission of signal control instructions

[1294] server

[1295] The server generates traffic light control instructions based on the calculated optimal lighting time. These instructions are instantly sent to each traffic light using a communication protocol such as MQTT. The input data is the optimized lighting time, and the output data is the specific control instructions for the traffic light. For example, this could include an instruction such as "Extend the green light at intersection A by 30 seconds."

[1296] Step 7:

[1297] Traffic light adjustments

[1298] Terminal

[1299] The terminal receives signal control instructions sent from the server. Based on the received instructions, the terminal controls the traffic light. The input data is the signal control instructions, and the output operation is to adjust the traffic light's lighting time. Specifically, the green light's lighting time is extended as specified. A PLC (Programmable Logic Controller) is used to control the traffic light.

[1300] Step 8:

[1301] Feedback data collection

[1302] Terminal

[1303] After the traffic light is adjusted, the terminal collects traffic data again according to the new signal pattern. The input data is the video data under the new signal pattern, and the output data is the preprocessed video data.

[1304] Step 9:

[1305] Analysis of feedback data and algorithm improvement

[1306] server

[1307] The server receives and analyzes feedback data sent from the terminal. The input data is preprocessed feedback video data, and the output data is the analysis results of the number of vehicles and their location information. Based on these results, the signal control algorithm is continuously improved to achieve adaptive signal control.

[1308] (Application example 1)

[1309] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1310] Conventional traffic systems have difficulty in controlling signals in a fast and flexible manner to respond to fluctuations in traffic volume, and congestion has not been sufficiently alleviated. Furthermore, conventional navigation systems for autonomous vehicles are unable to reflect real-time traffic information and may not be able to provide optimal driving routes.

[1311] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1312] In this invention, the server includes means for collecting video data acquired by cameras, means for analyzing the collected video data to evaluate traffic volume, means for dynamically adjusting the illumination time of traffic signals based on the evaluation results, and means for providing traffic information in real time and optimizing vehicle routes, thereby enabling dynamic adjustment of traffic volume and optimal navigation.

[1313] A "camera" is a device installed at a traffic light that captures images of road traffic conditions in real time.

[1314] "Video data" refers to digital video recordings of traffic conditions captured by a camera.

[1315] "Collection means" is a function that collects video data acquired by a camera in real time into a database or server.

[1316] The "analysis means" is a function that uses collected video data to evaluate traffic volume and identify the number of vehicles and congestion conditions.

[1317] "Evaluation results" refer to data indicating traffic volume and congestion conditions obtained by the analysis means.

[1318] The "signal lighting time adjustment means" is a function for dynamically changing the lighting time of the green or red signal of a traffic light based on the evaluation result.

[1319] "Signal control instructions" are control information for instructing the lighting time and operating state of a traffic light.

[1320] A "traffic light" is an optical signal device installed at road intersections and other locations to control traffic flow.

[1321] "Real-time traffic information provision means" is a function that provides the current optimal driving route based on traffic data.

[1322] The "driving route optimization means" is a function that navigates the vehicle along the optimal route based on real-time traffic information.

[1323] A system for carrying out the present invention comprises a camera, a server, and a terminal. Specific embodiments will be described in detail below.

[1324] Terminal

[1325] First, cameras installed at traffic lights capture real-time video data of road traffic conditions. The captured video data undergoes preprocessing such as noise removal and contrast adjustment on the device. After preprocessing, the data is sent to the server.

[1326] server

[1327] The server uses an image recognition algorithm to evaluate traffic volume based on preprocessed video data sent from the device. Specifically, it counts the number of vehicles and detects their movements. Based on the analysis results, it evaluates traffic volume and congestion conditions.

[1328] By running the signal control algorithm on the server, the optimal signal lighting time for each intersection is calculated. For example, dynamic adjustments are made, such as extending the green light time at intersections where congestion is occurring. The calculated optimal signal control instructions are sent to each traffic light in real time.

[1329] The server also provides real-time navigation information based on traffic information, allowing it to present optimal driving routes to autonomous vehicles.

[1330] Autonomous vehicles

[1331] The autonomous vehicle receives real-time traffic information sent from the server and optimizes its route, for example by changing its route to coincide with the time when the green light remains green at a particular intersection for a long period of time.

[1332] Specific examples

[1333] Video data obtained by the camera at intersection A is preprocessed on the terminal and sent to the server. The server uses an analysis algorithm to count the number of vehicles and evaluate traffic volume. If congestion occurs, a signal control algorithm is executed to generate signal control instructions that dynamically extend the green light duration and send them to the traffic lights. Real-time traffic information is also sent to the navigation system, which displays the optimal driving route.

[1334] Prompt Sentence Examples

[1335] "I want to build an application that analyzes video data from a traffic light camera at intersection A and suggests the optimal driving route based on traffic congestion. Please tell me the code to evaluate the current traffic volume, obtain signal control information based on that data, and calculate the optimal route."

[1336] In this way, traffic lights can be dynamically adjusted based on the analysis of video data acquired by cameras, providing autonomous vehicles with optimal routes to optimize traffic flow and alleviate congestion.

[1337] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1338] Step 1: Acquire video data with the camera

[1339] Cameras installed at traffic lights capture traffic conditions in real time. The video data obtained from the cameras is digital footage that captures the movement of vehicles on the road. Specifically, the video captured by the cameras is sent to a terminal in streaming format. The input of this step is the camera image itself, and the output is the unprocessed video data.

[1340] Step 2: Preprocess the video data

[1341] The device receives video data captured by the camera and performs preprocessing such as noise reduction and contrast adjustment. Specifically, the video data is converted to grayscale using a library such as OpenCV, and noise is removed by applying a Gaussian filter. The input of this step is the raw video data, and the output is the preprocessed video data.

[1342] Step 3: Send the preprocessed data to the server

[1343] The terminal sends the preprocessed video data to the server. In this step, data is transferred to the server using a communication protocol such as an HTTP request. The input is the preprocessed video data, and the output is the data reception on the server.

[1344] Step 4: Analyze video data to assess traffic volume

[1345] The server analyzes the received preprocessed video data and evaluates traffic volume. This process uses an image recognition algorithm to count the number of vehicles and identify their movements. For example, it uses YOLO or OpenCV vehicle detection models. The input is the preprocessed video data, and the output is an evaluation result showing the number of vehicles and congestion status.

[1346] Step 5: Run the signal control algorithm

[1347] The server executes a signal control algorithm based on the evaluation results. Specifically, it dynamically adjusts signals, such as extending the green light duration at intersections with congestion. The input is evaluation data based on vehicle detection results, and the output is optimal signal control instructions.

[1348] Step 6: Send signal control instructions to the traffic light

[1349] The signal control instructions generated by the server are sent to the terminal to control the traffic lights. Here, signal control information is sent to the traffic lights using a communication protocol. The input is the signal control instructions, and the output is the transfer of instructions to the traffic lights.

[1350] Step 7: Providing real-time traffic information

[1351] The server provides real-time traffic information to autonomous vehicles based on traffic volume evaluation results and signal control information. Specifically, it sends route optimization information via API. The input is the latest traffic data, and the output is route optimization information.

[1352] Step 8: Optimize the autonomous vehicle's route

[1353] An autonomous vehicle optimizes its driving route based on traffic information received from a server. Here, the vehicle's navigation system analyzes the information from the server and presents the optimal route. The input is the optimized driving route information, and the output is the optimal route the vehicle should follow.

[1354] By going through the above processing steps, it is possible to optimize traffic flow and alleviate congestion.

[1355] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1356] This invention relates to a system that alleviates traffic congestion by utilizing cameras installed at traffic lights and combining them with an emotion engine that recognizes the emotions of users. Specific embodiments of this system are described below.

[1357] (Data collection and preprocessing)

[1358] Terminal

[1359] The device is responsible for acquiring video data in real time from cameras installed at traffic lights. The acquired video data undergoes pre-processing to analyze traffic volume and vehicle congestion. Specifically, noise removal and contrast adjustment are performed on the video data. The camera also captures the facial expressions of the vehicle's users, which are used as emotional data.

[1360] (Data analysis and traffic volume evaluation)

[1361] server

[1362] The server receives preprocessed video data and emotion data sent from the device. Using the received data, it analyzes traffic volume and then analyzes the user's emotions using an emotion engine. Traffic volume analysis uses an image recognition algorithm to detect and count the number of vehicles and their movements. Meanwhile, the emotion engine measures the user's stress level and irritation from their facial expressions.

[1363] (traffic volume and user sentiment evaluation)

[1364] server

[1365] The server comprehensively evaluates traffic volume and user emotion data, and determines that traffic conditions are deteriorating or the user's stress level is high, and prioritizes traffic light adjustment.

[1366] (Execution of signal control algorithm)

[1367] server

[1368] The server then executes a traffic light control algorithm based on the evaluation results. The algorithm calculates the optimal traffic light duration for each intersection. For example, if traffic volume is heavy and the user is feeling stressed, the algorithm dynamically adjusts the duration of the green light, such as extending it.

[1369] (Generation and transmission of signal control instructions)

[1370] server

[1371] The server generates signal control instructions based on the calculated optimal lighting time. These instructions are instantly sent to each traffic light, and the lighting time of the traffic light is adjusted in real time. For example, an instruction to extend the green light at intersection A by 30 seconds is sent from the server to the terminal.

[1372] (Traffic signal adjustment)

[1373] Terminal

[1374] The device receives the signal control instructions sent from the server and automatically adjusts the traffic light duration, for example, extending the green light duration by 30 seconds.

[1375] (Feedback collection and analysis)

[1376] Terminal

[1377] After the traffic light is adjusted, the terminal collects traffic data and user emotion data according to the new traffic light pattern again and transmits them to the server.

[1378] server

[1379] The server analyzes this feedback data and evaluates the effectiveness of the signal control algorithm. Based on the analysis results, the algorithm is improved and used for future adjustments.

[1380] (Example)

[1381] Example 1: Reducing congestion at intersection A

[1382] 1. Data Collection

[1383] The terminal acquires video data and emotion data from a camera installed at intersection A.

[1384] Preprocess the video data and send it to the server.

[1385] 2. Data Analysis

[1386] The server analyzes the received video data using an image recognition algorithm and counts the number of vehicles at intersection A.

[1387] An emotion engine is used to analyze the user's emotion data and assess their stress level.

[1388] 3. Evaluation of traffic volume and emotion data

[1389] The server comprehensively evaluates the traffic volume at intersection A and the user's emotional data to determine the traffic situation and the user's stress level.

[1390] 4. Execution of signal control algorithm

[1391] The server executes a traffic light control algorithm to extend the green light duration at intersection A by 30 seconds.

[1392] 5. Transmission of signal control instructions

[1393] The server transmits the generated signal control instruction to the terminal.

[1394] 6. Traffic light adjustment

[1395] The device adjusts and controls the traffic light to extend the green light duration by 30 seconds.

[1396] 7. Feedback Collection and Analysis

[1397] The terminal again collects traffic data and user emotion data based on the new signal pattern and transmits them to the server.

[1398] The server analyzes the feedback data and improves the traffic light control algorithm.

[1399] In this way, traffic lights can be dynamically adjusted based on the video data acquired by the camera and the user's emotional data obtained from the emotion engine, thereby optimizing traffic flow, easing congestion, and reducing user stress.

[1400] The processing flow will be explained below.

[1401] Step 1:

[1402] Terminal

[1403] The device acquires real-time video data from cameras installed at traffic lights. The cameras are high-resolution video cameras that capture the entire intersection and extract frames at regular intervals, providing data for monitoring temporary congestion and fluctuations in traffic volume.

[1404] Step 2:

[1405] Terminal

[1406] Preprocessing is performed on the video data captured by the camera. Specifically, processes such as noise removal, contrast adjustment, and resolution optimization are performed. This improves the quality of the video data and increases the accuracy of analysis. In addition, if the user is in the vehicle, their facial expressions are captured and extracted as emotion data.

[1407] Step 3:

[1408] Terminal

[1409] The preprocessed video data and emotion data are sent to the server. The data is set to be sent in real time via the network, allowing the server to constantly monitor the latest traffic conditions and the user's emotional state.

[1410] Step 4:

[1411] server

[1412] The server receives the preprocessed video data and emotion data sent from the device. It then applies an image recognition algorithm to analyze the received video data. Specifically, the analysis counts the number of vehicles in each frame and measures their movement and dwell time.

[1413] Step 5:

[1414] server

[1415] The server uses an emotion engine to analyze the user's emotional data. Specifically, it uses facial expression recognition technology to evaluate the user's stress level and irritation. This evaluation result is combined with traffic volume data to help understand the overall traffic situation.

[1416] Step 6:

[1417] server

[1418] The system evaluates traffic volume and user emotional data in an integrated manner. For example, when traffic volume is heavy and vehicles are backed up, and when a high stress level is detected from the user's emotional data, it determines that it is necessary to raise the priority of traffic signal control.

[1419] Step 7:

[1420] server

[1421] Based on the evaluation results, a traffic light control algorithm is executed. The algorithm calculates the optimal traffic light duration for each intersection. Dynamic adjustments are made, such as extending the green light duration at intersection A by 30 seconds.

[1422] Step 8:

[1423] server

[1424] It generates signal control instructions, including the specific time to extend the green light and the priority of roads, and converts these instructions into data packets for transmission to the traffic light controller.

[1425] Step 9:

[1426] server

[1427] A signal control instruction is sent to the terminal. The instruction is sent immediately via the network, and the server receives a response confirming receipt, thereby confirming that the instruction was transmitted correctly.

[1428] Step 10:

[1429] Terminal

[1430] The terminal receives the signal control instructions sent from the server. It applies these to the internal control system and sets the lighting time and sequence of the traffic lights. Specifically, it controls the signal by extending the green light time by 30 seconds, for example.

[1431] Step 11:

[1432] Terminal

[1433] The operation of the traffic lights after adjustment is monitored. Traffic data and user emotion data based on the new traffic light patterns are collected again and sent to the server.

[1434] Step 12:

[1435] server

[1436] The server receives feedback data from the devices and evaluates the effectiveness of the signal control algorithm. Based on the analysis results, the algorithm is improved and used for future adjustments.

[1437] Step 13:

[1438] User

[1439] Users can check current traffic conditions and traffic light change information through a dedicated app or website, and receive advice and notifications based on their stress and irritation levels to help them drive with peace of mind.

[1440] Example 2

[1441] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1442] Conventional traffic signal control systems are unable to respond quickly to fluctuations in traffic volume, making it difficult to effectively alleviate traffic congestion. Furthermore, because they do not take into account the user's stress or emotional state, drivers experience increased stress and the risk of traffic accidents increases as they spend longer at intersections. The present invention aims to solve these problems by analyzing traffic volume and the user's emotional state in real time and dynamically adjusting signal control.

[1443] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1444] In this invention, the server includes means for collecting video data captured by cameras installed at traffic lights in real time, means for performing noise reduction and contrast adjustment on the collected video data, means for capturing a user's facial expressions to collect emotion data, means for analyzing the preprocessed video data and emotion data to evaluate traffic volume and the user's emotional state, means for executing a signal control algorithm based on the evaluation results, means for generating and transmitting signal control instructions to the traffic lights, means for re-collecting new traffic data and emotion data based on the adjusted signal pattern, and means for analyzing the feedback data to improve the signal control algorithm, thereby enabling fast and effective signal control based on fluctuations in traffic volume and the user's emotional state.

[1445] A "traffic light" is an optical signaling device installed to control the flow of road traffic.

[1446] A "camera" is a photographing device for acquiring video data.

[1447] "Video Data" refers to visual information captured by a camera.

[1448] "Real-time" means that data acquisition and processing occur almost simultaneously.

[1449] "Noise reduction" is the process of removing unnecessary information and errors from collected video data.

[1450] "Contrast adjustment" is a process for appropriately adjusting the contrast between light and dark in the video data.

[1451] "Facial expressions" express emotions based on the shape and movement of the user's face.

[1452] "Emotion data" is information that represents the user's emotional state.

[1453] "Analysis" is the process of examining data in detail to clarify its content and meaning.

[1454] "Traffic volume" refers to the number and flow of vehicles in a particular time unit.

[1455] "Evaluation" is the act of judging a situation or condition based on data.

[1456] A "traffic light control algorithm" is a calculation method for determining the lighting time and control pattern of traffic lights.

[1457] A "signal control instruction" is a specific command for controlling the operation of a traffic light.

[1458] "Feedback data" is new data collected after traffic light adjustments.

[1459] "User" refers to the driver or passenger of a vehicle that uses this system.

[1460] This invention relates to a system that utilizes cameras installed at traffic lights and combines them with an emotion engine that recognizes the emotions of users to alleviate traffic congestion. Specific embodiments of this system are described below.

[1461] (System Configuration)

[1462] Hardware

[1463] Cameras installed at traffic lights:

[1464] The camera captures the traffic conditions at the intersection in real time, collecting video data, and also captures the user's facial expressions to obtain emotional data.

[1465] Device:

[1466] The terminal is installed at the traffic light, receives the video data from the camera, performs noise reduction and contrast adjustment, and sends the pre-processed data to the server.

[1467] server:

[1468] The server receives and analyzes the video and emotion data sent from the device. Based on the results of the analysis, it executes a traffic light control algorithm and generates instructions for the traffic light.

[1469] (software)

[1470] Image Recognition Algorithm

[1471] The server uses image recognition algorithms to analyze the number and movement of vehicles from pre-processed video data, allowing for accurate assessment of traffic volume.

[1472] Emotion Engine

[1473] The server uses an emotion engine to analyze the user's facial expression data and assess their stress level and irritation, thereby taking into account not only the traffic situation but also the user's emotional state.

[1474] Signal Control Algorithm

[1475] The server runs a traffic light control algorithm based on the results of traffic volume analysis and emotion analysis, which calculates the optimal signal lighting time for each intersection and dynamically adjusts the traffic lights.

[1476] Feedback Mechanism

[1477] After adjusting the traffic lights, the device will again collect traffic and emotion data based on the new signal patterns and send it to the server, which will analyze this feedback data, evaluate the effectiveness of the signal control algorithm, and make improvements.

[1478] (Example)

[1479] Example 1: Reducing congestion at intersection A

[1480] 1. Data Collection:

[1481] The terminal acquires video data and emotion data from a camera installed at intersection A. The acquired data is preprocessed and sent to the server.

[1482] 2. Data Analysis:

[1483] The server analyzes the received video data using an image recognition algorithm and counts the number of vehicles at intersection A. It uses an emotion engine to analyze the user's emotion data and evaluate their stress level.

[1484] 3. Traffic and Sentiment Data Evaluation:

[1485] The server comprehensively evaluates the traffic volume at intersection A and the user's emotional data to determine the traffic situation and the user's stress level.

[1486] 4. Execution of signal control algorithm:

[1487] The server executes a traffic light control algorithm to extend the green light duration at intersection A by 30 seconds.

[1488] 5. Sending signal control instructions:

[1489] The server transmits the generated signal control instruction to the terminal.

[1490] 6. Traffic light adjustment:

[1491] The device adjusts and controls the traffic light to extend the green light duration by 30 seconds.

[1492] 7. Feedback Collection and Analysis:

[1493] The terminal again collects traffic data and user emotion data based on the new signal pattern and transmits them to the server, which analyzes the feedback data and improves the signal control algorithm.

[1494] Prompt Sentence Examples

[1495] "Please explain a system that uses cameras and an emotion engine to collect and analyze traffic volume and user emotion data, and dynamically adjusts traffic light durations to alleviate congestion at intersections."

[1496] In this way, traffic lights can be dynamically adjusted based on the video data acquired by the camera and the user's emotional data obtained from the emotion engine, thereby optimizing traffic flow, easing congestion, and reducing user stress.

[1497] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1498] Step 1:

[1499] Data collection

[1500] The terminal acquires video data in real time from cameras installed at traffic lights.

[1501] Input: Video data from a camera installed at a traffic light.

[1502] Data processing and calculation: Capture of video data.

[1503] Output: Captured video data.

[1504] Step 2:

[1505] Video data preprocessing

[1506] The device performs noise removal and contrast adjustment on the acquired video data.

[1507] Input: Captured video data.

[1508] Data processing and calculation: noise removal and contrast adjustment.

[1509] Output: Pre-processed video data. Specific operations include applying noise filters and histogram equalization.

[1510] Step 3:

[1511] Collecting Emotional Data

[1512] The device's camera captures the facial expressions of the user riding in the vehicle and uses them as emotional data.

[1513] Input: Video data containing the user's facial expressions.

[1514] Data processing and calculation: Image analysis for facial expression recognition.

[1515] Output: User's emotional data. Specifically, the stress level and irritation level are measured from the user's facial expressions.

[1516] Step 4:

[1517] Video data analysis

[1518] The server receives the pre-processed video data sent from the device and analyzes it using image recognition algorithms.

[1519] Input: Preprocessed video data.

[1520] Data processing and calculation: The number and movement of vehicles are detected and counted using an image recognition algorithm.

[1521] Output: Parsed traffic data. Specific actions include vehicle identification and counting.

[1522] Step 5:

[1523] Emotional Data Analysis

[1524] The server uses an emotion engine to analyze the user's facial expression data and assess their stress level and irritability.

[1525] Input: User emotion data.

[1526] Data processing and calculation: Emotion analysis using an emotion engine.

[1527] Output: Analyzed emotional state data. Specifically, facial expression recognition technology is used to detect the user's micro-expressions and estimate their emotional state.

[1528] Step 6:

[1529] Integrated evaluation of traffic volume and emotion data

[1530] The server performs an integrated evaluation based on the analyzed traffic volume data and emotion data.

[1531] Input: Traffic data and emotional state data.

[1532] Data processing and calculation: Integrated evaluation of data.

[1533] Output: Integrated evaluation results. Specific actions include determining whether traffic conditions are worsening or the user's stress level is high.

[1534] Step 7:

[1535] Execution of signal control algorithms

[1536] The server executes the signal control algorithm based on the results of the integrated evaluation.

[1537] Input: Integrated evaluation results.

[1538] Data processing and calculation: Application of signal control algorithms.

[1539] Output: Optimal signal control instructions. Specifically, the optimal green and red light durations at each intersection are calculated.

[1540] Step 8:

[1541] Generation of signal control instructions

[1542] The server generates signal control instructions based on the results of the signal control algorithm.

[1543] Input: The result of the signal control algorithm.

[1544] Data processing and calculation: Generation of signal control instructions.

[1545] Output: Specific signal control instructions. Specifically, instructions such as "Extend the green light at intersection A by 30 seconds" are generated.

[1546] Step 9:

[1547] Sending signal control instructions

[1548] The server transmits the generated signal control instruction to the terminal.

[1549] Input: Signal control instructions.

[1550] Data processing and calculation: Sending signal control instructions.

[1551] Output: Signal control instructions sent to the terminal.

[1552] Step 10:

[1553] Traffic light adjustments

[1554] The terminal adjusts the lighting time of the traffic light based on the signal control instruction received from the server.

[1555] Input: Signal control instruction from the server.

[1556] Data processing and calculation: Traffic light control.

[1557] Output: Adjusted traffic lights. Specifically, the green light duration is extended by 30 seconds.

[1558] Step 11:

[1559] Collecting new data

[1560] After the traffic light is adjusted, the terminal collects traffic data and emotion data again according to the new traffic light pattern.

[1561] Input: Adjusted traffic data and sentiment data.

[1562] Data processing and calculation: Data collection.

[1563] Output: Recollected traffic data and emotion data.

[1564] Step 12:

[1565] Analysis of feedback data

[1566] The server again analyzes the received feedback data and evaluates the effectiveness of the signal control algorithm.

[1567] Input: Recollected traffic data and emotion data.

[1568] Data processing and calculation: Analysis of feedback data.

[1569] Output: Analysis results, specifically evaluating the effectiveness of signal conditioning and modifying algorithm parameters if necessary.

[1570] Through the above steps, traffic flow can be optimized, congestion can be alleviated, and user stress can be reduced by dynamically adjusting the illumination times of traffic lights based on the video data acquired by the camera and the user's emotional data obtained from the emotion engine.

[1571] (Application example 2)

[1572] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1573] Conventional traffic signal control systems rely solely on traffic volume and do not take into account passenger emotions or stress levels, which limits their effectiveness in easing traffic congestion and reducing passenger stress. This often leads to poor traffic efficiency and lower passenger satisfaction. Furthermore, conventional systems have fixed signal control, making real-time adjustments difficult and preventing flexible responses to changing situations.

[1574] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1575] In this invention, the server includes means for collecting video data acquired by a camera, means for analyzing the collected video data to evaluate traffic volume, and emotion recognition means for evaluating passenger emotions, thereby enabling traffic signal control based on traffic conditions and passenger emotion data in real time.

[1576] "Camera" refers to an image capture device installed on traffic lights and vehicles to capture traffic and passenger expressions.

[1577] "Video data" refers to real-time video information captured by a camera.

[1578] "Traffic volume" is a number or data that indicates the number or flow of vehicles on a road in a particular area or period of time.

[1579] "Signal duration" refers to the duration that each color of light remains on at a traffic light, and is an important parameter for traffic control.

[1580] "Signal control instructions" refer to commands to set specific lighting patterns and times for traffic lights.

[1581] "Emotion recognition means" is a technology that analyzes passengers' facial expressions from video data acquired by a camera and evaluates their emotions and stress levels.

[1582] "Emotion Data" refers to data indicative of passenger emotions and stress levels analyzed by the emotion recognition means.

[1583] The "evaluation result" is information that indicates a comprehensive assessment of the situation obtained from the analysis of traffic volume and emotion data.

[1584] A "traffic light control algorithm" is a method or program for calculating optimal traffic light lighting patterns based on traffic volume and emotion data.

[1585] The "information processing device" is a device that includes a computer system for analyzing acquired data and performing signal control, etc.

[1586] This invention is a system that aims to alleviate traffic congestion and reduce passenger stress by analyzing traffic conditions and passenger emotions and controlling traffic signals in real time. A specific embodiment of this system is described below.

[1587] 1. Data Collection and Preprocessing

[1588] Terminal

[1589] The device captures real-time video data from cameras installed on traffic lights and autonomous vehicles. The video data is preprocessed to analyze traffic volume and vehicle congestion. Preprocessing includes noise reduction and contrast adjustment. The camera also captures facial expressions of passengers in the vehicle and uses this data as emotion data.

[1590] 2. Data analysis and traffic volume evaluation

[1591] server

[1592] The server receives preprocessed video data and emotion data sent from the terminals. Using this data, it analyzes traffic volume and further analyzes passenger emotions using an emotion recognition engine. Traffic volume analysis uses an image recognition algorithm to detect and count the number of vehicles and their movements. Meanwhile, the emotion recognition engine measures passenger stress levels and irritation from their facial expressions.

[1593] 3. Integrated evaluation of traffic volume and user sentiment

[1594] server

[1595] The server comprehensively evaluates traffic volume and emotion data, and determines that traffic conditions are deteriorating or passenger stress levels are high, and prioritizes traffic signal adjustments.

[1596] 4. Execution of signal control algorithm

[1597] server

[1598] Based on the evaluation results, the server executes a traffic light control algorithm, which calculates the optimal signal duration for each intersection. For example, if traffic volume is high and passengers are feeling stressed, dynamic adjustments are made, such as extending the green light duration.

[1599] 5. Generation and transmission of signal control instructions

[1600] server

[1601] The server generates signal control instructions based on the calculated optimal lighting time and instantly sends them to each traffic light device. For example, an instruction to extend the green light at a certain intersection by 30 seconds is sent from the server to the terminal.

[1602] 6. Traffic light adjustment

[1603] Terminal

[1604] The terminal receives the signal control instruction sent from the server and automatically adjusts the lighting time of the signal device, for example, extending the green light lighting time by 30 seconds.

[1605] 7. Feedback Collection and Analysis

[1606] Terminal and Server

[1607] After adjusting the traffic lights, the device again collects traffic data and passenger sentiment data based on the new signal patterns and sends them to the server. The server analyzes this feedback data and evaluates the effectiveness of the signal control algorithm. Based on the analysis results, the algorithm is improved and used for future adjustments.

[1608] Specific examples

[1609] As a concrete example 1, consider alleviating congestion at intersection A. A device acquires video data and emotion data from a camera installed at intersection A and sends it to a server. The server analyzes the received data, generates instructions to extend the green light duration by 30 seconds as needed, and sends them to the device. The traffic light follows this instruction and actually extends the green light duration by 30 seconds. Feedback data is collected again, and the server analyzes it to improve the algorithm.

[1610] Prompt Sentence Examples

[1611] The following prompt sentences are input into the generative AI model to analyze passenger emotional data:

[1612] Recognize the facial expressions of passengers in your car to assess their stress levels.

[1613] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1614] Step 1:

[1615] Data collection

[1616] The device acquires video data in real time from cameras installed on traffic lights and autonomous vehicles.

[1617] Input: Raw video data from the camera.

[1618] Output: Captured video data.

[1619] Specifically, the device captures video from a designated camera at specific time intervals and stores or transmits it as a data stream.

[1620] Step 2:

[1621] Video data preprocessing

[1622] Pre-processing such as noise removal and contrast adjustment is performed on the video data acquired by the device.

[1623] Input: Acquired raw video data.

[1624] Output: Preprocessed video data.

[1625] Specifically, it uses an image processing library (e.g., OpenCV) to apply filters to reduce noise in the image and adjust the contrast and brightness appropriately.

[1626] Step 3:

[1627] Sending data

[1628] The terminal transmits the preprocessed video data to the server.

[1629] Input: Preprocessed video data.

[1630] Output: The data sent to the server.

[1631] Specifically, the device sends preprocessed data to the server in real time through a specific API endpoint.

[1632] Step 4:

[1633] Traffic volume assessment and emotion recognition

[1634] The server receives the preprocessed video data and performs traffic analysis and emotion recognition.

[1635] Input: Preprocessed video data.

[1636] Output: Traffic data and sentiment data.

[1637] Specifically, the server uses image recognition algorithms to detect and count the number and movement of vehicles, and an emotion recognition engine (e.g., facial expression analysis software) to analyze passengers' emotions and stress levels from their facial expressions in the video data.

[1638] Step 5:

[1639] Integrated evaluation

[1640] The server comprehensively evaluates the traffic volume data and emotion data.

[1641] Input: Traffic data and sentiment data.

[1642] Output: Overall evaluation result.

[1643] Specifically, the server uses the analysis results, predefined rules or generative AI models to evaluate the relationship between traffic conditions and stress levels, and determines the necessary adjustments.

[1644] Step 6:

[1645] Execution of signal control algorithms

[1646] The server executes the signal control algorithm based on the comprehensive evaluation result.

[1647] Input: Overall evaluation result.

[1648] Output: Signal control instructions.

[1649] Specifically, the server runs an algorithm to calculate the optimal signal duration and determine, for example, the green light extension time.

[1650] Step 7:

[1651] Sending signal control instructions

[1652] The server transmits the calculated signal control instructions to the terminal.

[1653] Input: Signal control instructions.

[1654] Output: Signal control instructions sent to the terminal.

[1655] As a specific operation, the server transmits instruction data including a command to adjust the signal timing to the terminal.

[1656] Step 8:

[1657] Traffic light adjustments

[1658] The terminal adjusts the signaling device based on the received signal control instructions.

[1659] Input: Signal control instructions.

[1660] Output: Traffic light operating status after adjustment.

[1661] Specifically, the terminal sends instructions to the traffic light control unit to change its timing.

[1662] Step 9:

[1663] Collecting and sending feedback

[1664] The terminal again collects traffic data and emotion data for the new signal pattern and transmits them to the server.

[1665] Input: Traffic data and emotion data with adjusted signal patterns.

[1666] Output: Feedback data sent to the server.

[1667] As a specific operation, the terminal collects data in the same manner as in the first step and transmits it to the server as feedback data.

[1668] Step 10:

[1669] Feedback analysis and algorithm improvement

[1670] The server analyzes the feedback data and evaluates the effectiveness of the signal control algorithm.

[1671] Input: Feedback data.

[1672] Output: Algorithm improvements.

[1673] Specifically, the server compares the collected data with the results of the algorithm and adjusts and improves the control algorithm as needed.

[1674] The above are the processing steps of the system for implementing the present invention.

[1675] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1676] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1677] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1678] [Fourth embodiment]

[1679] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1680] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1681] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1682] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1683] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1684] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1685] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1686] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1687] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1688] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1689] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1690] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1691] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1692] The present invention relates to a system for mitigating traffic congestion by utilizing cameras installed at traffic lights. Specific embodiments of the system are described below.

[1693] (Data collection and preprocessing)

[1694] Terminal

[1695] The terminal is responsible for acquiring video data in real time from cameras installed at traffic lights. The acquired video data undergoes pre-processing to analyze traffic volume and vehicle congestion. Specifically, the device performs noise removal and contrast adjustment on the video data.

[1696] (Data analysis and traffic volume evaluation)

[1697] server

[1698] The server analyzes pre-processed video data sent from the devices to assess traffic volume. The analysis is performed using image recognition algorithms to detect and count the number of vehicles and their movements. This data is used to assess the traffic situation at specific intersections and road sections in real time.

[1699] (Execution of signal control algorithm)

[1700] server

[1701] The server then executes a traffic light control algorithm based on the results of its assessment of traffic volume and congestion. This algorithm calculates the optimal signal lighting time for each intersection. For example, dynamic adjustments are made, such as extending the green light time at intersections where vehicles are congested.

[1702] (Generation and transmission of signal control instructions)

[1703] server

[1704] The server generates signal control instructions based on the calculated optimal lighting time. These instructions are instantly sent to each traffic light, and the lighting time of the traffic light is adjusted in real time. For example, an instruction to extend the green light at intersection A by 30 seconds is sent from the server to the terminal.

[1705] (Traffic signal adjustment)

[1706] Terminal

[1707] The terminal receives the signal control instruction sent from the server and automatically adjusts the lighting time of the traffic light, for example, by extending the green light time by 30 seconds.

[1708] (Feedback collection and analysis)

[1709] Terminal

[1710] After adjusting the traffic lights, the terminal again collects traffic data based on the new signal pattern and transmits it to the server.

[1711] server

[1712] The server analyzes this feedback data and continuously improves the signal control algorithm, allowing it to flexibly respond to changes in traffic conditions and optimize the entire system.

[1713] (Example)

[1714] Example 1: Reducing congestion at intersection A

[1715] 1. Data Collection

[1716] The terminal acquires video data from a camera installed at intersection A.

[1717] Preprocess the video data and send it to the server.

[1718] 2. Data Analysis

[1719] The server analyzes the received video data using an image recognition algorithm and counts the number of vehicles at intersection A.

[1720] Based on the analysis results, the traffic volume and congestion situation at intersection A are evaluated.

[1721] 3. Execution of signal control algorithm

[1722] The server executes a traffic light control algorithm to extend the green light duration at intersection A by 30 seconds.

[1723] 4. Transmission of signal control instructions

[1724] The server transmits the generated signal control instruction to the terminal.

[1725] 5. Traffic light adjustments

[1726] The device adjusts and controls the traffic light to extend the green light duration by 30 seconds.

[1727] 6. Feedback Collection and Analysis

[1728] The terminal again collects traffic data based on the new signal pattern and transmits it to the server.

[1729] The server analyzes the feedback data and improves the traffic light control algorithm.

[1730] In this way, traffic flow can be optimized and congestion can be alleviated by dynamically adjusting the illumination duration of traffic lights based on the analysis of video data acquired by the camera.

[1731] The processing flow will be explained below.

[1732] Step 1:

[1733] Terminal

[1734] The device acquires video data in real time from cameras installed at traffic lights. Specifically, it acquires video within the camera's field of view and extracts frames at regular intervals. This video data is used as basic data for analyzing traffic volume and vehicle congestion.

[1735] Step 2:

[1736] Terminal

[1737] The device performs preprocessing on the captured video data, including noise reduction, contrast adjustment, and resolution optimization, to improve analysis accuracy and reduce the risk of misidentification.

[1738] Step 3:

[1739] Terminal

[1740] The pre-processed video data is sent to the server, which converts the data packets into an appropriate format and transfers them to the server in real time over the network.

[1741] Step 4:

[1742] server

[1743] The server receives pre-processed video data sent from the device, ready for immediate analysis.

[1744] Step 5:

[1745] server

[1746] The server analyzes the received video data using an image recognition algorithm. Specifically, it counts the number of vehicles in each frame and measures their movement speed and stationary time. Based on the results of this analysis, it evaluates traffic volume and stationary conditions.

[1747] Step 6:

[1748] server

[1749] Based on the results of the traffic volume and congestion assessment, the system executes a traffic light control algorithm that calculates the optimal lighting times (red, green, and yellow) for each intersection. For example, this may include extending the green light time on roads with heavy traffic.

[1750] Step 7:

[1751] server

[1752] Based on the calculated optimal lighting time, specific traffic light control instructions are generated, and the server converts these instructions into data packets for transmission to the target traffic light.

[1753] Step 8:

[1754] server

[1755] The server transmits the signal control instructions, including the traffic light control parameters (lighting time, sequence, etc.), to the terminal via the network.

[1756] Step 9:

[1757] Terminal

[1758] The terminal receives the signal control instructions sent from the server. This is applied to the internal control system to adjust the lighting time of the traffic light. Specifically, it controls the time the green light stays on by 30 seconds, for example.

[1759] Step 10:

[1760] Terminal

[1761] The operation of the adjusted traffic lights is monitored and traffic data is collected again based on the new signal pattern, which is then sent as feedback to the server.

[1762] Step 11:

[1763] server

[1764] The server receives feedback data from the devices and evaluates the effectiveness of the signal control algorithm. Based on the analysis results, the algorithm is improved and used for future adjustments.

[1765] Step 12:

[1766] User

[1767] Users can check current traffic conditions and traffic light change information through a dedicated app or website, and user feedback will also be incorporated into the system for further optimization.

[1768] Example 1

[1769] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1770] Conventional traffic systems face the challenge of effectively alleviating traffic congestion in real time. Specifically, they lack the ability to properly grasp ongoing traffic conditions and dynamically adjust traffic light illumination times accordingly. As a result, traffic flow becomes unsmooth and congestion occurs.

[1771] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1772] In this invention, the server includes means for collecting video data acquired by a camera, means for pre-processing the video data acquired by the camera, means for analyzing the collected video data to evaluate traffic volume, means for executing a signal control algorithm based on the evaluation results, means for generating and transmitting signal control instructions to the traffic lights, means for adjusting the illumination duration of the traffic lights based on the signal control instructions, and means for collecting and analyzing feedback data. This makes it possible to monitor traffic conditions in real time and perform appropriate traffic light control. Furthermore, the collected feedback data can be used to continuously improve the signal control algorithm, thereby effectively alleviating traffic congestion.

[1773] "Video data acquired by a camera" refers to video information captured in real time by a camera installed at an intersection or road section.

[1774] "Preprocessing" refers to a data cleaning process, such as noise removal and contrast adjustment, that is carried out before analyzing the video data acquired by the camera.

[1775] "Analysis" is a method of evaluating traffic volume and vehicle congestion based on preprocessed video data, and is carried out using image recognition algorithms.

[1776] "Evaluating traffic volume" means detecting the number and movement of vehicles at a particular intersection or road section from the analyzed data and determining the traffic situation.

[1777] A "signal control algorithm" refers to the calculation procedures and rules for calculating the optimal lighting time of a traffic light based on the results of traffic volume evaluation.

[1778] A "signal control instruction" is a command that indicates the lighting time of a specific traffic light, generated by a signal control algorithm.

[1779] "Adjusting the lighting time of a traffic light" means changing the display time of a green light, red light, etc. of a traffic light based on a signal control instruction.

[1780] "Feedback data" refers to new traffic data that is re-collected after adjusting traffic light durations and is used to continuously improve the algorithm.

[1781] A "traffic light" is a display device installed at an intersection or road section to control the flow of traffic.

[1782] MODE FOR CARRYING OUT THE INVENTION

[1783] The present invention is a system that uses cameras installed at traffic lights to alleviate traffic congestion. A specific embodiment of this system is described below.

[1784] Data collection and preprocessing

[1785] Terminal

[1786] The device acquires video data in real time from cameras installed at traffic lights. The hardware used includes commercially available surveillance cameras and a small computer for data collection. This video data undergoes preprocessing before analyzing traffic volume and vehicle congestion. Specifically, libraries such as OpenCV are used to remove noise and adjust contrast.

[1787] Data analysis and traffic volume evaluation

[1788] server

[1789] The server receives and analyzes the preprocessed video data sent from the device. This analysis uses image recognition algorithms such as YOLO (You Only Look Once) to detect and count vehicles in the video in real time. Based on the analysis results, the server evaluates the traffic volume and congestion status at specific intersections and road sections. The evaluation results are temporarily stored in a database.

[1790] Execution of signal control algorithms

[1791] server

[1792] The server then executes a traffic light control algorithm based on the results of its assessment of traffic volume and congestion. This algorithm calculates the optimal signal lighting time for each intersection. For example, at intersections where traffic volume has increased and congestion has occurred, dynamic adjustments are made, such as extending the green light time by 30 seconds.

[1793] Generation and transmission of signal control instructions

[1794] server

[1795] The server generates traffic light control instructions based on the calculated optimal lighting time. These instructions are instantly sent to each traffic light. Communication protocols such as MQTT are used to send specific instructions, such as "Extend the green light at intersection A by 30 seconds," to the terminal in JSON format.

[1796] Traffic light adjustments

[1797] Terminal

[1798] The terminal receives the signal control instructions sent from the server and automatically adjusts the traffic lights based on the instructions. A PLC (Programmable Logic Controller) is used to control the traffic lights, and the green light duration is extended as specified.

[1799] Collecting and analyzing feedback

[1800] Terminal

[1801] After adjusting the traffic lights, the device collects traffic data again based on the new signal patterns and sends it to the server. This data, also in the form of video data, is again pre-processed and analyzed.

[1802] server

[1803] The server analyzes this feedback data and adjusts parameters to continuously improve the signal control algorithm, allowing it to flexibly respond to changing traffic conditions.

[1804] Specific examples

[1805] As a specific example, the case of easing traffic congestion at intersection A will be shown below.

[1806] 1. Data Collection

[1807] The terminal acquires video data from a camera installed at intersection A, preprocesses it, and sends it to the server.

[1808] 2. Data Analysis

[1809] The server analyzes the received video data using an image recognition algorithm and counts the number of vehicles at intersection A.

[1810] Based on the analysis results, the traffic volume and congestion situation at intersection A are evaluated.

[1811] 3. Execution of signal control algorithm

[1812] The server executes a traffic light control algorithm to extend the green light duration at intersection A by 30 seconds.

[1813] 4. Transmission of signal control instructions

[1814] The server transmits the generated signal control instruction to the terminal.

[1815] 5. Traffic light adjustments

[1816] The device adjusts and controls the traffic light to extend the green light duration by 30 seconds.

[1817] 6. Feedback Collection and Analysis

[1818] The terminal again collects traffic data based on the new signal pattern and transmits it to the server.

[1819] The server analyzes the feedback data and improves the traffic light control algorithm.

[1820] Prompt Sentence Examples

[1821] "How will this system alleviate traffic congestion? What are the specific steps and technologies used?"

[1822] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1823] Step 1:

[1824] Data collection

[1825] Terminal

[1826] The terminal acquires video data in real time from cameras installed at traffic lights. Specifically, commercially available surveillance cameras capture video at a set frame rate. The input data is the captured raw video data. This data is preprocessed in the next step.

[1827] Step 2:

[1828] Video data preprocessing

[1829] Terminal

[1830] The device preprocesses the acquired video data. Specifically, it uses the OpenCV library to remove noise and adjust the contrast of the video data. The input data is raw video data. The output data is preprocessed video data with noise removed and contrast adjusted. This enables subsequent image analysis to be performed with high accuracy.

[1831] Step 3:

[1832] Data analysis

[1833] server

[1834] The server receives preprocessed video data sent from the device. The server analyzes this data using image recognition algorithms such as YOLO. The input data is the preprocessed video data, and the output data is the number and location of detected vehicles. Based on the analyzed data, traffic volume and congestion at specific intersections and road sections are evaluated.

[1835] Step 4:

[1836] Traffic volume evaluation

[1837] server

[1838] The server evaluates traffic volume based on the analysis results. Specifically, it calculates the actual traffic volume and congestion status at specific intersections and road sections based on the number of vehicles and location information obtained from the analysis. The input data is the number of vehicles and location information from the analysis results, and the output data is the evaluated traffic volume and congestion status. This evaluation result is temporarily stored in a database.

[1839] Step 5:

[1840] Execution of signal control algorithms

[1841] server

[1842] The server executes a signal control algorithm based on the evaluation results of traffic volume and congestion status. The input data is the traffic volume and congestion status of the evaluation results, and the output data is the optimal signal lighting time for each intersection. Specifically, at intersections where vehicles are congested, dynamic adjustments are made, such as extending the green light time.

[1843] Step 6:

[1844] Generation and transmission of signal control instructions

[1845] server

[1846] The server generates traffic light control instructions based on the calculated optimal lighting time. These instructions are instantly sent to each traffic light using a communication protocol such as MQTT. The input data is the optimized lighting time, and the output data is the specific control instructions for the traffic light. For example, this could include an instruction such as "Extend the green light at intersection A by 30 seconds."

[1847] Step 7:

[1848] Traffic light adjustments

[1849] Terminal

[1850] The terminal receives signal control instructions sent from the server. Based on the received instructions, the terminal controls the traffic light. The input data is the signal control instructions, and the output operation is to adjust the traffic light's lighting time. Specifically, the green light's lighting time is extended as specified. A PLC (Programmable Logic Controller) is used to control the traffic light.

[1851] Step 8:

[1852] Feedback data collection

[1853] Terminal

[1854] After the traffic light is adjusted, the terminal collects traffic data again according to the new signal pattern, and the input data is the video data under the new signal pattern, and the output data is the preprocessed video data.

[1855] Step 9:

[1856] Analysis of feedback data and algorithm improvement

[1857] server

[1858] The server receives and analyzes feedback data sent from the terminal. The input data is preprocessed feedback video data, and the output data is the analysis results of the number of vehicles and their location information. Based on these results, the signal control algorithm is continuously improved to achieve adaptive signal control.

[1859] (Application example 1)

[1860] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1861] Conventional traffic systems have difficulty in controlling signals in a fast and flexible manner to respond to fluctuations in traffic volume, and congestion has not been sufficiently alleviated. Furthermore, conventional navigation systems for autonomous vehicles are unable to reflect real-time traffic information and may not be able to provide optimal driving routes.

[1862] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1863] In this invention, the server includes means for collecting video data acquired by cameras, means for analyzing the collected video data to evaluate traffic volume, means for dynamically adjusting the illumination time of traffic signals based on the evaluation results, and means for providing traffic information in real time and optimizing vehicle routes, thereby enabling dynamic adjustment of traffic volume and optimal navigation.

[1864] A "camera" is a device installed at a traffic light that captures images of road traffic conditions in real time.

[1865] "Video data" refers to digital video recordings of traffic conditions captured by a camera.

[1866] "Collection means" is a function that collects video data acquired by a camera in real time into a database or server.

[1867] The "analysis means" is a function that uses collected video data to evaluate traffic volume and identify the number of vehicles and congestion conditions.

[1868] "Evaluation results" refer to data indicating traffic volume and congestion conditions obtained by the analysis means.

[1869] The "signal lighting time adjustment means" is a function for dynamically changing the lighting time of the green or red signal of a traffic light based on the evaluation result.

[1870] "Signal control instructions" are control information for instructing the lighting time and operating state of a traffic light.

[1871] A "traffic light" is an optical signal device installed at road intersections and other locations to control traffic flow.

[1872] "Real-time traffic information provision means" is a function that provides the current optimal driving route based on traffic data.

[1873] The "driving route optimization means" is a function that navigates the vehicle along the optimal route based on real-time traffic information.

[1874] A system for carrying out the present invention comprises a camera, a server, and a terminal. Specific embodiments will be described in detail below.

[1875] Terminal

[1876] First, cameras installed at traffic lights capture real-time video data of road traffic conditions. The captured video data undergoes preprocessing such as noise removal and contrast adjustment on the device. After preprocessing, the data is sent to the server.

[1877] server

[1878] The server uses an image recognition algorithm to evaluate traffic volume based on preprocessed video data sent from the device. Specifically, it counts the number of vehicles and detects their movements. Based on the analysis results, it evaluates traffic volume and congestion conditions.

[1879] By running the signal control algorithm on the server, the optimal signal lighting time for each intersection is calculated. For example, dynamic adjustments are made, such as extending the green light time at intersections where congestion is occurring. The calculated optimal signal control instructions are sent to each traffic light in real time.

[1880] The server also provides real-time navigation information based on traffic information, allowing it to present optimal driving routes to autonomous vehicles.

[1881] Autonomous vehicles

[1882] The autonomous vehicle receives real-time traffic information sent from the server and optimizes its route, for example by changing its route to coincide with the time when the green light remains green at a particular intersection for a long period of time.

[1883] Specific examples

[1884] Video data obtained by the camera at intersection A is preprocessed on the terminal and sent to the server. The server uses an analysis algorithm to count the number of vehicles and evaluate traffic volume. If congestion occurs, a signal control algorithm is executed to generate signal control instructions that dynamically extend the green light duration and send them to the traffic lights. Real-time traffic information is also sent to the navigation system, which displays the optimal driving route.

[1885] Prompt Sentence Examples

[1886] "I want to build an application that analyzes video data from a traffic light camera at intersection A and suggests the optimal driving route based on traffic congestion. Please tell me the code to evaluate the current traffic volume, obtain signal control information based on that data, and calculate the optimal route."

[1887] In this way, traffic lights can be dynamically adjusted based on the analysis of video data acquired by cameras, providing autonomous vehicles with optimal routes to optimize traffic flow and alleviate congestion.

[1888] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1889] Step 1: Acquire video data with the camera

[1890] Cameras installed at traffic lights capture traffic conditions in real time. The video data obtained from the cameras is digital footage that captures the movement of vehicles on the road. Specifically, the video captured by the cameras is sent to a terminal in streaming format. The input of this step is the camera image itself, and the output is the unprocessed video data.

[1891] Step 2: Preprocess the video data

[1892] The device receives video data captured by the camera and performs preprocessing such as noise reduction and contrast adjustment. Specifically, the video data is converted to grayscale using a library such as OpenCV, and noise is removed by applying a Gaussian filter. The input of this step is the raw video data, and the output is the preprocessed video data.

[1893] Step 3: Send the preprocessed data to the server

[1894] The terminal sends the preprocessed video data to the server. In this step, data is transferred to the server using a communication protocol such as an HTTP request. The input is the preprocessed video data, and the output is the data reception on the server.

[1895] Step 4: Analyze video data to assess traffic volume

[1896] The server analyzes the received preprocessed video data and evaluates traffic volume. This process uses an image recognition algorithm to count the number of vehicles and identify their movements. For example, it uses YOLO or OpenCV vehicle detection models. The input is the preprocessed video data, and the output is an evaluation result showing the number of vehicles and congestion status.

[1897] Step 5: Run the signal control algorithm

[1898] The server executes a signal control algorithm based on the evaluation results. Specifically, it dynamically adjusts signals, such as extending the green light duration at intersections with congestion. The input is evaluation data based on vehicle detection results, and the output is optimal signal control instructions.

[1899] Step 6: Send signal control instructions to the traffic light

[1900] The signal control instructions generated by the server are sent to the terminal to control the traffic lights. Here, signal control information is sent to the traffic lights using a communication protocol. The input is the signal control instructions, and the output is the transfer of instructions to the traffic lights.

[1901] Step 7: Providing real-time traffic information

[1902] The server provides real-time traffic information to autonomous vehicles based on traffic volume evaluation results and signal control information. Specifically, it sends route optimization information via API. The input is the latest traffic data, and the output is route optimization information.

[1903] Step 8: Optimize the autonomous vehicle's route

[1904] An autonomous vehicle optimizes its driving route based on traffic information received from a server. Here, the vehicle's navigation system analyzes the information from the server and presents the optimal route. The input is the optimized driving route information, and the output is the optimal route the vehicle should follow.

[1905] By going through the above processing steps, it is possible to optimize traffic flow and alleviate congestion.

[1906] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1907] This invention relates to a system that alleviates traffic congestion by utilizing cameras installed at traffic lights and combining them with an emotion engine that recognizes the emotions of users. Specific embodiments of this system are described below.

[1908] (Data collection and preprocessing)

[1909] Terminal

[1910] The device is responsible for acquiring video data in real time from cameras installed at traffic lights. The acquired video data undergoes pre-processing to analyze traffic volume and vehicle congestion. Specifically, noise removal and contrast adjustment are performed on the video data. The camera also captures the facial expressions of the vehicle's users, which are used as emotional data.

[1911] (Data analysis and traffic volume evaluation)

[1912] server

[1913] The server receives preprocessed video data and emotion data sent from the device. Using the received data, it analyzes traffic volume and then analyzes the user's emotions using an emotion engine. Traffic volume analysis uses an image recognition algorithm to detect and count the number of vehicles and their movements. Meanwhile, the emotion engine measures the user's stress level and irritation from their facial expressions.

[1914] (traffic volume and user sentiment evaluation)

[1915] server

[1916] The server comprehensively evaluates traffic volume and user emotion data, and determines that traffic conditions are deteriorating or the user's stress level is high, and prioritizes traffic light adjustment.

[1917] (Execution of signal control algorithm)

[1918] server

[1919] The server then executes a traffic light control algorithm based on the evaluation results. The algorithm calculates the optimal traffic light duration for each intersection. For example, if traffic volume is heavy and the user is feeling stressed, the algorithm dynamically adjusts the duration of the green light, such as extending it.

[1920] (Generation and transmission of signal control instructions)

[1921] server

[1922] The server generates signal control instructions based on the calculated optimal lighting time. These instructions are instantly sent to each traffic light, and the lighting time of the traffic light is adjusted in real time. For example, an instruction to extend the green light at intersection A by 30 seconds is sent from the server to the terminal.

[1923] (Traffic signal adjustment)

[1924] Terminal

[1925] The device receives the signal control instructions sent from the server and automatically adjusts the traffic light duration, for example, extending the green light duration by 30 seconds.

[1926] (Feedback collection and analysis)

[1927] Terminal

[1928] After the traffic light is adjusted, the terminal collects traffic data and user emotion data according to the new traffic light pattern again and transmits them to the server.

[1929] server

[1930] The server analyzes this feedback data and evaluates the effectiveness of the signal control algorithm. Based on the analysis results, the algorithm is improved and used for future adjustments.

[1931] (Example)

[1932] Example 1: Reducing congestion at intersection A

[1933] 1. Data Collection

[1934] The terminal acquires video data and emotion data from a camera installed at intersection A.

[1935] Preprocess the video data and send it to the server.

[1936] 2. Data Analysis

[1937] The server analyzes the received video data using an image recognition algorithm and counts the number of vehicles at intersection A.

[1938] An emotion engine is used to analyze the user's emotion data and assess their stress level.

[1939] 3. Evaluation of traffic volume and emotion data

[1940] The server comprehensively evaluates the traffic volume at intersection A and the user's emotional data to determine the traffic situation and the user's stress level.

[1941] 4. Execution of signal control algorithm

[1942] The server executes a traffic light control algorithm to extend the green light duration at intersection A by 30 seconds.

[1943] 5. Transmission of signal control instructions

[1944] The server transmits the generated signal control instruction to the terminal.

[1945] 6. Traffic light adjustment

[1946] The device adjusts and controls the traffic light to extend the green light duration by 30 seconds.

[1947] 7. Feedback Collection and Analysis

[1948] The terminal again collects traffic data and user emotion data based on the new signal pattern and transmits them to the server.

[1949] The server analyzes the feedback data and improves the traffic light control algorithm.

[1950] In this way, traffic lights can be dynamically adjusted based on the video data acquired by the camera and the user's emotional data obtained from the emotion engine, thereby optimizing traffic flow, easing congestion, and reducing user stress.

[1951] The processing flow will be explained below.

[1952] Step 1:

[1953] Terminal

[1954] The device acquires real-time video data from cameras installed at traffic lights. The cameras are high-resolution video cameras that capture the entire intersection and extract frames at regular intervals, providing data for monitoring temporary congestion and fluctuations in traffic volume.

[1955] Step 2:

[1956] Terminal

[1957] Preprocessing is performed on the video data captured by the camera. Specifically, processes such as noise removal, contrast adjustment, and resolution optimization are performed. This improves the quality of the video data and increases the accuracy of analysis. In addition, if the user is in the vehicle, their facial expressions are captured and extracted as emotion data.

[1958] Step 3:

[1959] Terminal

[1960] The preprocessed video data and emotion data are sent to the server. The data is set to be sent in real time via the network, allowing the server to constantly monitor the latest traffic conditions and the user's emotional state.

[1961] Step 4:

[1962] server

[1963] The server receives the preprocessed video data and emotion data sent from the device. It then applies an image recognition algorithm to analyze the received video data. Specifically, the analysis counts the number of vehicles in each frame and measures their movement and dwell time.

[1964] Step 5:

[1965] server

[1966] The server uses an emotion engine to analyze the user's emotional data. Specifically, it uses facial expression recognition technology to evaluate the user's stress level and irritation. This evaluation result is combined with traffic volume data to help understand the overall traffic situation.

[1967] Step 6:

[1968] server

[1969] The system evaluates traffic volume and user emotional data in an integrated manner. For example, when traffic volume is heavy and vehicles are backed up, and when a high stress level is detected from the user's emotional data, it determines that it is necessary to raise the priority of traffic signal control.

[1970] Step 7:

[1971] server

[1972] Based on the evaluation results, a traffic light control algorithm is executed. The algorithm calculates the optimal traffic light duration for each intersection. Dynamic adjustments are made, such as extending the green light duration at intersection A by 30 seconds.

[1973] Step 8:

[1974] server

[1975] It generates signal control instructions, including the specific time to extend the green light and the priority of roads, and converts these instructions into data packets for transmission to the traffic light controller.

[1976] Step 9:

[1977] server

[1978] A signal control instruction is sent to the terminal. The instruction is sent immediately via the network, and the server receives a response confirming receipt, thereby confirming that the instruction was transmitted correctly.

[1979] Step 10:

[1980] Terminal

[1981] The terminal receives the signal control instructions sent from the server. It applies these to the internal control system and sets the lighting time and sequence of the traffic lights. Specifically, it controls the signal by extending the green light time by 30 seconds, for example.

[1982] Step 11:

[1983] Terminal

[1984] The operation of the traffic lights after adjustment is monitored. Traffic data and user emotion data based on the new traffic light patterns are collected again and sent to the server.

[1985] Step 12:

[1986] server

[1987] The server receives feedback data from the devices and evaluates the effectiveness of the signal control algorithm. Based on the analysis results, the algorithm is improved and used for future adjustments.

[1988] Step 13:

[1989] User

[1990] Users can check current traffic conditions and traffic light change information through a dedicated app or website, and receive advice and notifications based on their stress and irritation levels to help them drive with peace of mind.

[1991] Example 2

[1992] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1993] Conventional traffic signal control systems are unable to respond quickly to fluctuations in traffic volume, making it difficult to effectively alleviate traffic congestion. Furthermore, because they do not take into account the user's stress or emotional state, drivers experience increased stress and the risk of traffic accidents increases as they spend longer at intersections. The present invention aims to solve these problems by analyzing traffic volume and the user's emotional state in real time and dynamically adjusting signal control.

[1994] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1995] In this invention, the server includes means for collecting video data captured by cameras installed at traffic lights in real time, means for performing noise reduction and contrast adjustment on the collected video data, means for capturing a user's facial expressions to collect emotion data, means for analyzing the preprocessed video data and emotion data to evaluate traffic volume and the user's emotional state, means for executing a signal control algorithm based on the evaluation results, means for generating and transmitting signal control instructions to the traffic lights, means for re-collecting new traffic data and emotion data based on the adjusted signal pattern, and means for analyzing the feedback data to improve the signal control algorithm, thereby enabling fast and effective signal control based on fluctuations in traffic volume and the user's emotional state.

[1996] A "traffic light" is an optical signaling device installed to control the flow of road traffic.

[1997] A "camera" is a photographing device for acquiring video data.

[1998] "Video Data" refers to visual information captured by a camera.

[1999] "Real-time" means that data acquisition and processing occur almost simultaneously.

[2000] "Noise reduction" is the process of removing unnecessary information and errors from collected video data.

[2001] "Contrast adjustment" is a process for appropriately adjusting the contrast between light and dark in the video data.

[2002] "Facial expressions" express emotions based on the shape and movement of the user's face.

[2003] "Emotion data" is information that represents the user's emotional state.

[2004] "Analysis" is the process of examining data in detail to clarify its content and meaning.

[2005] "Traffic volume" refers to the number and flow of vehicles in a particular time unit.

[2006] "Evaluation" is the act of judging a situation or condition based on data.

[2007] A "traffic light control algorithm" is a calculation method for determining the lighting time and control pattern of traffic lights.

[2008] A "signal control instruction" is a specific command for controlling the operation of a traffic light.

[2009] "Feedback data" is new data collected after traffic light adjustments.

[2010] "User" refers to the driver or passenger of a vehicle that uses this system.

[2011] This invention relates to a system that utilizes cameras installed at traffic lights and combines them with an emotion engine that recognizes the emotions of users to alleviate traffic congestion. Specific embodiments of this system are described below.

[2012] (System Configuration)

[2013] Hardware

[2014] Cameras installed at traffic lights:

[2015] The camera captures the traffic conditions at the intersection in real time, collecting video data, and also captures the user's facial expressions to obtain emotional data.

[2016] Device:

[2017] The terminal is installed at the traffic light, receives the video data from the camera, performs noise reduction and contrast adjustment, and sends the pre-processed data to the server.

[2018] server:

[2019] The server receives and analyzes the video and emotion data sent from the device. Based on the results of the analysis, it executes a traffic light control algorithm and generates instructions for the traffic light.

[2020] (software)

[2021] Image Recognition Algorithm

[2022] The server uses image recognition algorithms to analyze the number and movement of vehicles from pre-processed video data, allowing for accurate assessment of traffic volume.

[2023] Emotion Engine

[2024] The server uses an emotion engine to analyze the user's facial expression data and assess their stress level and irritation, thereby taking into account not only the traffic situation but also the user's emotional state.

[2025] Signal Control Algorithm

[2026] The server runs a traffic light control algorithm based on the results of traffic volume analysis and emotion analysis, which calculates the optimal signal lighting time for each intersection and dynamically adjusts the traffic lights.

[2027] Feedback Mechanism

[2028] After adjusting the traffic lights, the device will again collect traffic and emotion data based on the new signal patterns and send it to the server, which will analyze this feedback data, evaluate the effectiveness of the signal control algorithm, and make improvements.

[2029] (Example)

[2030] Example 1: Reducing congestion at intersection A

[2031] 1. Data Collection:

[2032] The terminal acquires video data and emotion data from a camera installed at intersection A. The acquired data is preprocessed and sent to the server.

[2033] 2. Data Analysis:

[2034] The server analyzes the received video data using an image recognition algorithm and counts the number of vehicles at intersection A. It uses an emotion engine to analyze the user's emotion data and evaluate their stress level.

[2035] 3. Traffic and Sentiment Data Evaluation:

[2036] The server comprehensively evaluates the traffic volume at intersection A and the user's emotional data to determine the traffic situation and the user's stress level.

[2037] 4. Execution of signal control algorithm:

[2038] The server executes a traffic light control algorithm to extend the green light duration at intersection A by 30 seconds.

[2039] 5. Sending signal control instructions:

[2040] The server transmits the generated signal control instruction to the terminal.

[2041] 6. Traffic light adjustment:

[2042] The device adjusts and controls the traffic light to extend the green light duration by 30 seconds.

[2043] 7. Feedback Collection and Analysis:

[2044] The terminal again collects traffic data and user emotion data based on the new signal pattern and transmits them to the server, which analyzes the feedback data and improves the signal control algorithm.

[2045] Prompt Sentence Examples

[2046] "Please explain a system that uses cameras and an emotion engine to collect and analyze traffic volume and user emotion data, and dynamically adjusts traffic light durations to alleviate congestion at intersections."

[2047] In this way, traffic lights can be dynamically adjusted based on the video data acquired by the camera and the user's emotional data obtained from the emotion engine, thereby optimizing traffic flow, easing congestion, and reducing user stress.

[2048] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2049] Step 1:

[2050] Data collection

[2051] The terminal acquires video data in real time from cameras installed at traffic lights.

[2052] Input: Video data from a camera installed at a traffic light.

[2053] Data processing and calculation: Capture of video data.

[2054] Output: Captured video data.

[2055] Step 2:

[2056] Video data preprocessing

[2057] The device performs noise removal and contrast adjustment on the acquired video data.

[2058] Input: Captured video data.

[2059] Data processing and calculation: noise removal and contrast adjustment.

[2060] Output: Pre-processed video data. Specific operations include applying noise filters and histogram equalization.

[2061] Step 3:

[2062] Collecting Emotional Data

[2063] The device's camera captures the facial expressions of the user riding in the vehicle and uses them as emotional data.

[2064] Input: Video data containing the user's facial expressions.

[2065] Data processing and calculation: Image analysis for facial expression recognition.

[2066] Output: User's emotional data. Specifically, the stress level and irritation level are measured from the user's facial expressions.

[2067] Step 4:

[2068] Video data analysis

[2069] The server receives the pre-processed video data sent from the device and analyzes it using image recognition algorithms.

[2070] Input: Preprocessed video data.

[2071] Data processing and calculation: The number and movement of vehicles are detected and counted using an image recognition algorithm.

[2072] Output: Parsed traffic data. Specific actions include vehicle identification and counting.

[2073] Step 5:

[2074] Emotional Data Analysis

[2075] The server uses an emotion engine to analyze the user's facial expression data and assess their stress level and irritability.

[2076] Input: User emotion data.

[2077] Data processing and calculation: Emotion analysis using an emotion engine.

[2078] Output: Analyzed emotional state data. Specifically, facial expression recognition technology is used to detect the user's micro-expressions and estimate their emotional state.

[2079] Step 6:

[2080] Integrated evaluation of traffic volume and emotion data

[2081] The server performs an integrated evaluation based on the analyzed traffic volume data and emotion data.

[2082] Input: Traffic data and emotional state data.

[2083] Data processing and calculation: Integrated evaluation of data.

[2084] Output: Integrated evaluation results. Specific actions include determining whether traffic conditions are worsening or the user's stress level is high.

[2085] Step 7:

[2086] Execution of signal control algorithms

[2087] The server executes the signal control algorithm based on the results of the integrated evaluation.

[2088] Input: Integrated evaluation results.

[2089] Data processing and calculation: Application of signal control algorithms.

[2090] Output: Optimal signal control instructions. Specifically, the optimal green and red light durations at each intersection are calculated.

[2091] Step 8:

[2092] Generation of signal control instructions

[2093] The server generates signal control instructions based on the results of the signal control algorithm.

[2094] Input: The result of the signal control algorithm.

[2095] Data processing and calculation: Generation of signal control instructions.

[2096] Output: Specific signal control instructions. Specifically, instructions such as "Extend the green light at intersection A by 30 seconds" are generated.

[2097] Step 9:

[2098] Sending signal control instructions

[2099] The server transmits the generated signal control instruction to the terminal.

[2100] Input: Signal control instructions.

[2101] Data processing and calculation: Sending signal control instructions.

[2102] Output: Signal control instructions sent to the terminal.

[2103] Step 10:

[2104] Traffic light adjustments

[2105] The terminal adjusts the lighting time of the traffic light based on the signal control instruction received from the server.

[2106] Input: Signal control instruction from the server.

[2107] Data processing and calculation: Traffic light control.

[2108] Output: Adjusted traffic lights. Specifically, the green light duration is extended by 30 seconds.

[2109] Step 11:

[2110] Collecting new data

[2111] After the traffic light is adjusted, the terminal collects traffic data and emotion data again according to the new traffic light pattern.

[2112] Input: Adjusted traffic data and sentiment data.

[2113] Data processing and calculation: Data collection.

[2114] Output: Recollected traffic data and emotion data.

[2115] Step 12:

[2116] Analysis of feedback data

[2117] The server again analyzes the received feedback data and evaluates the effectiveness of the signal control algorithm.

[2118] Input: Recollected traffic data and emotion data.

[2119] Data processing and calculation: Analysis of feedback data.

[2120] Output: Analysis results, specifically evaluating the effectiveness of signal conditioning and modifying algorithm parameters if necessary.

[2121] Through the above steps, traffic flow can be optimized, congestion can be alleviated, and user stress can be reduced by dynamically adjusting the illumination times of traffic lights based on the video data acquired by the camera and the user's emotional data obtained from the emotion engine.

[2122] (Application example 2)

[2123] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2124] Conventional traffic signal control systems rely solely on traffic volume and do not take into account passenger emotions or stress levels, which limits their effectiveness in easing traffic congestion and reducing passenger stress. This often leads to poor traffic efficiency and lower passenger satisfaction. Furthermore, conventional systems have fixed signal control, making real-time adjustments difficult and preventing flexible responses to changing situations.

[2125] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2126] In this invention, the server includes means for collecting video data acquired by a camera, means for analyzing the collected video data to evaluate traffic volume, and emotion recognition means for evaluating passenger emotions, thereby enabling traffic signal control based on traffic conditions and passenger emotion data in real time.

[2127] "Camera" refers to an image capture device installed on traffic lights and vehicles to capture traffic and passenger expressions.

[2128] "Video data" refers to real-time video information captured by a camera.

[2129] "Traffic volume" is a number or data that indicates the number or flow of vehicles on a road in a particular area or period of time.

[2130] "Signal duration" refers to the duration that each color of light remains on at a traffic light, and is an important parameter for traffic control.

[2131] "Signal control instructions" refer to commands to set specific lighting patterns and times for traffic lights.

[2132] "Emotion recognition means" is a technology that analyzes passengers' facial expressions from video data acquired by a camera and evaluates their emotions and stress levels.

[2133] "Emotion Data" refers to data indicative of passenger emotions and stress levels analyzed by the emotion recognition means.

[2134] The "evaluation result" is information that indicates a comprehensive assessment of the situation obtained from the analysis of traffic volume and emotion data.

[2135] A "traffic light control algorithm" is a method or program for calculating optimal traffic light lighting patterns based on traffic volume and emotion data.

[2136] The "information processing device" is a device that includes a computer system for analyzing acquired data and performing signal control, etc.

[2137] This invention is a system that aims to alleviate traffic congestion and reduce passenger stress by analyzing traffic conditions and passenger emotions and controlling traffic signals in real time. A specific embodiment of this system is described below.

[2138] 1. Data Collection and Preprocessing

[2139] Terminal

[2140] The device captures real-time video data from cameras installed on traffic lights and autonomous vehicles. The video data is preprocessed to analyze traffic volume and vehicle congestion. Preprocessing includes noise reduction and contrast adjustment. The camera also captures facial expressions of passengers in the vehicle and uses this data as emotion data.

[2141] 2. Data analysis and traffic volume evaluation

[2142] server

[2143] The server receives preprocessed video data and emotion data sent from the terminals. Using this data, it analyzes traffic volume and further analyzes passenger emotions using an emotion recognition engine. Traffic volume analysis uses an image recognition algorithm to detect and count the number of vehicles and their movements. Meanwhile, the emotion recognition engine measures passenger stress levels and irritation from their facial expressions.

[2144] 3. Integrated evaluation of traffic volume and user sentiment

[2145] server

[2146] The server comprehensively evaluates traffic volume and emotion data, and determines that traffic conditions are deteriorating or passenger stress levels are high, and prioritizes traffic signal adjustments.

[2147] 4. Execution of signal control algorithm

[2148] server

[2149] Based on the evaluation results, the server executes a traffic light control algorithm, which calculates the optimal signal duration for each intersection. For example, if traffic volume is high and passengers are feeling stressed, dynamic adjustments are made, such as extending the green light duration.

[2150] 5. Generation and transmission of signal control instructions

[2151] server

[2152] The server generates signal control instructions based on the calculated optimal lighting time and instantly sends them to each traffic light device. For example, an instruction to extend the green light at a certain intersection by 30 seconds is sent from the server to the terminal.

[2153] 6. Traffic light adjustment

[2154] Terminal

[2155] The terminal receives the signal control instruction sent from the server and automatically adjusts the lighting time of the signal device, for example, extending the green light lighting time by 30 seconds.

[2156] 7. Feedback Collection and Analysis

[2157] Terminal and Server

[2158] After adjusting the traffic lights, the device again collects traffic data and passenger sentiment data based on the new signal patterns and sends them to the server. The server analyzes this feedback data and evaluates the effectiveness of the signal control algorithm. Based on the analysis results, the algorithm is improved and used for future adjustments.

[2159] Specific examples

[2160] As a concrete example 1, consider alleviating congestion at intersection A. A device acquires video data and emotion data from a camera installed at intersection A and sends it to a server. The server analyzes the received data, generates instructions to extend the green light duration by 30 seconds as needed, and sends them to the device. The traffic light follows this instruction and actually extends the green light duration by 30 seconds. Feedback data is collected again, and the server analyzes it to improve the algorithm.

[2161] Prompt Sentence Examples

[2162] The following prompt sentences are input into the generative AI model to analyze passenger emotional data:

[2163] Recognize the facial expressions of passengers in your car to assess their stress levels.

[2164] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2165] Step 1:

[2166] Data collection

[2167] The device acquires video data in real time from cameras installed on traffic lights and autonomous vehicles.

[2168] Input: Raw video data from the camera.

[2169] Output: Captured video data.

[2170] Specifically, the device captures video from a designated camera at specific time intervals and stores or transmits it as a data stream.

[2171] Step 2:

[2172] Video data preprocessing

[2173] Pre-processing such as noise removal and contrast adjustment is performed on the video data acquired by the device.

[2174] Input: Acquired raw video data.

[2175] Output: Preprocessed video data.

[2176] Specifically, it uses an image processing library (e.g., OpenCV) to apply filters to reduce noise in the image and adjust the contrast and brightness appropriately.

[2177] Step 3:

[2178] Sending data

[2179] The terminal transmits the preprocessed video data to the server.

[2180] Input: Preprocessed video data.

[2181] Output: The data sent to the server.

[2182] Specifically, the device sends preprocessed data to the server in real time through a specific API endpoint.

[2183] Step 4:

[2184] Traffic volume assessment and emotion recognition

[2185] The server receives the preprocessed video data and performs traffic analysis and emotion recognition.

[2186] Input: Preprocessed video data.

[2187] Output: Traffic data and sentiment data.

[2188] Specifically, the server uses image recognition algorithms to detect and count the number and movement of vehicles, and an emotion recognition engine (e.g., facial expression analysis software) to analyze passengers' emotions and stress levels from their facial expressions in the video data.

[2189] Step 5:

[2190] Integrated evaluation

[2191] The server comprehensively evaluates the traffic volume data and emotion data.

[2192] Input: Traffic data and sentiment data.

[2193] Output: Overall evaluation result.

[2194] Specifically, the server uses the analysis results, predefined rules or generative AI models to evaluate the relationship between traffic conditions and stress levels, and determines the necessary adjustments.

[2195] Step 6:

[2196] Execution of signal control algorithms

[2197] The server executes the signal control algorithm based on the comprehensive evaluation result.

[2198] Input: Overall evaluation result.

[2199] Output: Signal control instructions.

[2200] Specifically, the server runs an algorithm to calculate the optimal signal duration and determine, for example, the green light extension time.

[2201] Step 7:

[2202] Sending signal control instructions

[2203] The server transmits the calculated signal control instructions to the terminal.

[2204] Input: Signal control instructions.

[2205] Output: Signal control instructions sent to the terminal.

[2206] As a specific operation, the server transmits instruction data including a command to adjust the signal timing to the terminal.

[2207] Step 8:

[2208] Traffic light adjustments

[2209] The terminal adjusts the signaling device based on the received signal control instructions.

[2210] Input: Signal control instructions.

[2211] Output: Traffic light operating status after adjustment.

[2212] Specifically, the terminal sends instructions to the traffic light control unit to change its timing.

[2213] Step 9:

[2214] Collecting and sending feedback

[2215] The terminal again collects traffic data and emotion data for the new signal pattern and transmits them to the server.

[2216] Input: Traffic data and emotion data with adjusted signal patterns.

[2217] Output: Feedback data sent to the server.

[2218] As a specific operation, the terminal collects data in the same manner as in the first step and transmits it to the server as feedback data.

[2219] Step 10:

[2220] Feedback analysis and algorithm improvement

[2221] The server analyzes the feedback data and evaluates the effectiveness of the signal control algorithm.

[2222] Input: Feedback data.

[2223] Output: Algorithm improvements.

[2224] Specifically, the server compares the collected data with the results of the algorithm and adjusts and improves the control algorithm as needed.

[2225] The above are the processing steps of the system for implementing the present invention.

[2226] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2227] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2228] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2229] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2230] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2231] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2232] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2233] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2234] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2235] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2236] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2237] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2238] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2239] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2240] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2241] The hardware resource for executing a specific process can be any of the following processors: A CPU is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A dedicated electrical circuit, such as a field-programmable gate array (FPGA), a programmable logic device (PLD), or an application-specific integrated circuit (ASIC), is a processor with a circuit configuration specifically designed to execute a specific process. Each processor has built-in or connected memory, and uses the memory to execute the specific process.

[2242] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2243] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2244] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2245] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2246] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2247] The following is further disclosed regarding the above embodiment.

[2248] (Claim 1)

[2249] means for collecting video data acquired by the camera;

[2250] a means for analyzing the collected video data to assess traffic volume;

[2251] means for dynamically adjusting the illumination duration of the signal based on the evaluation results;

[2252] means for transmitting a signal control instruction to a traffic light;

[2253] A system including:

[2254] (Claim 2)

[2255] 10. The system of claim 1, further comprising means for pre-processing video data acquired by the camera.

[2256] (Claim 3)

[2257] 10. The system of claim 1, further comprising: means for executing a signal control algorithm based on the evaluation results.

[2258] "Example 1"

[2259] (Claim 1)

[2260] means for collecting video data acquired by the camera;

[2261] means for pre-processing video data acquired by the camera;

[2262] a means for analyzing the collected video data to assess traffic volume;

[2263] means for executing a signal control algorithm based on the evaluation results;

[2264] means for generating and transmitting signal control instructions to a traffic light;

[2265] a means for adjusting the lighting time of a traffic light based on a signal control instruction;

[2266] a means for collecting and analyzing feedback data;

[2267] A system including:

[2268] (Claim 2)

[2269] 10. The system of claim 1, further comprising means for dynamically adjusting the illumination duration of the signal based on the evaluation result.

[2270] (Claim 3)

[2271] 10. The system of claim 1, further comprising means for continuously improving the signal control algorithm to flexibly respond to changes in ...

Claims

1. means for collecting video data acquired by the camera; a means for analyzing the collected video data to assess traffic volume; means for dynamically adjusting the illumination duration of the signal based on the evaluation results; means for transmitting a signal control instruction to a traffic light; A system including:

2. The system of claim 1 further comprising means for pre-processing video data acquired by the camera.

3. The system of claim 1 further comprising means for executing a signal control algorithm based on the evaluation results.

Images