system

Abstract

The system according to this embodiment aims to analyze the surrounding conditions and optimize traffic flow through the cooperation of traffic signals. [Solution] The system according to the embodiment comprises an acquisition unit, a determination unit, a control unit, a calculation unit, and a coordination unit. The acquisition unit acquires camera images attached to the traffic signals. The determination unit analyzes the camera images acquired by the acquisition unit and determines the surrounding conditions. The control unit controls the signals based on the surrounding conditions determined by the determination unit. The calculation unit calculates the waiting cost of the signals controlled by the control unit. The coordination unit coordinates the traffic signals based on the waiting cost calculated by the calculation unit.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, there is a problem that the control of traffic lights is fixed and it is difficult to optimize the traffic flow.

[0005] The system according to the embodiment aims to analyze the surrounding situation and optimize the traffic flow by cooperation between traffic lights.

Means for Solving the Problems

[0006] The system according to this embodiment comprises an acquisition unit, a determination unit, a control unit, a calculation unit, and a coordination unit. The acquisition unit acquires camera images attached to the traffic signals. The determination unit analyzes the camera images acquired by the acquisition unit and determines the surrounding conditions. The control unit controls the signals based on the surrounding conditions determined by the determination unit. The calculation unit calculates the waiting cost of the signals controlled by the control unit. The coordination unit coordinates the traffic signals based on the waiting costs calculated by the calculation unit. [Effects of the Invention]

[0007] The system according to this embodiment can analyze the surrounding conditions and optimize traffic flow through the cooperation of traffic signals. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10]This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor, an antenna, etc. The communication I / F manages communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network), etc.

[0019] The smart device 14 comprises a computer 36, a receiving device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The receiving device 38, output device 40, and camera 42 are also connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The signal control system according to an embodiment of the present invention is a system that uses AI to determine the surrounding conditions, calculates the cost of making vehicles wait, and then controls the signals. The signal control system uses a generating AI to analyze camera images attached to traffic lights and controls the signals. By installing and coordinating these systems on a large scale, it becomes possible to control signals optimally for the overall traffic flow. This ensures that logistics proceed smoothly without delays. For example, the signal control system uses cameras attached to traffic lights to capture images of the surrounding conditions. These camera images are input to the generating AI, which analyzes the surrounding conditions. For example, if there are no vehicles or pedestrians at an intersection, the generating AI can change the signal to green based on this information. Next, the signal control system uses the generating AI to calculate the cost of making vehicles wait. For example, it considers the time a vehicle is made to wait at a red light at an intersection and the amount of fuel consumed during that time. This allows the signal to be changed to green earlier if the cost of making vehicles wait is high. Furthermore, by coordinating a large number of traffic lights, it becomes possible to control signals optimally for the overall traffic flow. For example, traffic lights on major arterial roads can coordinate to adjust the timing of the signals to smooth the flow of vehicles. This reduces traffic congestion and ensures that logistics proceed without delays. Traffic signal control systems not only ensure the smooth and uninterrupted flow of goods, but also offer benefits such as reduced traffic congestion and lower fuel consumption. For example, shorter waiting times at intersections reduce driver stress and fuel consumption. Furthermore, safer traffic signal control for pedestrians reduces the risk of traffic accidents. Thus, traffic signal control systems, which use AI to assess surrounding conditions and calculate the cost of waiting before controlling traffic signals, bring about numerous benefits, including smoother logistics, reduced traffic congestion, and lower fuel consumption.

[0029] The signal control system according to the embodiment comprises an acquisition unit, a determination unit, a control unit, a calculation unit, and a coordination unit. The acquisition unit acquires camera images attached to the traffic lights. The acquisition unit, for example, uses the camera attached to the traffic lights to photograph the situation at the intersection. The acquisition unit can acquire the optimal image by adjusting the camera's resolution and frame rate, for example. The acquisition unit can grasp a wide area of ​​the situation by adjusting the camera's field of view, for example. The determination unit uses a generation AI to analyze the camera images acquired by the acquisition unit and determine the surrounding situation. The determination unit, for example, uses the generation AI to analyze the camera images and determine whether there are vehicles or pedestrians at the intersection. The determination unit, for example, uses the generation AI to analyze the camera images and determine the traffic volume and pedestrian movement. The determination unit, for example, uses the generation AI to analyze the camera images and determine the weather and visibility. The control unit controls the signal based on the surrounding situation determined by the determination unit. The control unit, for example, changes the signal to green if there are no vehicles or pedestrians at the intersection. The control unit, for example, adjusts the timing of the signal if there is heavy traffic. The control unit prioritizes pedestrian signals, for example, when there are many pedestrians. The calculation unit calculates the waiting cost of signals controlled by the control unit. The calculation unit calculates, for example, the time a vehicle is made to wait at a red light at an intersection. The calculation unit calculates fuel consumption based on the waiting time. The calculation unit calculates driver stress based on the waiting time. The coordination unit coordinates traffic lights based on the waiting cost calculated by the calculation unit. The coordination unit coordinates traffic lights on major arterial roads to adjust the timing of signals to smooth the flow of traffic. The coordination unit coordinates traffic lights at intersections to reduce traffic congestion. The coordination unit communicates with traffic lights to share information in real time. As a result, the signal control system according to the embodiment can optimize traffic flow by analyzing camera images of traffic lights, calculating the waiting cost, and controlling the signals.

[0030] The acquisition unit acquires images from cameras mounted on traffic lights. For example, the acquisition unit uses cameras mounted on traffic lights to photograph the situation at intersections. Specifically, the cameras are high-resolution and can provide clear images day and night. The camera's resolution and frame rate are dynamically adjusted according to the traffic conditions at the intersection. For example, the frame rate is set higher during peak traffic hours to capture detailed movements. The camera's field of view is also adjustable, and a wide-angle lens can be used to grasp a wide area of ​​the situation. This allows for a complete overview of the intersection at once. Furthermore, the cameras are weather-resistant and operate normally even in adverse weather conditions such as rain and snow. The camera's installation location is also important; mounting it on the top or side of the traffic light minimizes blind spots. The acquisition unit collects image data from these cameras in real time and transmits it to a central database. The database efficiently manages the acquired image data and makes it accessible to the analysis unit and other systems. This allows the acquisition unit to accurately and quickly grasp the situation at intersections and improve the overall performance of the traffic signal control system.

[0031] The judgment unit uses a generation AI to analyze camera images acquired by the acquisition unit and determine the surrounding situation. The generation AI utilizes image recognition technology to detect the presence of vehicles and pedestrians at intersections with high accuracy. Specifically, the generation AI analyzes the shape and movement of vehicles from camera images to identify the type of vehicle, speed, and direction of travel. It also analyzes the movement of pedestrians to determine the number of pedestrians, direction of movement, and speed. Furthermore, the generation AI analyzes weather and visibility, taking into account the deterioration of visibility in adverse weather conditions such as rain and fog. As a result, the judgment unit can comprehensively grasp the situation at the intersection and provide the information necessary for signal control. The generation AI uses a deep learning model and continuously improves its analysis accuracy by learning from past data. For example, it can learn from past traffic data and weather data to predict traffic patterns under specific conditions. As a result, the judgment unit can not only grasp the situation in real time but also make future predictions. Furthermore, the generation AI can use an anomaly detection algorithm to detect unusual patterns and abnormal movements early and issue warnings. This allows the judgment unit to improve intersection safety and reduce the risk of traffic accidents.

[0032] The control unit controls the traffic signals based on the surrounding conditions determined by the judgment unit. Specifically, if there are no vehicles or pedestrians at the intersection, it changes the signal to green to smooth the flow of traffic. In cases of heavy traffic, it adjusts the timing of the signals to minimize vehicle congestion. For example, on major arterial roads, it extends the green time of the signal to prioritize the flow of vehicles. On the other hand, if there are many pedestrians, it prioritizes pedestrian signals to allow them to cross safely. The control unit performs these controls in real time, enabling flexible responses according to traffic conditions. Furthermore, the control unit also has a function to prioritize the passage of emergency vehicles; when an emergency vehicle approaches the intersection, it immediately changes the signal to green to support its rapid passage. To perform these controls, the control unit is equipped with a processor with high processing power, enabling real-time signal control. As a result, the control unit can optimize the flow of traffic and improve the safety and efficiency of intersections.

[0033] The calculation unit calculates the cost of making vehicles wait at traffic lights controlled by the control unit. Specifically, it calculates the time vehicles are made to wait at red lights at intersections and evaluates fuel consumption and driver stress based on that time. For example, if vehicles are made to wait at red lights for a long time, fuel consumption increases and the environmental burden increases. Driver stress also increases, raising the risk of traffic accidents. The calculation unit comprehensively evaluates these factors and uses the results to optimize traffic signal control. Furthermore, the calculation unit evaluates not only the cost of waiting but also the effect of traffic flow improvement through traffic signal control. For example, adjusting the timing of traffic lights can reduce traffic congestion and improve the average speed of vehicles. This can lead to improved overall traffic efficiency, reduced travel time, and lower fuel consumption. Based on these evaluation results, the calculation unit adjusts the parameters of traffic signal control to achieve optimal traffic signal control. In this way, the calculation unit can optimize traffic flow and contribute to reducing environmental impact and driver stress.

[0034] The coordination unit coordinates traffic signals based on the waiting costs calculated by the calculation unit. Specifically, traffic signals on major arterial roads coordinate to adjust signal timing to smooth the flow of traffic. For example, on major arterial roads, traffic signals coordinate to create a wave of green lights, allowing vehicles to pass through intersections continuously. Traffic signals at intersections also coordinate to adjust signal timing to reduce traffic congestion. For example, if traffic signals at an intersection are heavy in a particular direction, they prioritize the signal for that direction. The coordination unit achieves this coordination by having traffic signals communicate with each other and share information in real time. Furthermore, the coordination unit also cooperates with other traffic management systems and infrastructure to perform comprehensive traffic management. For example, it cooperates with traffic information centers and emergency vehicle management systems to respond quickly in emergencies. In this way, the coordination unit can optimize traffic flow through the coordination of traffic signals, reduce traffic congestion, and support emergency response.

[0035] The acquisition unit can acquire camera images mounted on traffic lights. The acquisition unit can, for example, use the camera mounted on the traffic light to photograph the situation at the intersection. The acquisition unit can, for example, adjust the camera's resolution and frame rate to acquire the optimal image. The acquisition unit can, for example, adjust the camera's field of view to grasp a wide area of ​​the situation. As a result, by acquiring camera images mounted on traffic lights, the surrounding situation can be understood. Some or all of the above processing in the acquisition unit may be performed using AI, for example, or without AI. For example, the acquisition unit can input the camera image mounted on the traffic light into a generating AI and have the generating AI perform the image acquisition.

[0036] The judgment unit can analyze camera images using a generating AI to determine the surrounding conditions. For example, the judgment unit can use the generating AI to analyze camera images and determine whether there are vehicles or pedestrians at an intersection. For example, the judgment unit can use the generating AI to analyze camera images and determine traffic volume and pedestrian movement. For example, the judgment unit can use the generating AI to analyze camera images and determine the weather and visibility. As a result, using the generating AI improves the accuracy of camera image analysis and allows for accurate determination of the surrounding conditions. Some or all of the above-described processes in the judgment unit are performed using the generating AI. For example, the judgment unit can input camera images into the generating AI and have the generating AI perform the determination of the surrounding conditions.

[0037] The control unit can control the traffic signals based on the determined surrounding conditions. For example, the control unit changes the signal to green if there are no vehicles or pedestrians at the intersection. For example, the control unit adjusts the timing of the signals if there is heavy traffic. For example, the control unit prioritizes pedestrian signals if there are many pedestrians. In this way, traffic flow can be optimized by controlling the signals based on the surrounding conditions. Some or all of the above processing in the control unit may be performed using AI or not. For example, the control unit can input the determined surrounding conditions into a generating AI and have the generating AI perform the signal control.

[0038] The calculation unit can calculate the cost of making vehicles wait at traffic lights. For example, the calculation unit calculates the amount of time a vehicle is made to wait at a red light at an intersection. For example, the calculation unit calculates fuel consumption based on the waiting time. For example, the calculation unit calculates driver stress based on the waiting time. By calculating the cost of waiting, the efficiency of traffic light control can be improved. Some or all of the above processing in the calculation unit may be performed using AI or not. For example, the calculation unit can have a generating AI perform the calculation of the cost of waiting.

[0039] The coordination unit allows traffic lights to coordinate with each other based on the calculated waiting cost. For example, the coordination unit allows traffic lights on major arterial roads to coordinate and adjust the timing of signals to smooth the flow of traffic. For example, the coordination unit allows traffic lights at intersections to coordinate and reduce traffic congestion. For example, the coordination unit allows traffic lights to communicate with each other and share information in real time. This allows the overall traffic flow to be optimized through the coordination of traffic lights. Some or all of the above processes in the coordination unit may be performed using AI or not. For example, the coordination unit can have a generating AI perform the coordination of traffic lights.

[0040] The control unit can reduce waiting times at intersections. For example, the control unit changes the traffic light to green when there are no vehicles or pedestrians at the intersection. For example, the control unit adjusts the timing of the traffic lights when there is heavy traffic. For example, the control unit prioritizes pedestrian signals when there are many pedestrians. This reduces waiting times at intersections and makes traffic flow smoother. Some or all of the above-described processes in the control unit may be performed using AI or not. For example, the control unit can have a generating AI perform the task of reducing waiting times at intersections.

[0041] The control unit can reduce traffic congestion. For example, the control unit can coordinate traffic lights on major arterial roads and adjust the timing of signals to smooth the flow of vehicles. For example, the control unit can coordinate traffic lights at intersections to reduce traffic congestion. For example, the control unit can have traffic lights communicate with each other and share information in real time. This can smooth the flow of traffic by reducing traffic congestion. Some or all of the above processing in the control unit may be performed using AI or not. For example, the control unit can have a generating AI perform the reduction of traffic congestion.

[0042] The control unit can reduce fuel consumption. For example, the control unit can reduce fuel consumption by shortening waiting times at intersections. For example, the control unit can reduce fuel consumption by reducing traffic congestion. For example, the control unit can reduce fuel consumption by adjusting the timing of traffic signals. By reducing fuel consumption, the environmental burden can be reduced. Some or all of the above-described processes in the control unit may be performed using AI or not. For example, the control unit can have a generating AI perform the reduction of fuel consumption.

[0043] The control unit can implement traffic signal control that is safe for pedestrians. For example, the control unit prioritizes pedestrian signals when there are many pedestrians. For example, the control unit detects pedestrian movement and changes the signal to green. For example, the control unit changes signals preferentially based on pedestrian location information. This reduces the risk of traffic accidents by enabling traffic signal control that is safe for pedestrians. Some or all of the above-described processes in the control unit may be performed using AI or not. For example, the control unit can have a generating AI execute traffic signal control that is safe for pedestrians.

[0044] The acquisition unit can dynamically adjust the camera's field of view to acquire the optimal image. For example, the acquisition unit widens the camera's field of view to grasp the overall situation depending on the congestion level of an intersection. For example, the acquisition unit narrows the camera's field of view to acquire a detailed image in order to focus on a specific vehicle or pedestrian. For example, the acquisition unit adjusts the camera's field of view depending on the weather or time of day to acquire the optimal image. In this way, the optimal image can be acquired by dynamically adjusting the camera's field of view. Some or all of the above processing in the acquisition unit may be performed using AI or not. For example, the acquisition unit can have a generating AI perform the adjustment of the camera's field of view.

[0045] The acquisition unit can automatically change the resolution of the acquired image according to the surrounding conditions. For example, the acquisition unit can increase the image resolution to grasp detailed conditions depending on the congestion at an intersection. For example, the acquisition unit can increase the image resolution to improve visibility at night or in bad weather. For example, the acquisition unit can reduce the system load by lowering the image resolution under normal conditions. In this way, by automatically changing the image resolution, the optimal image can be acquired according to the situation. Some or all of the above processing in the acquisition unit may be performed using AI or not. For example, the acquisition unit can have a generating AI perform the change in image resolution.

[0046] The acquisition unit can integrate images from multiple cameras to grasp a more detailed situation. For example, the acquisition unit can integrate images from multiple cameras at an intersection to grasp the overall situation. For example, the acquisition unit can integrate images from multiple cameras in a specific area to grasp a detailed situation. For example, the acquisition unit can integrate images from multiple cameras according to weather and time of day to grasp the optimal situation. This makes it possible to grasp a detailed situation by integrating images from multiple cameras. Some or all of the above processing in the acquisition unit may be performed using AI or not. For example, the acquisition unit can have a generating AI perform the integration of images from multiple cameras.

[0047] The acquisition unit can acquire images from above using a drone and combine them with images from a ground camera. For example, the acquisition unit can acquire images from above an intersection and combine them with images from a ground camera to understand the overall situation. For example, the acquisition unit can acquire images from above a specific area and combine them with images from a ground camera to understand the detailed situation. For example, the acquisition unit can acquire images from above depending on the weather and time of day and combine them with images from a ground camera to understand the optimal situation. In this way, by acquiring images from above and combining them with images from a ground camera, the overall situation can be understood. Some or all of the above processing in the acquisition unit may be performed using AI or not. For example, the acquisition unit can have a generating AI perform the acquisition of images from above.

[0048] The judgment unit can apply different analysis methods depending on the weather and time of day during analysis. For example, in rainy weather, the judgment unit applies a specific filter to perform analysis because visibility is reduced. For example, at night, the judgment unit applies an analysis method that is suitable for low-light environments. For example, during congested daytime hours, the judgment unit applies a method that performs rapid analysis to smooth traffic flow. In this way, the accuracy of the analysis is improved by applying analysis methods according to the weather and time of day. Some or all of the above processing in the judgment unit is performed using a generation AI. For example, the judgment unit can have the generation AI execute the application of analysis methods according to the weather and time of day.

[0049] The judgment unit can predict the current situation by referring to past data during analysis. For example, the judgment unit predicts and analyzes the current traffic situation based on past traffic data. For example, the judgment unit predicts and analyzes the current weather situation based on past weather data. For example, the judgment unit predicts and analyzes the current accident risk based on past accident data. In this way, by referring to past data, the current situation can be predicted more accurately. Some or all of the above processing in the judgment unit is performed using a generation AI. For example, the judgment unit can have the generation AI perform the referencing of past data and the prediction of the current situation.

[0050] The judgment unit can analyze audio data and make a judgment while also considering the surrounding sound conditions. For example, the judgment unit can analyze audio data from an intersection and make a judgment by detecting vehicle engine sounds and horn sounds. For example, the judgment unit can analyze audio data of pedestrians and determine the presence of pedestrians. For example, the judgment unit can analyze ambient sounds and determine traffic conditions and weather conditions. In this way, by analyzing audio data, it becomes possible to make a judgment that also considers the surrounding sound conditions. Some or all of the above processing in the judgment unit is performed using a generation AI. For example, the judgment unit can have the generation AI perform the analysis of audio data.

[0051] The judgment unit can analyze environmental data such as ambient temperature and humidity to determine the overall situation. For example, the judgment unit analyzes ambient temperature data to determine factors affecting traffic conditions. For example, the judgment unit analyzes ambient humidity data to determine visibility and road surface conditions. For example, the judgment unit comprehensively analyzes environmental data to provide information for optimal signal control. By comprehensively analyzing environmental data, more accurate situation determination becomes possible. Some or all of the above processing in the judgment unit is performed using a generation AI. For example, the judgment unit can have the generation AI perform the analysis of environmental data.

[0052] The control unit can detect the approach of an emergency vehicle and change the signal accordingly. For example, the control unit can detect the sound of an emergency vehicle's siren and change the signal to green. For example, the control unit can change the signal accordingly based on the location information of an emergency vehicle. For example, the control unit can predict the approach of an emergency vehicle and change the signal in advance. This allows for smoother passage of emergency vehicles by detecting their approach and changing the signal accordingly. Some or all of the above-described processes in the control unit may be performed using AI or not. For example, the control unit can have a generating AI perform the detection of approaching emergency vehicles and the preferential change of signals.

[0053] The control unit can apply algorithms that prioritize specific vehicles (e.g., public transport). For example, the control unit may change signals preferentially based on the location information of public transport. For example, the control unit may change signals preferentially considering the operating schedule of public transport. For example, the control unit may predict the approach of public transport and change signals in advance. This allows for smoother operation of public transport by prioritizing specific vehicles. Some or all of the above-described processes in the control unit may be performed using AI or not. For example, the control unit may have a generating AI perform the priority of specific vehicles.

[0054] The control unit can detect pedestrian movement and perform pedestrian-only signal control. For example, the control unit can detect pedestrian movement with a camera and change the signal to green. For example, the control unit can change the signal preferentially based on pedestrian location information. For example, the control unit can predict the approach of a pedestrian and change the signal in advance. This makes pedestrian-only signal control possible by detecting pedestrian movement. Some or all of the above processing in the control unit may be performed using AI or not. For example, the control unit can have a generating AI perform the detection of pedestrian movement and the change of the signal.

[0055] The control unit can optimize traffic flow by performing dedicated signal control for bicycles. For example, the control unit can detect the movement of a bicycle using a camera and change the signal to green. For example, the control unit can prioritize changing the signal based on the bicycle's location information. For example, the control unit can predict the approach of a bicycle and change the signal in advance. In this way, traffic flow can be optimized by performing dedicated signal control for bicycles. Some or all of the above processes in the control unit may be performed using AI or not. For example, the control unit can have a generating AI perform the detection of bicycle movement and the changing of the signal.

[0056] The calculation unit can consider not only the vehicle's fuel consumption but also its exhaust emissions during calculations. For example, the calculation unit calculates the waiting cost based on the vehicle's fuel consumption and exhaust emissions. For example, the calculation unit sets a higher waiting cost for vehicles with high fuel consumption. For example, the calculation unit sets a higher waiting cost for vehicles with high exhaust emissions. By considering fuel consumption and exhaust emissions, the environmental impact can be reduced. Some or all of the above processing in the calculation unit may be performed using AI or not. For example, the calculation unit can have a generating AI perform the consideration of fuel consumption and exhaust emissions.

[0057] The calculation unit can perform different cost calculations depending on the type of vehicle (e.g., electric vehicle or hybrid vehicle) during the calculation process. For example, the calculation unit can set the waiting cost low for electric vehicles, medium for hybrid vehicles, and high for gasoline vehicles. This allows for more accurate cost calculations by performing cost calculations according to the type of vehicle. Some or all of the above processing in the calculation unit may be performed using AI or not. For example, the calculation unit can have a generating AI perform cost calculations according to the type of vehicle.

[0058] The calculation unit can calculate costs while also considering the risk of traffic accidents. For example, the calculation unit calculates the cost of making people wait based on past accident data at an intersection. For example, if the risk of traffic accidents is high, the calculation unit sets a higher cost for making people wait. For example, if the risk of traffic accidents is low, the calculation unit sets a lower cost for making people wait. This makes it possible to control traffic signals more safely by considering the risk of traffic accidents. Some or all of the above processing in the calculation unit may be performed using AI or not. For example, the calculation unit can have a generating AI perform the consideration of the risk of traffic accidents.

[0059] The calculation unit can calculate costs while also considering the congestion levels of surrounding commercial facilities. For example, the calculation unit calculates the cost of making customers wait based on the congestion levels of surrounding commercial facilities. For example, if a commercial facility is crowded, the calculation unit sets a higher cost for making customers wait. For example, if a commercial facility is not crowded, the calculation unit sets a lower cost for making customers wait. This makes it possible to calculate costs more appropriately by considering the congestion levels of commercial facilities. Some or all of the above processing in the calculation unit may be performed using AI or not. For example, the calculation unit can have a generating AI perform the task of considering the congestion levels of commercial facilities.

[0060] The coordinating unit can apply algorithms that prioritize traffic flow on major arterial roads. For example, the coordinating unit prioritizes traffic flow on major arterial roads and changes traffic lights to green. For example, the coordinating unit adjusts the timing of traffic lights based on the traffic volume on arterial roads. For example, the coordinating unit predicts traffic congestion on arterial roads and changes traffic lights in advance. This reduces traffic congestion by prioritizing traffic flow on major arterial roads. Some or all of the above processing in the coordinating unit may be performed using AI or not. For example, the coordinating unit can have a generating AI perform the priority of traffic flow on arterial roads.

[0061] The coordinating unit can adjust signal control according to specific events (e.g., sporting events or concerts). For example, the coordinating unit may prioritize the surrounding traffic flow when a sporting event is held. For example, the coordinating unit may prioritize the surrounding traffic flow when a concert is held. For example, the coordinating unit may predict traffic congestion and change signals in advance when a specific event is held. This allows for smoother traffic flow by providing signal control tailored to specific events. Some or all of the above processing in the coordinating unit may be performed using AI or not. For example, the coordinating unit may have a generating AI execute signal control tailored to specific events.

[0062] The integration unit can perform comprehensive traffic management in conjunction with other traffic management systems (e.g., parking management systems). For example, the integration unit can integrate with a parking management system to control traffic signals based on parking availability. For example, the integration unit can integrate with the operating status of public transportation to optimize traffic signal control. For example, the integration unit can integrate with other traffic management systems to perform comprehensive traffic management. This enables comprehensive traffic management by integrating with other traffic management systems. Some or all of the above-described processes in the integration unit may be performed using AI or not. For example, the integration unit can have a generating AI perform the integration with other traffic management systems.

[0063] The coordinating unit can optimize signal control in conjunction with the operating status of public transportation. For example, the coordinating unit optimizes signal control based on the operating status of public transportation. For example, the coordinating unit adjusts signal control considering delay information of public transportation. For example, the coordinating unit optimizes signal control in conjunction with the operating schedule of public transportation. In this way, signal control can be optimized by coordinating with the operating status of public transportation. Some or all of the above processing in the coordinating unit may be performed using AI or not. For example, the coordinating unit can have a generating AI perform the coordination with the operating status of public transportation.

[0064] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0065] The signal control system can also be equipped with a voice recognition unit. This unit can acquire and analyze audio data from around the intersection. For example, it can detect vehicle engine noises and horn sounds to estimate traffic volume. It can also detect pedestrian voices and footsteps to confirm their presence. Furthermore, the voice recognition unit can detect emergency vehicle sirens and prioritize the change of traffic signals accordingly. This allows for a more accurate understanding of traffic conditions by utilizing audio data.

[0066] The signal control system can also be equipped with an environmental sensor unit. This unit can acquire and analyze environmental data such as ambient temperature, humidity, and atmospheric pressure. For example, the environmental sensor unit can shorten signal waiting times when the temperature is high, preventing vehicle engine overheating. It can also adjust signal timing considering road surface slipperiness when humidity is high. Furthermore, it can detect changes in atmospheric pressure and perform signal control to respond to sudden weather changes. This allows for safer and more efficient signal control by utilizing environmental data.

[0067] The traffic signal control system can also be equipped with a drone unit. The drone unit can acquire aerial footage and analyze it in combination with ground-based camera images. For example, the drone unit can acquire aerial footage of an intersection to understand the overall traffic situation. It can also fly a drone to acquire real-time footage to check congestion in a specific area in detail. Furthermore, the drone unit can quickly check the situation on site in the event of an emergency and implement appropriate traffic signal control. This makes it possible to understand traffic conditions more accurately and over a wider area by utilizing an aerial perspective.

[0068] The traffic signal control system can also be equipped with a user feedback unit. This unit can collect feedback from drivers and pedestrians and incorporate it into the signal control. For example, the user feedback unit can collect opinions and requests from drivers via a smartphone app and adjust the timing of the signals. It can also change the priority of pedestrian signals based on feedback from pedestrians. Furthermore, the user feedback unit can analyze the collected feedback and use it to improve the traffic signal control system. This allows for a more user-friendly traffic signal control system by incorporating user feedback.

[0069] The traffic signal control system can also be equipped with a prediction unit. This unit can predict current traffic conditions based on past traffic data and reflect this in traffic signal control. For example, the prediction unit can analyze past traffic volume data to predict fluctuations in traffic volume during specific time periods or on specific days of the week. It can also predict changes in traffic conditions due to weather changes based on past weather data. Furthermore, the prediction unit can predict the risk of accidents based on past accident data and reflect this in traffic signal control. This allows for more accurate prediction of traffic conditions and traffic signal control by utilizing past data.

[0070] The following briefly describes the processing flow for example form 1.

[0071] Step 1: The acquisition unit acquires images from a camera mounted on a traffic light. For example, the acquisition unit uses the camera mounted on the traffic light to photograph the situation at the intersection and adjusts the camera's resolution and frame rate to acquire the optimal image. It can also adjust the camera's field of view to grasp a wide area of ​​the situation. Step 2: The judgment unit uses a generation AI to analyze the camera images acquired by the acquisition unit and determine the surrounding conditions. For example, the generation AI analyzes the camera images to determine whether there are vehicles or pedestrians at the intersection, the volume of traffic, pedestrian movement, weather, and visibility. Step 3: The control unit controls the signals based on the surrounding conditions determined by the determination unit. For example, if there are no vehicles or pedestrians at the intersection, the signal is changed to green; if there is heavy traffic, the timing of the signals is adjusted; and if there are many pedestrians, pedestrian signals are given priority. Step 4: The calculation unit calculates the cost of waiting for signals controlled by the control unit. For example, it calculates the time a vehicle is made to wait at a red light at an intersection, the fuel consumption based on the waiting time, and the stress on the driver. Step 5: The coordination unit coordinates traffic signals based on the waiting costs calculated by the calculation unit. For example, traffic signals on major arterial roads coordinate to adjust signal timing to smooth the flow of traffic, and traffic signals at intersections coordinate to reduce traffic congestion. In addition, traffic signals communicate with each other and share information in real time.

[0072] (Example of form 2) The signal control system according to an embodiment of the present invention is a system that uses AI to determine the surrounding conditions, calculates the cost of making vehicles wait, and then controls the signals. The signal control system uses a generating AI to analyze camera images attached to traffic lights and controls the signals. By installing and coordinating these systems on a large scale, it becomes possible to control signals optimally for the overall traffic flow. This ensures that logistics proceed smoothly without delays. For example, the signal control system uses cameras attached to traffic lights to capture images of the surrounding conditions. These camera images are input to the generating AI, which analyzes the surrounding conditions. For example, if there are no vehicles or pedestrians at an intersection, the generating AI can change the signal to green based on this information. Next, the signal control system uses the generating AI to calculate the cost of making vehicles wait. For example, it considers the time a vehicle is made to wait at a red light at an intersection and the amount of fuel consumed during that time. This allows the signal to be changed to green earlier if the cost of making vehicles wait is high. Furthermore, by coordinating a large number of traffic lights, it becomes possible to control signals optimally for the overall traffic flow. For example, traffic lights on major arterial roads can coordinate to adjust the timing of the signals to smooth the flow of vehicles. This reduces traffic congestion and ensures that logistics proceed without delays. Traffic signal control systems not only ensure the smooth and uninterrupted flow of goods, but also offer benefits such as reduced traffic congestion and lower fuel consumption. For example, shorter waiting times at intersections reduce driver stress and fuel consumption. Furthermore, safer traffic signal control for pedestrians reduces the risk of traffic accidents. Thus, traffic signal control systems, which use AI to assess surrounding conditions and calculate the cost of waiting before controlling traffic signals, bring about numerous benefits, including smoother logistics, reduced traffic congestion, and lower fuel consumption.

[0073] The signal control system according to the embodiment comprises an acquisition unit, a determination unit, a control unit, a calculation unit, and a coordination unit. The acquisition unit acquires camera images attached to the traffic lights. The acquisition unit, for example, uses the camera attached to the traffic lights to photograph the situation at the intersection. The acquisition unit can acquire the optimal image by adjusting the camera's resolution and frame rate, for example. The acquisition unit can grasp a wide area of ​​the situation by adjusting the camera's field of view, for example. The determination unit uses a generation AI to analyze the camera images acquired by the acquisition unit and determine the surrounding situation. The determination unit, for example, uses the generation AI to analyze the camera images and determine whether there are vehicles or pedestrians at the intersection. The determination unit, for example, uses the generation AI to analyze the camera images and determine the traffic volume and pedestrian movement. The determination unit, for example, uses the generation AI to analyze the camera images and determine the weather and visibility. The control unit controls the signal based on the surrounding situation determined by the determination unit. The control unit, for example, changes the signal to green if there are no vehicles or pedestrians at the intersection. The control unit, for example, adjusts the timing of the signal if there is heavy traffic. The control unit prioritizes pedestrian signals, for example, when there are many pedestrians. The calculation unit calculates the waiting cost of signals controlled by the control unit. The calculation unit calculates, for example, the time a vehicle is made to wait at a red light at an intersection. The calculation unit calculates fuel consumption based on the waiting time. The calculation unit calculates driver stress based on the waiting time. The coordination unit coordinates traffic lights based on the waiting cost calculated by the calculation unit. The coordination unit coordinates traffic lights on major arterial roads to adjust the timing of signals to smooth the flow of traffic. The coordination unit coordinates traffic lights at intersections to reduce traffic congestion. The coordination unit communicates with traffic lights to share information in real time. As a result, the signal control system according to the embodiment can optimize traffic flow by analyzing camera images of traffic lights, calculating the waiting cost, and controlling the signals.

[0074] The acquisition unit acquires images from cameras mounted on traffic lights. For example, the acquisition unit uses cameras mounted on traffic lights to photograph the situation at intersections. Specifically, the cameras are high-resolution and can provide clear images day and night. The camera's resolution and frame rate are dynamically adjusted according to the traffic conditions at the intersection. For example, the frame rate is set higher during peak traffic hours to capture detailed movements. The camera's field of view is also adjustable, and a wide-angle lens can be used to grasp a wide area of ​​the situation. This allows for a complete overview of the intersection at once. Furthermore, the cameras are weather-resistant and operate normally even in adverse weather conditions such as rain and snow. The camera's installation location is also important; mounting it on the top or side of the traffic light minimizes blind spots. The acquisition unit collects image data from these cameras in real time and transmits it to a central database. The database efficiently manages the acquired image data and makes it accessible to the analysis unit and other systems. This allows the acquisition unit to accurately and quickly grasp the situation at intersections and improve the overall performance of the traffic signal control system.

[0075] The judgment unit uses a generation AI to analyze camera images acquired by the acquisition unit and determine the surrounding situation. The generation AI utilizes image recognition technology to detect the presence of vehicles and pedestrians at intersections with high accuracy. Specifically, the generation AI analyzes the shape and movement of vehicles from camera images to identify the type of vehicle, speed, and direction of travel. It also analyzes the movement of pedestrians to determine the number of pedestrians, direction of movement, and speed. Furthermore, the generation AI analyzes weather and visibility, taking into account the deterioration of visibility in adverse weather conditions such as rain and fog. As a result, the judgment unit can comprehensively grasp the situation at the intersection and provide the information necessary for signal control. The generation AI uses a deep learning model and continuously improves its analysis accuracy by learning from past data. For example, it can learn from past traffic data and weather data to predict traffic patterns under specific conditions. As a result, the judgment unit can not only grasp the situation in real time but also make future predictions. Furthermore, the generation AI can use an anomaly detection algorithm to detect unusual patterns and abnormal movements early and issue warnings. This allows the judgment unit to improve intersection safety and reduce the risk of traffic accidents.

[0076] The control unit controls the traffic signals based on the surrounding conditions determined by the judgment unit. Specifically, if there are no vehicles or pedestrians at the intersection, it changes the signal to green to smooth the flow of traffic. In cases of heavy traffic, it adjusts the timing of the signals to minimize vehicle congestion. For example, on major arterial roads, it extends the green time of the signal to prioritize the flow of vehicles. On the other hand, if there are many pedestrians, it prioritizes pedestrian signals to allow them to cross safely. The control unit performs these controls in real time, enabling flexible responses according to traffic conditions. Furthermore, the control unit also has a function to prioritize the passage of emergency vehicles; when an emergency vehicle approaches the intersection, it immediately changes the signal to green to support its rapid passage. To perform these controls, the control unit is equipped with a processor with high processing power, enabling real-time signal control. As a result, the control unit can optimize the flow of traffic and improve the safety and efficiency of intersections.

[0077] The calculation unit calculates the cost of making vehicles wait at traffic lights controlled by the control unit. Specifically, it calculates the time vehicles are made to wait at red lights at intersections and evaluates fuel consumption and driver stress based on that time. For example, if vehicles are made to wait at red lights for a long time, fuel consumption increases and the environmental burden increases. Driver stress also increases, raising the risk of traffic accidents. The calculation unit comprehensively evaluates these factors and uses the results to optimize traffic signal control. Furthermore, the calculation unit evaluates not only the cost of waiting but also the effect of traffic flow improvement through traffic signal control. For example, adjusting the timing of traffic lights can reduce traffic congestion and improve the average speed of vehicles. This can lead to improved overall traffic efficiency, reduced travel time, and lower fuel consumption. Based on these evaluation results, the calculation unit adjusts the parameters of traffic signal control to achieve optimal traffic signal control. In this way, the calculation unit can optimize traffic flow and contribute to reducing environmental impact and driver stress.

[0078] The coordination unit coordinates traffic signals based on the waiting costs calculated by the calculation unit. Specifically, traffic signals on major arterial roads coordinate to adjust signal timing to smooth the flow of traffic. For example, on major arterial roads, traffic signals coordinate to create a wave of green lights, allowing vehicles to pass through intersections continuously. Traffic signals at intersections also coordinate to adjust signal timing to reduce traffic congestion. For example, if traffic signals at an intersection are heavy in a particular direction, they prioritize the signal for that direction. The coordination unit achieves this coordination by having traffic signals communicate with each other and share information in real time. Furthermore, the coordination unit also cooperates with other traffic management systems and infrastructure to perform comprehensive traffic management. For example, it cooperates with traffic information centers and emergency vehicle management systems to respond quickly in emergencies. In this way, the coordination unit can optimize traffic flow through the coordination of traffic signals, reduce traffic congestion, and support emergency response.

[0079] The acquisition unit can acquire camera images mounted on traffic lights. The acquisition unit can, for example, use the camera mounted on the traffic light to photograph the situation at the intersection. The acquisition unit can, for example, adjust the camera's resolution and frame rate to acquire the optimal image. The acquisition unit can, for example, adjust the camera's field of view to grasp a wide area of ​​the situation. As a result, by acquiring camera images mounted on traffic lights, the surrounding situation can be understood. Some or all of the above processing in the acquisition unit may be performed using AI, for example, or without AI. For example, the acquisition unit can input the camera image mounted on the traffic light into a generating AI and have the generating AI perform the image acquisition.

[0080] The judgment unit can analyze camera images using a generating AI to determine the surrounding conditions. For example, the judgment unit can use the generating AI to analyze camera images and determine whether there are vehicles or pedestrians at an intersection. For example, the judgment unit can use the generating AI to analyze camera images and determine traffic volume and pedestrian movement. For example, the judgment unit can use the generating AI to analyze camera images and determine the weather and visibility. As a result, using the generating AI improves the accuracy of camera image analysis and allows for accurate determination of the surrounding conditions. Some or all of the above-described processes in the judgment unit are performed using the generating AI. For example, the judgment unit can input camera images into the generating AI and have the generating AI perform the determination of the surrounding conditions.

[0081] The control unit can control the traffic signals based on the determined surrounding conditions. For example, the control unit changes the signal to green if there are no vehicles or pedestrians at the intersection. For example, the control unit adjusts the timing of the signals if there is heavy traffic. For example, the control unit prioritizes pedestrian signals if there are many pedestrians. In this way, traffic flow can be optimized by controlling the signals based on the surrounding conditions. Some or all of the above processing in the control unit may be performed using AI or not. For example, the control unit can input the determined surrounding conditions into a generating AI and have the generating AI perform the signal control.

[0082] The calculation unit can calculate the cost of making vehicles wait at traffic lights. For example, the calculation unit calculates the amount of time a vehicle is made to wait at a red light at an intersection. For example, the calculation unit calculates fuel consumption based on the waiting time. For example, the calculation unit calculates driver stress based on the waiting time. By calculating the cost of waiting, the efficiency of traffic light control can be improved. Some or all of the above processing in the calculation unit may be performed using AI or not. For example, the calculation unit can have a generating AI perform the calculation of the cost of waiting.

[0083] The coordination unit allows traffic lights to coordinate with each other based on the calculated waiting cost. For example, the coordination unit allows traffic lights on major arterial roads to coordinate and adjust the timing of signals to smooth the flow of traffic. For example, the coordination unit allows traffic lights at intersections to coordinate and reduce traffic congestion. For example, the coordination unit allows traffic lights to communicate with each other and share information in real time. This allows the overall traffic flow to be optimized through the coordination of traffic lights. Some or all of the above processes in the coordination unit may be performed using AI or not. For example, the coordination unit can have a generating AI perform the coordination of traffic lights.

[0084] The control unit can reduce waiting times at intersections. For example, the control unit changes the traffic light to green when there are no vehicles or pedestrians at the intersection. For example, the control unit adjusts the timing of the traffic lights when there is heavy traffic. For example, the control unit prioritizes pedestrian signals when there are many pedestrians. This reduces waiting times at intersections and makes traffic flow smoother. Some or all of the above-described processes in the control unit may be performed using AI or not. For example, the control unit can have a generating AI perform the task of reducing waiting times at intersections.

[0085] The control unit can reduce traffic congestion. For example, the control unit can coordinate traffic lights on major arterial roads and adjust the timing of signals to smooth the flow of vehicles. For example, the control unit can coordinate traffic lights at intersections to reduce traffic congestion. For example, the control unit can have traffic lights communicate with each other and share information in real time. This can smooth the flow of traffic by reducing traffic congestion. Some or all of the above processing in the control unit may be performed using AI or not. For example, the control unit can have a generating AI perform the reduction of traffic congestion.

[0086] The control unit can reduce fuel consumption. For example, the control unit can reduce fuel consumption by shortening waiting times at intersections. For example, the control unit can reduce fuel consumption by reducing traffic congestion. For example, the control unit can reduce fuel consumption by adjusting the timing of traffic signals. By reducing fuel consumption, the environmental burden can be reduced. Some or all of the above-described processes in the control unit may be performed using AI or not. For example, the control unit can have a generating AI perform the reduction of fuel consumption.

[0087] The control unit can implement traffic signal control that is safe for pedestrians. For example, the control unit prioritizes pedestrian signals when there are many pedestrians. For example, the control unit detects pedestrian movement and changes the signal to green. For example, the control unit changes signals preferentially based on pedestrian location information. This reduces the risk of traffic accidents by enabling traffic signal control that is safe for pedestrians. Some or all of the above-described processes in the control unit may be performed using AI or not. For example, the control unit can have a generating AI execute traffic signal control that is safe for pedestrians.

[0088] The acquisition unit can estimate the user's emotions and adjust the timing of camera image acquisition based on the estimated user emotions. For example, if the user is stressed, the acquisition unit increases the frequency of camera image acquisition to grasp the situation in more detail. For example, if the user is relaxed, the acquisition unit decreases the frequency of camera image acquisition to reduce the system load. For example, if the user is in a hurry, the acquisition unit adjusts the timing of camera image acquisition in real time to enable a quick response. This allows for a more appropriate understanding of the situation by adjusting the timing of camera image acquisition based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the acquisition unit may be performed using AI or not. For example, the acquisition unit can have a generative AI perform the estimation of the user's emotions.

[0089] The acquisition unit can dynamically adjust the camera's field of view to acquire the optimal image. For example, the acquisition unit widens the camera's field of view to grasp the overall situation depending on the congestion level of an intersection. For example, the acquisition unit narrows the camera's field of view to acquire a detailed image in order to focus on a specific vehicle or pedestrian. For example, the acquisition unit adjusts the camera's field of view depending on the weather or time of day to acquire the optimal image. In this way, the optimal image can be acquired by dynamically adjusting the camera's field of view. Some or all of the above processing in the acquisition unit may be performed using AI or not. For example, the acquisition unit can have a generating AI perform the adjustment of the camera's field of view.

[0090] The acquisition unit can automatically change the resolution of the acquired image according to the surrounding conditions. For example, the acquisition unit can increase the image resolution to grasp detailed conditions depending on the congestion at an intersection. For example, the acquisition unit can increase the image resolution to improve visibility at night or in bad weather. For example, the acquisition unit can reduce the system load by lowering the image resolution under normal conditions. In this way, by automatically changing the image resolution, the optimal image can be acquired according to the situation. Some or all of the above processing in the acquisition unit may be performed using AI or not. For example, the acquisition unit can have a generating AI perform the change in image resolution.

[0091] The acquisition unit can estimate the user's emotions and determine the priority of images to acquire based on the estimated emotions. For example, if the user is stressed, the acquisition unit will prioritize acquiring images of important areas. For example, if the user is relaxed, the acquisition unit will acquire wide-angle images to grasp the overall situation. For example, if the user is in a hurry, the acquisition unit will prioritize acquiring images of specific areas to enable a quick response. In this way, important information can be acquired preferentially by prioritizing images based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the acquisition unit may be performed using AI or not. For example, the acquisition unit can have a generative AI perform the estimation of the user's emotions.

[0092] The acquisition unit can integrate images from multiple cameras to grasp a more detailed situation. For example, the acquisition unit can integrate images from multiple cameras at an intersection to grasp the overall situation. For example, the acquisition unit can integrate images from multiple cameras in a specific area to grasp a detailed situation. For example, the acquisition unit can integrate images from multiple cameras according to weather and time of day to grasp the optimal situation. This makes it possible to grasp a detailed situation by integrating images from multiple cameras. Some or all of the above processing in the acquisition unit may be performed using AI or not. For example, the acquisition unit can have a generating AI perform the integration of images from multiple cameras.

[0093] The acquisition unit can acquire images from above using a drone and combine them with images from a ground camera. For example, the acquisition unit can acquire images from above an intersection and combine them with images from a ground camera to understand the overall situation. For example, the acquisition unit can acquire images from above a specific area and combine them with images from a ground camera to understand the detailed situation. For example, the acquisition unit can acquire images from above depending on the weather and time of day and combine them with images from a ground camera to understand the optimal situation. In this way, by acquiring images from above and combining them with images from a ground camera, the overall situation can be understood. Some or all of the above processing in the acquisition unit may be performed using AI or not. For example, the acquisition unit can have a generating AI perform the acquisition of images from above.

[0094] The judgment unit can estimate the user's emotions and adjust the analysis algorithm based on the estimated emotions. For example, if the user is stressed, the judgment unit applies an algorithm that performs a rapid analysis. For example, if the user is relaxed, the judgment unit applies an algorithm that performs a detailed analysis. For example, if the user is in a hurry, the judgment unit applies a simplified analysis algorithm to enable a quick response. This allows for more appropriate analysis by adjusting the analysis algorithm based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the judgment unit may be performed using AI or not using AI. For example, the judgment unit can have a generative AI perform the estimation of the user's emotions.

[0095] The judgment unit can apply different analysis methods depending on the weather and time of day during analysis. For example, in rainy weather, the judgment unit applies a specific filter to perform analysis because visibility is reduced. For example, at night, the judgment unit applies an analysis method that is suitable for low-light environments. For example, during congested daytime hours, the judgment unit applies a method that performs rapid analysis to smooth traffic flow. In this way, the accuracy of the analysis is improved by applying analysis methods according to the weather and time of day. Some or all of the above processing in the judgment unit is performed using a generation AI. For example, the judgment unit can have the generation AI execute the application of analysis methods according to the weather and time of day.

[0096] The judgment unit can predict the current situation by referring to past data during analysis. For example, the judgment unit predicts and analyzes the current traffic situation based on past traffic data. For example, the judgment unit predicts and analyzes the current weather situation based on past weather data. For example, the judgment unit predicts and analyzes the current accident risk based on past accident data. In this way, by referring to past data, the current situation can be predicted more accurately. Some or all of the above processing in the judgment unit is performed using a generation AI. For example, the judgment unit can have the generation AI perform the referencing of past data and the prediction of the current situation.

[0097] The judgment unit can estimate the user's emotions and adjust the display method of the analysis results based on the estimated user emotions. For example, if the user is stressed, the judgment unit provides a simple and highly visible display method. For example, if the user is relaxed, the judgment unit provides a display method that includes detailed information. For example, if the user is in a hurry, the judgment unit provides a display method that gets straight to the point. By adjusting the display method of the analysis results based on the user's emotions, visibility is improved. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the judgment unit may be performed using AI or not using AI. For example, the judgment unit can have a generative AI perform the estimation of the user's emotions.

[0098] The judgment unit can analyze audio data and make a judgment while also considering the surrounding sound conditions. For example, the judgment unit can analyze audio data from an intersection and make a judgment by detecting vehicle engine sounds and horn sounds. For example, the judgment unit can analyze audio data of pedestrians and determine the presence of pedestrians. For example, the judgment unit can analyze ambient sounds and determine traffic conditions and weather conditions. In this way, by analyzing audio data, it becomes possible to make a judgment that also considers the surrounding sound conditions. Some or all of the above processing in the judgment unit is performed using a generation AI. For example, the judgment unit can have the generation AI perform the analysis of audio data.

[0099] The judgment unit can analyze environmental data such as ambient temperature and humidity to determine the overall situation. For example, the judgment unit analyzes ambient temperature data to determine factors affecting traffic conditions. For example, the judgment unit analyzes ambient humidity data to determine visibility and road surface conditions. For example, the judgment unit comprehensively analyzes environmental data to provide information for optimal signal control. By comprehensively analyzing environmental data, more accurate situation determination becomes possible. Some or all of the above processing in the judgment unit is performed using a generation AI. For example, the judgment unit can have the generation AI perform the analysis of environmental data.

[0100] The control unit can estimate the user's emotions and adjust the timing of signal control based on the estimated user emotions. For example, if the user is stressed, the control unit may change the signal to green earlier. For example, if the user is relaxed, the control unit may perform normal signal control. For example, if the user is in a hurry, the control unit may change the signal quickly to smooth the flow of traffic. In this way, traffic flow can be smoothed by adjusting the timing of signal control based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the control unit may be performed using AI or not using AI. For example, the control unit may have a generative AI perform the estimation of the user's emotions.

[0101] The control unit can detect the approach of an emergency vehicle and change the signal accordingly. For example, the control unit can detect the sound of an emergency vehicle's siren and change the signal to green. For example, the control unit can change the signal accordingly based on the location information of an emergency vehicle. For example, the control unit can predict the approach of an emergency vehicle and change the signal in advance. This allows for smoother passage of emergency vehicles by detecting their approach and changing the signal accordingly. Some or all of the above-described processes in the control unit may be performed using AI or not. For example, the control unit can have a generating AI perform the detection of approaching emergency vehicles and the preferential change of signals.

[0102] The control unit can apply algorithms that prioritize specific vehicles (e.g., public transport). For example, the control unit may change signals preferentially based on the location information of public transport. For example, the control unit may change signals preferentially considering the operating schedule of public transport. For example, the control unit may predict the approach of public transport and change signals in advance. This allows for smoother operation of public transport by prioritizing specific vehicles. Some or all of the above-described processes in the control unit may be performed using AI or not. For example, the control unit may have a generating AI perform the priority of specific vehicles.

[0103] The control unit can estimate the user's emotions and adjust the signal color and flashing pattern based on the estimated emotions. For example, if the user is stressed, the control unit brightens the signal color to improve visibility. For example, if the user is relaxed, the control unit maintains the normal signal color. For example, if the user is in a hurry, the control unit changes the signal flashing pattern to encourage a quick response. This improves visibility by adjusting the signal color and flashing pattern based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the control unit may be performed using AI or not. For example, the control unit can have a generative AI perform the estimation of the user's emotions.

[0104] The control unit can detect pedestrian movement and perform pedestrian-only signal control. For example, the control unit can detect pedestrian movement with a camera and change the signal to green. For example, the control unit can change the signal preferentially based on pedestrian location information. For example, the control unit can predict the approach of a pedestrian and change the signal in advance. This makes pedestrian-only signal control possible by detecting pedestrian movement. Some or all of the above processing in the control unit may be performed using AI or not. For example, the control unit can have a generating AI perform the detection of pedestrian movement and the change of the signal.

[0105] The control unit can optimize traffic flow by performing dedicated signal control for bicycles. For example, the control unit can detect the movement of a bicycle using a camera and change the signal to green. For example, the control unit can prioritize changing the signal based on the bicycle's location information. For example, the control unit can predict the approach of a bicycle and change the signal in advance. In this way, traffic flow can be optimized by performing dedicated signal control for bicycles. Some or all of the above processes in the control unit may be performed using AI or not. For example, the control unit can have a generating AI perform the detection of bicycle movement and the changing of the signal.

[0106] The calculation unit can estimate the user's emotions and adjust the method for calculating the waiting cost based on the estimated user emotions. For example, if the user is stressed, the calculation unit may set a high waiting cost to encourage a quick response. For example, if the user is relaxed, the calculation unit may set a low waiting cost and provide a normal response. For example, if the user is in a hurry, the calculation unit may set a very high waiting cost and provide top priority. This allows for more appropriate cost calculation by adjusting the method for calculating the waiting cost based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the calculation unit may be performed using AI or not. For example, the calculation unit may have a generative AI perform the estimation of the user's emotions.

[0107] The calculation unit can consider not only the vehicle's fuel consumption but also its exhaust emissions during calculations. For example, the calculation unit calculates the waiting cost based on the vehicle's fuel consumption and exhaust emissions. For example, the calculation unit sets a higher waiting cost for vehicles with high fuel consumption. For example, the calculation unit sets a higher waiting cost for vehicles with high exhaust emissions. By considering fuel consumption and exhaust emissions, the environmental impact can be reduced. Some or all of the above processing in the calculation unit may be performed using AI or not. For example, the calculation unit can have a generating AI perform the consideration of fuel consumption and exhaust emissions.

[0108] The calculation unit can perform different cost calculations depending on the type of vehicle (e.g., electric vehicle or hybrid vehicle) during the calculation process. For example, the calculation unit can set the waiting cost low for electric vehicles, medium for hybrid vehicles, and high for gasoline vehicles. This allows for more accurate cost calculations by performing cost calculations according to the type of vehicle. Some or all of the above processing in the calculation unit may be performed using AI or not. For example, the calculation unit can have a generating AI perform cost calculations according to the type of vehicle.

[0109] The calculation unit can estimate the user's emotions and adjust the display method for the waiting cost based on the estimated user emotions. For example, if the user is stressed, the calculation unit provides a simple and highly visible display method. For example, if the user is relaxed, the calculation unit provides a display method that includes detailed information. For example, if the user is in a hurry, the calculation unit provides a concise display method. This improves visibility by adjusting the display method for the waiting cost based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the calculation unit may be performed using AI or not. For example, the calculation unit can have a generative AI perform the estimation of the user's emotions.

[0110] The calculation unit can calculate costs while also considering the risk of traffic accidents. For example, the calculation unit calculates the cost of making people wait based on past accident data at an intersection. For example, if the risk of traffic accidents is high, the calculation unit sets a higher cost for making people wait. For example, if the risk of traffic accidents is low, the calculation unit sets a lower cost for making people wait. This makes it possible to control traffic signals more safely by considering the risk of traffic accidents. Some or all of the above processing in the calculation unit may be performed using AI or not. For example, the calculation unit can have a generating AI perform the consideration of the risk of traffic accidents.

[0111] The calculation unit can calculate costs while also considering the congestion levels of surrounding commercial facilities. For example, the calculation unit calculates the cost of making customers wait based on the congestion levels of surrounding commercial facilities. For example, if a commercial facility is crowded, the calculation unit sets a higher cost for making customers wait. For example, if a commercial facility is not crowded, the calculation unit sets a lower cost for making customers wait. This makes it possible to calculate costs more appropriately by considering the congestion levels of commercial facilities. Some or all of the above processing in the calculation unit may be performed using AI or not. For example, the calculation unit can have a generating AI perform the task of considering the congestion levels of commercial facilities.

[0112] The coordination unit can estimate the user's emotions and adjust the coordination method between traffic lights based on the estimated user emotions. For example, if the user is stressed, the coordination unit will coordinate quickly and change the light to green earlier. For example, if the user is relaxed, the coordination unit will apply the normal coordination method. For example, if the user is in a hurry, the coordination unit will prioritize coordination and change the light quickly. This allows for smoother traffic flow by adjusting the coordination method between traffic lights based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the coordination unit may be performed using AI or not. For example, the coordination unit can have a generative AI perform the estimation of the user's emotions.

[0113] The coordinating unit can apply algorithms that prioritize traffic flow on major arterial roads. For example, the coordinating unit prioritizes traffic flow on major arterial roads and changes traffic lights to green. For example, the coordinating unit adjusts the timing of traffic lights based on the traffic volume on arterial roads. For example, the coordinating unit predicts traffic congestion on arterial roads and changes traffic lights in advance. This reduces traffic congestion by prioritizing traffic flow on major arterial roads. Some or all of the above processing in the coordinating unit may be performed using AI or not. For example, the coordinating unit can have a generating AI perform the priority of traffic flow on arterial roads.

[0114] The coordinating unit can adjust signal control according to specific events (e.g., sporting events or concerts). For example, the coordinating unit may prioritize the surrounding traffic flow when a sporting event is held. For example, the coordinating unit may prioritize the surrounding traffic flow when a concert is held. For example, the coordinating unit may predict traffic congestion and change signals in advance when a specific event is held. This allows for smoother traffic flow by providing signal control tailored to specific events. Some or all of the above processing in the coordinating unit may be performed using AI or not. For example, the coordinating unit may have a generating AI execute signal control tailored to specific events.

[0115] The integration unit can estimate the user's emotions and adjust the display method of the integration results based on the estimated user emotions. For example, if the user is stressed, the integration unit provides a simple and highly visible display method. For example, if the user is relaxed, the integration unit provides a display method that includes detailed information. For example, if the user is in a hurry, the integration unit provides a display method that gets straight to the point. By adjusting the display method of the integration results based on the user's emotions, visibility is improved. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the integration unit may be performed using AI or not. For example, the integration unit can have a generative AI perform the estimation of the user's emotions.

[0116] The integration unit can perform comprehensive traffic management in conjunction with other traffic management systems (e.g., parking management systems). For example, the integration unit can integrate with a parking management system to control traffic signals based on parking availability. For example, the integration unit can integrate with the operating status of public transportation to optimize traffic signal control. For example, the integration unit can integrate with other traffic management systems to perform comprehensive traffic management. This enables comprehensive traffic management by integrating with other traffic management systems. Some or all of the above-described processes in the integration unit may be performed using AI or not. For example, the integration unit can have a generating AI perform the integration with other traffic management systems.

[0117] The coordinating unit can optimize signal control in conjunction with the operating status of public transportation. For example, the coordinating unit optimizes signal control based on the operating status of public transportation. For example, the coordinating unit adjusts signal control considering delay information of public transportation. For example, the coordinating unit optimizes signal control in conjunction with the operating schedule of public transportation. In this way, signal control can be optimized by coordinating with the operating status of public transportation. Some or all of the above processing in the coordinating unit may be performed using AI or not. For example, the coordinating unit can have a generating AI perform the coordination with the operating status of public transportation.

[0118] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0119] The signal control system can also be equipped with a voice recognition unit. This unit can acquire and analyze audio data from around the intersection. For example, it can detect vehicle engine noises and horn sounds to estimate traffic volume. It can also detect pedestrian voices and footsteps to confirm their presence. Furthermore, the voice recognition unit can detect emergency vehicle sirens and prioritize the change of traffic signals accordingly. This allows for a more accurate understanding of traffic conditions by utilizing audio data.

[0120] The signal control system can also be equipped with an environmental sensor unit. This unit can acquire and analyze environmental data such as ambient temperature, humidity, and atmospheric pressure. For example, the environmental sensor unit can shorten signal waiting times when the temperature is high, preventing vehicle engine overheating. It can also adjust signal timing considering road surface slipperiness when humidity is high. Furthermore, it can detect changes in atmospheric pressure and perform signal control to respond to sudden weather changes. This allows for safer and more efficient signal control by utilizing environmental data.

[0121] The traffic signal control system can also be equipped with a drone unit. The drone unit can acquire aerial footage and analyze it in combination with ground-based camera images. For example, the drone unit can acquire aerial footage of an intersection to understand the overall traffic situation. It can also fly a drone to acquire real-time footage to check congestion in a specific area in detail. Furthermore, the drone unit can quickly check the situation on site in the event of an emergency and implement appropriate traffic signal control. This makes it possible to understand traffic conditions more accurately and over a wider area by utilizing an aerial perspective.

[0122] The traffic signal control system can also be equipped with a user feedback unit. This unit can collect feedback from drivers and pedestrians and incorporate it into the signal control. For example, the user feedback unit can collect opinions and requests from drivers via a smartphone app and adjust the timing of the signals. It can also change the priority of pedestrian signals based on feedback from pedestrians. Furthermore, the user feedback unit can analyze the collected feedback and use it to improve the traffic signal control system. This allows for a more user-friendly traffic signal control system by incorporating user feedback.

[0123] The traffic signal control system can also be equipped with a prediction unit. This unit can predict current traffic conditions based on past traffic data and reflect this in traffic signal control. For example, the prediction unit can analyze past traffic volume data to predict fluctuations in traffic volume during specific time periods or on specific days of the week. It can also predict changes in traffic conditions due to weather changes based on past weather data. Furthermore, the prediction unit can predict the risk of accidents based on past accident data and reflect this in traffic signal control. This allows for more accurate prediction of traffic conditions and traffic signal control by utilizing past data.

[0124] The traffic signal control system can further estimate the user's emotions and control the signals based on those emotions. For example, if the user is stressed, the signal can be changed to green earlier to reduce waiting times. If the user is relaxed, the system can use normal signal control. Furthermore, if the user is in a hurry, the signal can be changed quickly to smooth the flow of traffic. In this way, a more comfortable traffic environment can be provided by controlling signals based on the user's emotions. Emotion estimation can be achieved using, for example, an emotion engine or generative AI.

[0125] The signal control system can further estimate the user's emotions and adjust the timing of camera image acquisition based on the estimated emotions. For example, if the user is stressed, the frequency of camera image acquisition can be increased to gain a more detailed understanding of the situation. Conversely, if the user is relaxed, the frequency of camera image acquisition can be decreased to reduce the system load. Furthermore, if the user is in a hurry, the timing of camera image acquisition can be adjusted in real time to enable a quick response. In this way, adjusting the timing of camera image acquisition based on the user's emotions allows for a more appropriate understanding of the situation. Emotion estimation can be achieved using, for example, an emotion engine or generative AI.

[0126] The signal control system can further estimate the user's emotions and adjust the analysis algorithm based on those emotions. For example, if the user is stressed, a rapid analysis algorithm can be applied. If the user is relaxed, a more detailed analysis algorithm can be applied. Furthermore, if the user is in a hurry, a simplified analysis algorithm can be applied to enable a quick response. This allows for more appropriate analysis by adjusting the analysis algorithm based on the user's emotions. Emotion estimation can be achieved using, for example, an emotion engine or generative AI.

[0127] The signal control system can further estimate the user's emotions and adjust the signal color and flashing pattern based on those emotions. For example, if the user is stressed, the signal color can be brightened to improve visibility. If the user is relaxed, the signal color can be maintained at its normal level. Furthermore, if the user is in a hurry, the signal flashing pattern can be changed to encourage a quicker response. This improves visibility by adjusting the signal color and flashing pattern based on the user's emotions. Emotion estimation can be achieved using, for example, an emotion engine or generative AI.

[0128] The signal control system can further estimate the user's emotions and adjust the display method of the interaction results based on the estimated emotions. For example, if the user is stressed, a simple and highly visible display method can be provided. If the user is relaxed, a display method including detailed information can be provided. Furthermore, if the user is in a hurry, a display method that gets straight to the point can be provided. In this way, visibility is improved by adjusting the display method of the interaction results based on the user's emotions. Emotion estimation is achieved, for example, using an emotion engine or generative AI.

[0129] The following briefly describes the processing flow for example form 2.

[0130] Step 1: The acquisition unit acquires images from a camera mounted on a traffic light. For example, the acquisition unit uses the camera mounted on the traffic light to photograph the situation at the intersection and adjusts the camera's resolution and frame rate to acquire the optimal image. It can also adjust the camera's field of view to grasp a wide area of ​​the situation. Step 2: The judgment unit uses a generation AI to analyze the camera images acquired by the acquisition unit and determine the surrounding conditions. For example, the generation AI analyzes the camera images to determine whether there are vehicles or pedestrians at the intersection, the volume of traffic, pedestrian movement, weather, and visibility. Step 3: The control unit controls the signals based on the surrounding conditions determined by the determination unit. For example, if there are no vehicles or pedestrians at the intersection, the signal is changed to green; if there is heavy traffic, the timing of the signals is adjusted; and if there are many pedestrians, pedestrian signals are given priority. Step 4: The calculation unit calculates the cost of waiting for signals controlled by the control unit. For example, it calculates the time a vehicle is made to wait at a red light at an intersection, the fuel consumption based on the waiting time, and the stress on the driver. Step 5: The coordination unit coordinates traffic signals based on the waiting costs calculated by the calculation unit. For example, traffic signals on major arterial roads coordinate to adjust signal timing to smooth the flow of traffic, and traffic signals at intersections coordinate to reduce traffic congestion. In addition, traffic signals communicate with each other and share information in real time.

[0131] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0132] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0133] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0134] Each of the multiple elements described above, including the acquisition unit, determination unit, control unit, calculation unit, and coordination unit, is implemented in at least one of the smart device 14 and the data processing device 12. For example, the acquisition unit uses the camera 42 of the smart device 14 to photograph the situation at the intersection, and the control unit 46A acquires the image. The determination unit is implemented in the specific processing unit 290 of the data processing device 12, for example, and the generating AI analyzes the camera image to determine the surrounding situation. The control unit is implemented in the specific processing unit 290 of the data processing device 12, for example, and controls the signal based on the determined situation. The calculation unit is implemented in the specific processing unit 290 of the data processing device 12, for example, and calculates the cost of making people wait. The coordination unit is implemented in the control unit 46A of the smart device 14, for example, and the traffic lights cooperate to optimize the traffic flow. The correspondence between each unit and the device or control unit is not limited to the example described above, and various changes are possible.

[0135] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0136] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0137] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0138] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0139] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0140] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0141] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0142] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0143] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0144] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0145] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0146] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0147] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0148] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0149] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0150] Each of the multiple elements described above, including the acquisition unit, determination unit, control unit, calculation unit, and coordination unit, is implemented, for example, in at least one of the smart glasses 214 and the data processing unit 12. For example, the acquisition unit uses the camera 42 of the smart glasses 214 to photograph the situation at the intersection, and the control unit 46A acquires the image. The determination unit is implemented, for example, in the identification processing unit 290 of the data processing unit 12, where the generating AI analyzes the camera image to determine the surrounding situation. The control unit is implemented, for example, in the identification processing unit 290 of the data processing unit 12, and controls the signal based on the determined situation. The calculation unit is implemented, for example, in the identification processing unit 290 of the data processing unit 12, and calculates the cost of making people wait. The coordination unit is implemented, for example, in the control unit 46A of the smart glasses 214, where the traffic lights cooperate to optimize the traffic flow. The correspondence between each unit and the device or control unit is not limited to the example described above, and various changes are possible.

[0151] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0152] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0153] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0154] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0155] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0156] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0157] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0158] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0159] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0160] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0161] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0162] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0163] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0164] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0165] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0166] Each of the multiple elements described above, including the acquisition unit, determination unit, control unit, calculation unit, and coordination unit, is implemented in at least one of the headset terminal 314 and the data processing unit 12. For example, the acquisition unit uses the camera 42 of the headset terminal 314 to photograph the situation at the intersection, and the control unit 46A acquires the image. The determination unit is implemented in the specific processing unit 290 of the data processing unit 12, and the generating AI analyzes the camera image to determine the surrounding situation. The control unit is implemented in the specific processing unit 290 of the data processing unit 12, and controls the signal based on the determined situation. The calculation unit is implemented in the specific processing unit 290 of the data processing unit 12, and calculates the cost of making people wait. The coordination unit is implemented in the specific processing unit 46A of the headset terminal 314, and the traffic lights cooperate to optimize the traffic flow. The correspondence between each unit and the device or control unit is not limited to the example described above, and various changes are possible.

[0167] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0168] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0169] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0170] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0171] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0172] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0173] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0174] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0175] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0176] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0177] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0178] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0179] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0180] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0181] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0182] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0183] Each of the multiple elements described above, including the acquisition unit, determination unit, control unit, calculation unit, and coordination unit, is implemented in at least one of the following: the robot 414 and the data processing unit 12. For example, the acquisition unit uses the camera 42 of the robot 414 to photograph the situation at the intersection, and the control unit 46A acquires the image. The determination unit is implemented in the specific processing unit 290 of the data processing unit 12, and the generating AI analyzes the camera image to determine the surrounding situation. The control unit is implemented in the specific processing unit 290 of the data processing unit 12, and controls the signal based on the determined situation. The calculation unit is implemented in the specific processing unit 290 of the data processing unit 12, and calculates the cost of making people wait. The coordination unit is implemented in the control unit 46A of the robot 414, and the traffic lights cooperate to optimize the traffic flow. The correspondence between each unit and the devices and control units is not limited to the example described above, and various changes are possible.

[0184] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0185] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0186] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0187] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0188] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0189] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0190] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0191] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0192] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0193] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0194] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0195] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0196] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0197] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0198] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0199] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0200] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0201] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0202] (Note 1) An acquisition unit that acquires camera images attached to a traffic light, A determination unit analyzes the camera image acquired by the acquisition unit and determines the surrounding conditions, A control unit that controls a signal based on the surrounding conditions determined by the determination unit, A calculation unit that calculates the cost of delaying the signal controlled by the control unit, The system includes a coordination unit that coordinates traffic signals based on the waiting cost calculated by the aforementioned calculation unit. A system characterized by the following features. (Note 2) The acquisition unit is, Acquire camera images attached to traffic lights. The system described in Appendix 1, characterized by the features described herein. (Note 3) The determination unit, The AI ​​generates images from the camera to determine the surrounding environment. The system described in Appendix 1, characterized by the features described herein. (Note 4) The control unit, The signal is controlled based on the determined surrounding conditions. The system described in Appendix 1, characterized by the features described herein. (Note 5) The calculation unit, Calculate the cost of making the signal wait. The system described in Appendix 1, characterized by the features described herein. (Note 6) The aforementioned linkage unit is, Traffic lights coordinate with each other based on the calculated waiting cost. The system described in Appendix 1, characterized by the features described herein. (Note 7) The control unit, Reduce waiting times at intersections. The system described in Appendix 1, characterized by the features described herein. (Note 8) The control unit, Reduce traffic congestion The system described in Appendix 1, characterized by the features described herein. (Note 9) The control unit, Reduce fuel consumption The system described in Appendix 1, characterized by the features described herein. (Note 10) The control unit, To enable safe traffic signal control for pedestrians. The system described in Appendix 1, characterized by the features described herein. (Note 11) The acquisition unit is, The system estimates the user's emotions and adjusts the timing of camera image acquisition based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 12) The acquisition unit is, The camera's field of view is dynamically adjusted to obtain the optimal image. The system described in Appendix 1, characterized by the features described herein. (Note 13) The acquisition unit is, The resolution of the images to be acquired is automatically changed according to the surrounding environment. The system described in Appendix 1, characterized by the features described herein. (Note 14) The acquisition unit is, It estimates the user's emotions and determines the priority of images to retrieve based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 15) The acquisition unit is, By integrating images from multiple cameras, a more detailed understanding of the situation can be obtained. The system described in Appendix 1, characterized by the features described herein. (Note 16) The acquisition unit is, A drone is used to acquire images from above, which are then combined with images from ground-based cameras. The system described in Appendix 1, characterized by the features described herein. (Note 17) The determination unit, It estimates the user's emotions and adjusts the analysis algorithm based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 18) The determination unit, During the analysis, different analytical methods are applied depending on the weather and time of day. The system described in Appendix 1, characterized by the features described herein. (Note 19) The determination unit, During analysis, past data is referenced to predict the current situation. The system described in Appendix 1, characterized by the features described herein. (Note 20) The determination unit, It estimates the user's emotions and adjusts how the analysis results are displayed based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 21) The determination unit, The system analyzes audio data and makes decisions while also considering the surrounding sound environment. The system described in Appendix 1, characterized by the features described herein. (Note 22) The determination unit, Environmental data such as ambient temperature and humidity are also analyzed to determine the overall situation. The system described in Appendix 1, characterized by the features described herein. (Note 23) The control unit, It estimates the user's emotions and adjusts the timing of signal control based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 24) The control unit, The system detects the approach of emergency vehicles and prioritizes changing the traffic signals accordingly. The system described in Appendix 1, characterized by the features described herein. (Note 25) The control unit, Apply an algorithm that prioritizes specific vehicles. The system described in Appendix 1, characterized by the features described herein. (Note 26) The control unit, It estimates the user's emotions and adjusts the signal color and flashing pattern based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 27) The control unit, It detects pedestrian movement and controls traffic signals specifically for pedestrians. The system described in Appendix 1, characterized by the features described herein. (Note 28) The control unit, Implementing dedicated traffic signal control for bicycles optimizes traffic flow. The system described in Appendix 1, characterized by the features described herein. (Note 29) The calculation unit, We estimate the user's emotions and adjust the calculation method for the cost of making users wait based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 30) The calculation unit, When calculating, the vehicle's fuel consumption and exhaust emissions are both taken into consideration. The system described in Appendix 1, characterized by the features described herein. (Note 31) The calculation unit, During the calculation, different cost calculations are performed depending on the type of vehicle. The system described in Appendix 1, characterized by the features described herein. (Note 32) The calculation unit, We estimate the user's emotions and adjust how the waiting cost is displayed based on those emotions. The system described in Appendix 1, characterized by the features described herein. (Note 33) The calculation unit, The cost is calculated taking into account the risk of traffic accidents. The system described in Appendix 1, characterized by the features described herein. (Note 34) The calculation unit, The cost is calculated taking into account the congestion levels of surrounding commercial facilities. The system described in Appendix 1, characterized by the features described herein. (Note 35) The aforementioned linkage unit is, It estimates the user's emotions and adjusts the way traffic lights interact based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 36) The aforementioned linkage unit is, Apply an algorithm that prioritizes traffic flow on major arterial roads. The system described in Appendix 1, characterized by the features described herein. (Note 37) The aforementioned linkage unit is, Adjust signal control in response to specific events. The system described in Appendix 1, characterized by the features described herein. (Note 38) The aforementioned linkage unit is, It estimates the user's emotions and adjusts how the collaboration results are displayed based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 39) The aforementioned linkage unit is, It integrates with other traffic management systems to provide comprehensive traffic management. The system described in Appendix 1, characterized by the features described herein. (Note 40) The aforementioned linkage unit is, Optimizing signal control in conjunction with the operating status of public transportation. The system described in Appendix 1, characterized by the features described herein. [Explanation of Symbols]

[0203] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. An acquisition unit that acquires camera images attached to a traffic light, A determination unit analyzes the camera image acquired by the acquisition unit and determines the surrounding conditions, A control unit that controls a signal based on the surrounding conditions determined by the determination unit, A calculation unit that calculates the cost of delaying the signal controlled by the control unit, The system includes a coordination unit that coordinates traffic signals based on the waiting cost calculated by the aforementioned calculation unit. A system characterized by the following features.

2. The acquisition unit is, Acquire camera images attached to traffic lights. The system according to feature 1.

3. The determination unit, The AI ​​generates images from the camera to determine the surrounding environment. The system according to feature 1.

4. The control unit, The signal is controlled based on the determined surrounding conditions. The system according to feature 1.

5. The calculation unit, Calculate the cost of making the signal wait. The system according to feature 1.

6. The aforementioned linkage unit is, Traffic lights coordinate with each other based on the calculated waiting cost. The system according to feature 1.

7. The control unit, Reduce waiting times at intersections. The system according to feature 1.

8. The control unit, Reduce traffic congestion The system according to feature 1.

Images