Regional traffic signal cooperative control method and system based on multi-agent reinforcement learning

Abstract

The application discloses a regional traffic signal cooperative control method and system based on multi-agent reinforcement learning, and relates to the technical field of intelligent traffic control. The method comprises the following steps: acquiring lamp pole configuration information and a preset multi-agent cooperative strategy, wherein the lamp pole configuration information records lamp pole identifiers and lamp pole positions of intelligent lamp poles in a preset region; a plurality of devices are integrated on each intelligent lamp pole, and the plurality of devices are respectively abstracted as corresponding sub-agents; determining a required lamp pole level to be activated in the preset region according to a current time period and / or a scene type; and selecting a target intelligent lamp pole corresponding to the lamp pole level in the lamp pole configuration information based on the multi-agent cooperative strategy, and activating all or part of the sub-agents on the target intelligent lamp pole. The application realizes efficient cooperative management of regional traffic signals by applying multi-agent reinforcement learning technology to intelligent lamp pole cooperative control.

Description

Technical Field

[0001] This invention relates to the field of intelligent traffic control technology, and more specifically, to a method and system for regional traffic signal cooperative control based on multi-agent reinforcement learning. Background Technology

[0002] With the accelerated development of smart city construction, smart streetlights, as a core component of urban road infrastructure, have gradually evolved into comprehensive roadside intelligent platforms integrating various devices such as lighting equipment, traffic signal equipment, environmental sensors, AI cameras, broadcasting equipment, LED displays, and one-button alarm systems. Simultaneously, the continuous growth of urban traffic flow has placed higher demands on the intelligence and precision of regional traffic signal control. Multi-agent reinforcement learning technology, due to its inherent advantages in multi-device collaborative decision-making, has been gradually explored and applied in the field of traffic signal control in recent years, providing a new technological path for urban traffic management.

[0003] However, in existing traffic signal control schemes based on multi-agent reinforcement learning, the control granularity of the agents is only at the intersection level, using the entire intersection as the decision-making unit. This fails to utilize fine-grained sensing data at the light pole level, resulting in a lack of precise micro-information supporting traffic flow for signal timing decisions. Furthermore, the various devices integrated on smart light poles operate independently for extended periods, lacking collaborative interaction mechanisms and thus failing to form joint decision-making within the light pole. In addition, existing technologies isolate traffic efficiency optimization from equipment energy consumption optimization, lacking a unified multi-objective optimization framework. Moreover, the centralized cloud-based decision-making mode suffers from high communication overhead and response latency, making it difficult to meet the stringent real-time requirements of traffic signal control. To address these issues, this invention abstracts multiple devices at the light pole level into corresponding sub-agents and constructs a hierarchical decision-making mechanism, achieving collaborative decision-making among multiple devices deployed within the light pole and joint optimization control of regional traffic signals. Summary of the Invention

[0004] To overcome the problems of existing technologies, such as coarse-grained agents that cannot utilize fine-grained sensing data at the light pole level, lack of coordination in independent operation of multiple devices, disconnect between traffic efficiency and energy consumption optimization, and large delays in cloud-based decision-making, this invention discloses a regional traffic signal cooperative control method and system based on multi-agent reinforcement learning, which can effectively solve the above-mentioned technical problems.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: A regional traffic signal cooperative control method based on multi-agent reinforcement learning includes: The system acquires light pole configuration information and a preset multi-agent collaborative strategy. The light pole configuration information records the light pole identifier and light pole position of each smart light pole in a preset area. Each smart light pole integrates multiple devices, and the multiple devices are abstracted into corresponding sub-agents. Based on the current time period and / or scene type, determine the required level of light poles to be activated in the preset area; Based on the multi-agent collaborative strategy, the target smart light pole corresponding to the light pole level is selected in the light pole configuration information, and all or some of the sub-agents on the target smart light pole are activated; The multi-agent collaborative strategy includes a hierarchical decision-making mechanism within the light pole, which includes: In the real-time response layer, latency-sensitive tasks are decided independently by each sub-agent, and the response time is less than or equal to the first preset time. In the collaborative optimization layer, for tasks requiring multi-device collaboration, the edge gateway of the light pole initiates joint negotiation among sub-agents, and a consensus-based distributed decision-making algorithm is used to reach a consensus within a time frame of less than or equal to the second preset time. At the global planning layer, tasks that require coordination with adjacent light poles or intersection coordinators are reported from the light pole edge gateway to the intersection coordinator.

[0006] Further, determining the required light pole hierarchy to be activated in the preset area based on the current time period and / or scene type includes: When the current time is the set intersection coordination trigger time, the number of light poles to be activated in the next time period will be used as the number of light poles corresponding to the required light pole level.

[0007] Furthermore, before acquiring the light pole configuration information and the preset multi-agent collaborative strategy, the following steps are also included: The multi-agent cooperative strategy is set, which includes a fault-priority selection mechanism and a location-priority selection mechanism.

[0008] Furthermore, when the multi-agent collaborative strategy is a fault-priority selection mechanism, the multi-agent collaborative strategy records the fault weight of each smart light pole. The step of selecting the target smart light pole corresponding to the light pole level in the light pole configuration information based on the multi-agent collaborative strategy includes: Based on the pole location and the fault weight, the target smart pole corresponding to the pole level is selected in the pole configuration information.

[0009] Further, the step of selecting the target smart light pole corresponding to the light pole level in the light pole configuration information based on the light pole location and the fault weight includes: Based on the lamp post hierarchy, the same number of lamp post position groups are determined, and each lamp post position group includes multiple lamp post positions; In each group of light pole locations, the smart light pole with the highest fault weight is selected as the target smart light pole.

[0010] Furthermore, when the multi-agent collaborative strategy is a location-priority selection mechanism, the multi-agent collaborative strategy records the location weight of each smart light pole. The step of selecting the target smart light pole corresponding to the light pole level in the light pole configuration information based on the multi-agent collaborative strategy includes: Based on the pole position and the position weight, the target smart pole corresponding to the pole level is selected in the pole configuration information.

[0011] Further, the step of selecting the target smart light pole corresponding to the light pole level in the light pole configuration information based on the light pole position and the position weight includes: Based on the position weight, smart light poles are selected sequentially from high to low for non-adjacent positions until the number of light poles corresponding to the light pole level is reached.

[0012] Furthermore, the smart light pole integrates multiple devices including lighting equipment, traffic signal equipment, environmental sensors, AI cameras, broadcasting equipment, LED displays, and one-button alarm buttons; the sub-intelligent agents include lighting sub-intelligent agents, signal control sub-intelligent agents, environmental perception sub-intelligent agents, AI event monitoring sub-intelligent agents, information dissemination sub-intelligent agents, and one-button alarm sub-intelligent agents.

[0013] A regional traffic signal cooperative control system based on multi-agent reinforcement learning includes: The information acquisition module is used to acquire light pole configuration information and preset multi-agent collaborative strategies. The light pole configuration information records the light pole identifier and light pole position of each smart light pole in the preset area. Each smart light pole integrates multiple devices, and the multiple devices are abstracted into corresponding sub-agents. The level determination module is used to determine the required level of light poles to be activated in the preset area based on the current time period and / or scene type. The light pole selection module is used to select the target smart light pole corresponding to the light pole level in the light pole configuration information based on the multi-agent collaborative strategy, and activate all or some of the sub-agents on the target smart light pole.

[0014] Compared with existing technologies, the beneficial effects of this invention are as follows: By acquiring light pole configuration information and a preset multi-agent collaborative strategy, this invention abstracts the lighting equipment, traffic signal equipment, environmental sensors, AI cameras, broadcasting equipment, LED displays, and one-button alarms integrated on each smart light pole into lighting sub-agents, signal control sub-agents, environmental perception sub-agents, AI event monitoring sub-agents, information dissemination sub-agents, and one-button alarm sub-agents, respectively. Because a fine-grained multi-agent system architecture at the light pole level is established, each sub-agent can independently perceive the environmental state at the light pole level and... By making local decisions, the control granularity of the intelligent agents is refined from the intersection level to the light pole-device level, effectively solving the technical challenge of not being able to utilize fine-grained sensing data at the light pole level, and providing a more accurate micro-decision basis for traffic signal timing. Furthermore, because each sub-agent communicates and negotiates internally within the light pole through the light pole edge gateway, a hierarchical decision-making mechanism is formed within the light pole. For latency-sensitive tasks in the real-time response layer, each sub-agent makes independent decisions with a response time less than or equal to a first preset time. For tasks requiring multi-device collaboration in the collaborative optimization layer, the light pole edge gateway initiates joint coordination among sub-agents. The system employs a consensus-based distributed decision-making algorithm to reach agreement within a second preset timeframe. The global planning layer reports tasks requiring coordination with adjacent light poles or intersection coordinators to the intersection coordinator via the light pole edge gateway. This allows emergency tasks to be completed at the light pole edge without waiting for instructions from the intersection coordinator or the cloud, keeping collaborative decision-making delays within a preset timeframe. Simultaneously, it enables unified scheduling and coordinated response for previously isolated functions such as lighting, signal control, environmental perception, event monitoring, information dissemination, and emergency alarms, overcoming the lack of coordination inherent in independent multi-device operation and reducing costs. This reduces communication overhead and response latency, meeting the real-time requirements of traffic signal control. Furthermore, since the light pole level is dynamically determined based on the current time period and / or scenario type, and the multi-agent collaborative strategy incorporates fault-priority selection and location-priority selection mechanisms, along with scenario-adaptive dynamic weight adjustment, traffic efficiency optimization and equipment energy consumption optimization are integrated into a unified light pole-level multi-objective optimization framework. This allows for adaptive adjustment of optimization focus based on different time periods and scenarios, ultimately improving the regional traffic signal collaborative response speed, overall energy efficiency, and traffic flow efficiency. Attached Figure Description

[0015] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary. For those skilled in the art, other embodiments can be derived from the provided drawings without creative effort.

[0016] Figure 1 A flowchart of a regional traffic signal cooperative control method based on multi-agent reinforcement learning provided in an embodiment of this application; Figure 2 This is a structural diagram of a regional traffic signal cooperative control system based on multi-agent reinforcement learning, provided in an embodiment of this application. Detailed Implementation

[0017] The following is in conjunction with the appendix Figures 1 to 2 The embodiments further illustrate the regional traffic signal cooperative control method and system based on multi-agent reinforcement learning provided in this application.

[0018] It is understood that the specific embodiments described herein are merely illustrative of the embodiments of this application and are not intended to limit the embodiments of this application. Furthermore, it should be noted that, for ease of description, the accompanying drawings only show the parts related to the embodiments of this application, not all structures. Those skilled in the art, after reading this specification, should be able to realize that any combination of technical features can constitute an optional implementation method, provided that the technical features do not contradict each other.

[0019] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. In the description of this application, "multiple" means two or more, and "several" means one or more.

[0020] The same or similar reference numerals correspond to the same or similar parts; the terms used in the accompanying drawings to describe positional relationships are for illustrative purposes only and should not be construed as limiting this patent.

[0021] Figure 1 A flowchart of a regional traffic signal cooperative control method based on multi-agent reinforcement learning provided in an embodiment of this application. Figure 2 This is a structural diagram of a regional traffic signal cooperative control system based on multi-agent reinforcement learning, provided in an embodiment of this application.

[0022] This embodiment provides a regional traffic signal cooperative control method based on multi-agent reinforcement learning. Figure 1 A flowchart of the regional traffic signal cooperative control method based on multi-agent reinforcement learning provided in the embodiments of this application is shown below. Figure 1 As shown, the method includes the following steps: Step S101: Obtain the light pole configuration information and the preset multi-agent collaborative strategy.

[0023] In this step, the configuration information of each smart light pole in the preset area and the pre-configured multi-agent collaborative strategy are first obtained. For example, the light pole configuration information records the light pole identifier and light pole location of each smart light pole in the preset area. Each smart light pole integrates multiple devices, and the multiple devices are abstracted into corresponding sub-agents.

[0024] Specifically, smart light poles, as a core component of urban road infrastructure, integrate various devices such as lighting equipment, traffic signal equipment, environmental sensors, AI cameras, broadcasting equipment, LED displays, and one-button alarm systems. Pole identification is used to uniquely mark each smart light pole within a preset area, and can use a pole number, MAC address, or IP address as a unique identifier. Pole location records the geographical coordinates of each smart light pole within the preset area, which can be represented using latitude and longitude coordinates or relative coordinates to the center of the intersection.

[0025] For example, the multiple devices integrated on each smart light pole are abstracted into corresponding sub-intelligent agents. Specifically, the lighting equipment is abstracted as a lighting sub-intelligent agent, which is responsible for turning the smart light pole's lighting function on and off, adjusting brightness, and making lighting strategy decisions in conjunction with traffic signals; the traffic signal equipment is abstracted as a signal control sub-intelligent agent, which is responsible for the traffic light phase switching, signal timing optimization, and coordinated control with the traffic lights on adjacent light poles; the environmental sensors are abstracted as an environmental perception sub-intelligent agent, which is responsible for collecting and processing environmental data such as temperature, humidity, light intensity, PM2.5 concentration, and noise; and the AI ​​camera is abstracted as an AI event monitoring sub-intelligent agent, which is responsible for... The video streams captured by the cameras undergo real-time image recognition and target detection to identify traffic incidents such as traffic accidents, illegal parking, pedestrians running red lights, and congestion. The broadcasting equipment is abstracted as an information dissemination sub-agent, responsible for disseminating voice prompts, traffic guidance information, and emergency notices through the broadcasting equipment. The LED display screen is also abstracted as an information dissemination sub-agent, responsible for disseminating text prompts and traffic guidance information through the LED display screen. Finally, the one-button alarm is abstracted as a one-button alarm sub-agent, responsible for responding to the alarm button's trigger signal and reporting the alarm information, along with the associated light pole location and on-site audio and video data, to the intersection coordinator or the cloud.

[0026] A multi-agent collaborative strategy is a pre-configured set of strategies used to guide collaborative decision-making among multiple smart light poles in a preset area and among multiple sub-agents within the same smart light pole. For example, the multi-agent collaborative strategy includes a fault-priority selection mechanism, a location-priority selection mechanism, and a hierarchical decision-making mechanism within the light pole.

[0027] In one optional implementation, the specific implementation process of step S101 includes: after the system starts, the information acquisition module sends a configuration request to a preset configuration server. The configuration request carries a system identifier and an authentication token for the configuration server to authenticate the user. After the configuration server verifies the user's identity, it queries the corresponding light pole configuration information and multi-agent collaboration strategy based on the system identifier and returns the query result to the information acquisition module in JSON format. The information acquisition module receives the returned JSON data, performs data parsing and format verification. If the data parsing is successful and the format is correct, the light pole configuration information and multi-agent collaboration strategy are stored in the local cache.

[0028] Step S102: Determine the light pole level to be activated in the preset area based on the current time period and / or scene type.

[0029] In this step, the required level of smart light poles to be activated in the preset area is dynamically determined based on the current time period and / or scene type. The level of smart light poles is used to represent the quantity or specific number of smart light poles that need to be activated in the preset area.

[0030] For example, time periods can be divided into morning peak hours, daytime off-peak hours, evening peak hours, and nighttime hours. Different time periods correspond to different traffic flow characteristics and lighting needs, therefore the required number of smart light poles to be activated also varies. For instance, during morning and evening peak hours, traffic flow is high, requiring the activation of more smart light poles to achieve refined traffic signal coordination control; during nighttime hours, traffic flow is low, allowing for a reduction in the number of activated smart light poles to decrease energy consumption.

[0031] For example, scenario types may include normal traffic scenarios, traffic accident scenarios, severe weather scenarios, large-scale event scenarios, and emergency evacuation scenarios. Different scenario types correspond to different traffic management needs, and therefore the required number of smart light poles to be activated also differs. For instance, in a traffic accident scenario, a larger number of smart light poles need to be activated around the accident site to collect on-site information, disseminate guidance information, and coordinate the adjustment of signal timing; in a normal traffic scenario, a corresponding number of smart light poles can be activated according to conventional strategies.

[0032] In one optional implementation, the level determination module internally maintains a mapping table between time periods, scene types, and light pole levels. This mapping table records the number or proportion of activated light poles under different combinations of time periods and scene types. The level determination module obtains the current time through the system clock, obtains the current scene type through the event monitoring interface, and then determines the corresponding light pole level by querying the mapping table.

[0033] Step S103: Based on the multi-agent collaborative strategy, select the target smart light pole corresponding to the light pole level in the light pole configuration information, and activate all or some of the sub-agents on the target smart light pole.

[0034] In this step, based on the multi-agent collaborative strategy obtained in step S101, a target smart light pole corresponding to the light pole level determined in step S102 is selected from the smart light poles recorded in the light pole configuration information, and all or some of the sub-agents on the selected target smart light pole are activated.

[0035] Specifically, the target smart light poles corresponding to the light pole level can refer to the number and specific locations of smart light poles that need to be activated in a preset area. For example, when there are 20 light poles in the level, 20 smart light poles need to be selected as target smart light poles from all smart light poles in the preset area. The specific method of selecting target smart light poles depends on the selection mechanism configured in the multi-agent collaborative strategy, including a fault-priority selection mechanism and a location-priority selection mechanism.

[0036] After selecting a target smart light pole, activate all or some of the sub-agents on the target smart light pole. For example, the types of sub-agents to be activated can be determined based on the current time period and / or scenario type. For instance, during nighttime hours, the lighting sub-agent needs to be activated to control streetlight illumination; during peak traffic hours, the signal control sub-agent and AI event monitoring sub-agent need to be activated to optimize signal timing and monitor traffic events; in emergency scenarios, the one-click alarm sub-agent and information dissemination sub-agent need to be activated to respond to alarms and disseminate emergency information.

[0037] In one optional implementation, the light pole selection module sends an activation command to the edge gateway of each target smart light pole. The activation command carries a list of sub-agent types to be activated. After receiving the activation command, the light pole edge gateway starts the corresponding sub-agent process or thread according to the sub-agent type specified in the command, putting it into working state.

[0038] Based on Embodiment 1, this embodiment further defines the specific implementation method of determining the required light pole level in the preset area according to the current time period and / or scene type in step S102.

[0039] When the current time is the set intersection coordination trigger time, the number of light poles to be activated in the next time period will be used as the number of light poles corresponding to the required light pole level.

[0040] Specifically, the system pre-configures an intersection coordination trigger timetable, which records multiple trigger times and the number of light poles activated for each trigger time. For example, the intersection coordination trigger timetable can be stored on the local storage medium of the intersection coordinator or in a cloud-based configuration center.

[0041] When the system clock reaches the coordination trigger time at a certain intersection, the system automatically reads the number of activated light poles corresponding to that trigger time and uses this as the number of light poles corresponding to the current required activation level. Optionally, this number of activated light poles can be a specific numerical value or a proportional value (for example, activating 60% of all smart light poles in a preset area). When the number of activated light poles is a proportional value, the system calculates the specific number of activated light poles based on the total number of smart light poles in the preset area.

[0042] For example, assuming there are a total of 50 smart light poles in the preset area, the intersection coordination triggering timetable is configured as follows: 7:30-9:00 AM peak hour corresponds to an activation rate of 80%, i.e., 40 poles activated; 9:00-5:00 PM off-peak hour corresponds to an activation rate of 50%, i.e., 25 poles activated; 5:00-7:30 PM evening peak hour corresponds to an activation rate of 90%, i.e., 45 poles activated; 7:30 PM-7:30 AM the next day nighttime hour corresponds to an activation rate of 20%, i.e., 10 poles activated. When the system clock reaches 7:30, the level determination module uses the 40 activated light poles corresponding to the morning peak hour as the number of light poles required to be activated at the desired level.

[0043] In one optional implementation, the intersection coordination trigger timetable also supports dynamic adjustment, allowing the system to adaptively correct the trigger time and the corresponding number of activated light poles based on real-time traffic flow data for the day. For example, if the morning rush hour arrives earlier than usual, the system can trigger intersection coordination earlier and increase the number of activated light poles.

[0044] Based on Embodiment 1, this embodiment further includes a step of setting the multi-agent collaborative strategy before obtaining the light pole configuration information and the preset multi-agent collaborative strategy in step S101.

[0045] Specifically, the multi-agent collaborative strategy includes a fault-priority selection mechanism and a location-priority selection mechanism. When setting up the multi-agent collaborative strategy, the system administrator can choose to enable the fault-priority selection mechanism or the location-priority selection mechanism based on factors such as the traffic management needs of the preset area, the deployment density of smart light poles, and the hardware health status of each light pole.

[0046] For example, the fault-priority selection mechanism is suitable for scenarios where the hardware failure rate of smart light poles is high and system reliability needs to be prioritized. Under this mechanism, the system prioritizes smart light poles with lower fault weights (i.e., light poles with better health) as target smart light poles to reduce the risk of collaborative control failure due to light pole failure.

[0047] For example, the location-priority selection mechanism is applicable to scenarios where smart light poles are unevenly distributed and priority needs to be given to ensuring coverage of key areas. Under this mechanism, the system prioritizes smart light poles with higher location weights (i.e., light poles located at key traffic nodes or areas with high traffic volume) as target smart light poles to ensure the effectiveness of coordinated traffic signal control in core areas.

[0048] In one optional implementation, the multi-agent collaborative strategy is configured via a configuration interface. The system administrator can select to enable either a fault-priority selection mechanism or a location-priority selection mechanism in the configuration interface, and set the initial values ​​for the fault weight and location weight of each smart light pole, respectively. Optionally, the configuration of the multi-agent collaborative strategy may also include setting the response time thresholds for each level in the hierarchical decision-making mechanism within the light pole.

[0049] Based on Embodiment 3, this embodiment provides a detailed explanation of the specific implementation method for selecting the target smart light pole corresponding to the light pole level in the light pole configuration information in step S103 when the multi-agent collaborative strategy is a fault-priority selection mechanism.

[0050] When the multi-agent cooperative strategy employs a fault-priority selection mechanism, it records the fault weight for each smart light pole. The fault weight characterizes the hardware health status or fault probability of the smart light pole; a higher fault weight indicates a higher fault probability or worse health status, while a lower fault weight indicates a better health status.

[0051] For example, the fault weight can be calculated based on factors such as the smart light pole's historical fault records, equipment runtime, and abnormal parameters in the self-inspection report. For instance, if a smart light pole has experienced three faults in the past 30 days, its fault weight can be set to 0.7; another smart light pole has not experienced any faults in the past 30 days, and its fault weight can be set to 0.1. The fault weight can be set as a continuous value between 0 and 1, or as a discrete level value (such as level 1 to level 5).

[0052] Step S103 involves selecting the target smart light pole corresponding to the light pole level in the light pole configuration information based on the multi-agent collaborative strategy. This includes selecting the target smart light pole corresponding to the light pole level in the light pole configuration information based on the light pole location and fault weight.

[0053] Specifically, the number of smart light poles to be selected is first determined based on the pole hierarchy. Then, considering the location and fault weight of each smart light pole, the corresponding number of target smart light poles are selected from all smart light poles in the preset area. The selection principle is to prioritize smart light poles with lower fault weights to ensure the reliability of system operation.

[0054] In one optional implementation, the target smart light pole corresponding to the light pole level is selected from the light pole configuration information based on the light pole location and fault weight, specifically including the following sub-steps: The same number of light pole position groups are determined according to the light pole level, and each light pole position group includes multiple light pole positions.

[0055] For example, assuming there are 10 light poles corresponding to the light pole level, the system divides the preset area into 10 light pole location groups based on the light pole positions. Each light pole location group covers a sub-area, and each sub-area includes multiple smart light poles. The division of light pole location groups can be based on a uniform grid of geographical locations or on regional division based on traffic intersections. For example, if the preset area contains 5 intersections, and each intersection is surrounded by several smart light poles, the system can divide each intersection and its surrounding light poles into a light pole location group.

[0056] In each lamp post location group, the smart lamp post with the highest fault weight is selected as the target smart lamp post.

[0057] It's important to note that the concept of "highest fault weight" here needs to be understood in conjunction with the specific definition of fault weight: if a lower fault weight indicates a better health status, then the smart light pole with the lowest fault weight is selected; conversely, if a higher fault weight indicates a better health status, then the smart light pole with the highest fault weight is selected. For consistency, this article uses the example of a lower fault weight indicating a better health status. Therefore, within each light pole location group, the smart light pole with the lowest fault weight (i.e., the best health status) is selected as the target smart light pole for that location group.

[0058] For example, suppose a certain light pole location group contains 5 smart light poles with fault weights of 0.1, 0.3, 0.5, 0.7 and 0.9 respectively. Then the system selects the smart light pole with a fault weight of 0.1 as the target smart light pole for that location group.

[0059] In this way, the system selects the smart light pole with the best health status in each light pole location group as the target smart light pole, thereby ensuring full coverage of the preset area while minimizing the risk of collaborative control failure due to light pole malfunction.

[0060] Based on Embodiment 3, this embodiment provides a detailed description of the specific implementation method for selecting the target smart light pole corresponding to the light pole level in the light pole configuration information in step S103 when the multi-agent collaborative strategy is a location-first selection mechanism.

[0061] When the multi-agent cooperative strategy employs a location-priority selection mechanism, it records the location weight of each smart light pole. The location weight characterizes the importance of the smart light pole's location; a higher location weight indicates greater traffic importance at that location.

[0062] For example, the location weight can be calculated based on factors such as historical traffic flow data of the smart light pole's location, intersection level, and whether it is an accident-prone section. For instance, a smart light pole located at the intersection of a main road and a secondary road could have a location weight of 0.9; a smart light pole located at the intersection of two side roads could have a location weight of 0.3. The location weight can be set to a continuous value between 0 and 1.

[0063] Step S103, which involves selecting the target smart light pole corresponding to the light pole level in the light pole configuration information based on the multi-agent collaborative strategy, includes selecting the target smart light pole corresponding to the light pole level in the light pole configuration information according to the light pole position and position weight.

[0064] Specifically, the number of smart light poles to be selected is first determined based on the pole hierarchy. Then, considering the position and position weight of each smart light pole, a corresponding number of target smart light poles are selected from all smart light poles in the preset area. The selection principle is to prioritize smart light poles with higher position weights to ensure effective signal coordination control in key traffic areas.

[0065] In one optional implementation, the target smart light pole corresponding to the light pole level is selected from the light pole configuration information according to the light pole position and position weight. Specifically, this includes: selecting smart light poles at non-adjacent light pole positions in descending order of position weight until the number of light poles corresponding to the light pole level is reached.

[0066] For example, suppose there are 50 smart light poles in a preset area, and the position weight of each smart light pole has been pre-calculated and recorded in the multi-agent collaborative strategy. The system first sorts all 50 smart light poles according to their position weight from high to low. Then, it selects the smart light poles sequentially starting from the one with the highest position weight, but it needs to exclude light poles adjacent to the selected smart light poles during the selection process. Adjacent light poles can refer to adjacent light poles at the same intersection in the same direction, or light poles with a geographical distance of less than a preset threshold (such as 50 meters). By excluding adjacent light poles, the system can ensure that the selected target smart light poles have sufficient spatial dispersion, thereby covering a wider area with a smaller number of light poles.

[0067] For example, if there are 10 light poles corresponding to a light pole level, the system starts by selecting the smart light pole with the highest position weight. After selecting the first light pole, it excludes all light poles adjacent to it and continues to select the light pole with the highest position weight from the remaining light poles. This process is repeated until 10 target smart light poles are selected.

[0068] This embodiment provides a detailed description of the multiple devices integrated on the smart light pole and the specific types of their corresponding sub-intelligent agents.

[0069] The smart light pole integrates multiple devices, including lighting equipment, traffic signal equipment, environmental sensors, AI cameras, broadcasting equipment, LED displays, and a one-button alarm. Correspondingly, the sub-intelligent agents include a lighting sub-intelligent agent, a signal control sub-intelligent agent, an environmental perception sub-intelligent agent, an AI event monitoring sub-intelligent agent, an information dissemination sub-intelligent agent, and a one-button alarm sub-intelligent agent.

[0070] Specifically, the lighting sub-agent corresponds to the lighting equipment and is responsible for controlling the lighting functions of the smart light poles, including turning the streetlights on and off, adjusting brightness, and adjusting color temperature. The lighting sub-agent can receive light intensity data from the environmental perception sub-agent and automatically adjust the streetlight brightness according to the ambient light; it can also receive signal timing information from the signal control sub-agent and adjust the intersection lighting intensity in conjunction with traffic light switching.

[0071] Signal control sub-agent: Corresponding to traffic signal equipment, it is responsible for controlling the red and green light phase switching and signal timing optimization. The signal control sub-agent can receive traffic event detection results from the AI ​​event monitoring sub-agent and dynamically adjust the signal timing scheme according to the event type; it can also coordinate and negotiate with the signal control sub-agents of adjacent light poles to achieve trunk line green wave control or regional traffic signal collaborative optimization.

[0072] Environmental perception sub-agent: Corresponding to environmental sensors, it is responsible for collecting and processing environmental data such as temperature, humidity, light intensity, PM2.5 concentration, noise, wind speed, and wind direction. The environmental perception sub-agent reports the collected environmental data to the intersection coordinator or the cloud through the light pole edge gateway, providing environmental status information for the decision-making of other sub-agents.

[0073] The AI ​​event monitoring sub-agent, corresponding to the AI ​​camera, is responsible for real-time image recognition and target detection of the video stream captured by the camera. It can identify traffic events such as traffic accidents, illegal parking, pedestrians running red lights, non-motorized vehicles traveling in the wrong direction, congestion, and road obstacles, and report information such as event type, location, and confidence level to other sub-agents such as the signal control sub-agent and the information dissemination sub-agent.

[0074] The information dissemination sub-agent, corresponding to the broadcasting equipment and LED display screen, is responsible for disseminating voice prompts, traffic guidance information, and emergency notices through the broadcasting equipment, and for disseminating text prompts and traffic guidance information through the LED display screen. The information dissemination sub-agent can receive event information from the AI ​​event monitoring sub-agent and automatically generate and disseminate corresponding prompts.

[0075] One-click alarm sub-agent: Corresponding to the one-click alarm button, it is responsible for responding to the trigger signal of the alarm button. When the alarm button is triggered, the one-click alarm sub-agent immediately collects information such as the current position of the light pole, timestamp, and on-site audio and video data, and reports it to the intersection coordinator or cloud alarm platform through the light pole edge gateway.

[0076] This embodiment provides a detailed description of the hierarchical decision-making mechanism within the light pole, which is included in the multi-agent collaborative strategy.

[0077] The multi-agent collaborative strategy includes a hierarchical decision-making mechanism within the light pole, which comprises three levels: a real-time response layer, a collaborative optimization layer, and a global planning layer.

[0078] For latency-sensitive tasks in the real-time response layer, each sub-agent makes independent decisions, and the response time is less than or equal to a first preset time.

[0079] Specifically, the real-time response layer is the lowest level of the hierarchical decision-making mechanism within the light pole, responsible for handling tasks highly sensitive to latency. These tasks require extremely fast response times and cannot wait for negotiation with other sub-agents or communication with the intersection coordinator. For example, the first preset time can be set to 100 milliseconds. In the real-time response layer, each sub-agent makes decisions independently based on local perception data, without needing to communicate or negotiate with other sub-agents.

[0080] For example, when the AI ​​event monitoring sub-agent detects a pedestrian running a red light, it needs to immediately trigger the signal control sub-agent to extend the red light phase or end the green light phase for pedestrians crossing the street in advance. This task is a delay-sensitive task, and the AI ​​event monitoring sub-agent and the signal control sub-agent make independent decisions at the real-time response layer, with the response time controlled within 100 milliseconds.

[0081] For example, when the one-click alarm sub-agent detects that the alarm button has been triggered, it needs to immediately collect on-site audio and video data and report the alarm information. This task is also a delay-sensitive task, which is independently decided and executed by the one-click alarm sub-agent at the real-time response layer.

[0082] For tasks requiring multi-device collaboration, the collaborative optimization layer initiates joint negotiation among sub-agents through the light pole edge gateway, and uses a consensus-based distributed decision-making algorithm to reach a consensus within a time frame of less than or equal to a second preset time.

[0083] Specifically, the collaborative optimization layer is the intermediate layer of the hierarchical decision-making mechanism within the light pole, responsible for handling tasks that require collaboration among multiple devices within the same light pole. These tasks do not require communication with external light poles or intersection coordinators, but rather necessitate consensus among multiple sub-agents within the light pole. For example, the second preset time can be set to 500 milliseconds.

[0084] In the collaborative optimization layer, the lamppost edge gateway acts as the communication hub within the lamppost, initiating joint negotiations among the relevant sub-agents. Each sub-agent sends its decision preferences, state information, and confidence level to the lamppost edge gateway. The lamppost edge gateway then employs a consensus-based distributed decision-making algorithm to seek consensus among the multiple sub-agents, ultimately forming a unified collaborative decision-making scheme.

[0085] For example, in a nighttime traffic accident scenario, the lighting sub-agent needs to adjust the brightness of streetlights around the accident site to provide sufficient illumination, the signal control sub-agent needs to adjust the signal timing around the accident site to manage traffic, and the information dissemination sub-agent needs to broadcast accident warnings and detour information via LED displays. These three sub-agents need to collaborate under the organization of the light pole edge gateway, reaching a consensus-based collaborative decision-making scheme within 500 milliseconds based on a consensus algorithm, thus achieving coordinated responses from lighting, signal control, and information dissemination.

[0086] Consensus-based distributed decision-making algorithms can employ Raft consensus algorithm, PBFT (Practical Byzantine Fault Tolerance) algorithm, or a custom weighted voting consensus algorithm. For example, this algorithm is specifically a weighted voting consensus algorithm: each sub-agent is assigned a different voting weight based on its task priority. Each sub-agent first independently calculates its own decision proposal and then sends the proposal to the lamppost edge gateway. After collecting all proposals, the lamppost edge gateway performs a weighted aggregation according to the voting weights of each sub-agent, selecting the proposal with the highest total weight as the final collaborative decision result. When multiple proposals have the highest total weight, the proposal with the highest average confidence among all sub-agents is selected as the final collaborative decision result.

[0087] For tasks that require coordination with adjacent light poles or intersection coordinators, the global planning layer reports them from the light pole edge gateway to the intersection coordinator.

[0088] Specifically, the global planning layer is the top layer of the hierarchical decision-making mechanism within the light pole, responsible for handling tasks requiring coordination across light poles or intersections. These tasks exceed the decision-making scope of a single light pole and require information exchange and joint decision-making with adjacent light poles or intersection coordinators.

[0089] In the global planning layer, the light pole edge gateway reports task information requiring cross-light pole collaboration, local perception data, and decision suggestions from sub-agents to the intersection coordinator. The intersection coordinator aggregates the reported information from multiple light poles, makes global traffic signal collaborative optimization decisions, and distributes the decision results to each light pole for execution.

[0090] For example, in a trunk road green wave control scenario, signal control sub-agents at multiple intersections along the trunk road need to coordinate to adjust the signal timing phase difference to achieve green wave passage for vehicles. This task requires coordination between multiple intersection coordinators, as a single light pole cannot complete it independently. Therefore, each light pole's edge gateway reports local traffic flow data to the intersection coordinator, which then performs global green wave optimization calculations and distributes the optimized signal timing scheme to the signal control sub-agents at each light pole for execution.

[0091] This embodiment provides a regional traffic signal cooperative control system based on multi-agent reinforcement learning. Figure 2 The structural diagram of the regional traffic signal cooperative control system based on multi-agent reinforcement learning provided in the embodiments of this application is as follows: Figure 2 As shown, the system includes an information acquisition module 201, a hierarchy determination module 202, and a light pole selection module 203.

[0092] The information acquisition module 201 is used to acquire the light pole configuration information and the preset multi-agent collaborative strategy. The light pole configuration information records the light pole identifier and light pole position of each smart light pole in the preset area. Each smart light pole integrates multiple devices, and the multiple devices are abstracted into corresponding sub-agents.

[0093] Specifically, the information acquisition module 201 can read the light pole configuration information and multi-agent collaborative strategies from a configuration server, cloud platform, or local storage medium via wired or wireless communication interfaces. The light pole configuration information can be stored in JSON format, XML format, or database table format.

[0094] After obtaining the light pole configuration information, the information acquisition module 201 stores it in a local cache for use by the hierarchy determination module 202 and the light pole selection module 203.

[0095] In one optional implementation, the information acquisition module 201 periodically synchronizes light pole configuration information and multi-agent collaborative strategies from the cloud configuration platform via a wireless communication interface, and stores the synchronized data in the local ROM. When network communication is interrupted, the information acquisition module 201 can read the most recently synchronized configuration information from the local ROM, ensuring that the system can still operate normally in an offline state.

[0096] The level determination module 202 is used to determine the light pole level to be activated in the preset area based on the current time period and / or scene type.

[0097] Specifically, the hierarchy determination module 202 internally maintains a mapping table between time periods, scene types, and light pole levels. This mapping table records the number or proportion of activated light poles under different time periods and different scene type combinations. The hierarchy determination module 202 obtains the current time through the system clock, obtains the current scene type through the event monitoring interface (such as obtaining traffic accident event identifiers from the AI ​​event monitoring sub-agent), and then determines the corresponding light pole level by querying the mapping table.

[0098] For example, some records in the mapping table are as follows: Time period = Morning rush hour (7:30-9:00), Scene type = Normal traffic -- Number of activated light poles = 40 Time period = Evening rush hour (17:00-19:30), Scene type = Traffic accident - Number of light poles activated = 50 (all activated) Time period = Nighttime (7:30 PM - 7:30 AM the next day), Scene type = Normal traffic -- Number of activated light poles = 10 After determining the level of the light pole, the level determination module 202 transmits the light pole level information (including the number of activated light poles) to the light pole selection module 203.

[0099] In one optional implementation, the hierarchy determination module 202 obtains the current time through the system clock module and the current scene type through the interface of the AI ​​event monitoring sub-agent. The system clock module can use GPS time synchronization or the NTP network time protocol to ensure time accuracy at the millisecond level.

[0100] The light pole selection module 203 is used to select the target smart light pole corresponding to the light pole level in the light pole configuration information based on the multi-agent collaborative strategy, and activate all or some of the sub-agents on the target smart light pole.

[0101] Specifically, the light pole selection module 203 receives light pole hierarchy information from the hierarchy determination module 202 and obtains light pole configuration information and multi-agent collaboration strategy from the information acquisition module 201. Based on the selection mechanism configured in the multi-agent collaboration strategy (fault-priority selection mechanism or location-priority selection mechanism), and following the selection method described in Embodiment 4 or Embodiment 5, the light pole selection module 203 selects the target smart light pole corresponding to the light pole hierarchy from all smart light poles in the preset area.

[0102] After selecting a target smart light pole, the light pole selection module 203 sends an activation command to the edge gateway of each target smart light pole. The activation command carries a list of sub-agent types to be activated. After receiving the activation command, the light pole edge gateway starts the corresponding sub-agent process or thread according to the sub-agent type specified in the command, putting it into working state.

[0103] For example, during the morning rush hour, the activation instructions for the 40 target smart light poles selected by the light pole selection module 203 carry the following sub-agent type list: ["Signal Control Sub-Agent", "AI Event Monitoring Sub-Agent", "Environmental Perception Sub-Agent"], indicating that only these three types of sub-agents are activated, while the lighting sub-agent, information dissemination sub-agent, and one-click alarm sub-agent remain in sleep or low-power state. During the nighttime hours, the activation instructions for the 10 target smart light poles selected by the light pole selection module 203 carry the following sub-agent type list: ["Lighting Sub-Agent", "Environmental Perception Sub-Agent"], indicating that only the lighting sub-agent and the environment perception sub-agent are activated.

[0104] In one optional implementation, the light pole selection module 203 sends an activation command to the edge gateway of each target smart light pole via a wired Ethernet interface or a wireless communication interface. The activation command is transmitted using an encrypted communication protocol (such as TLS 1.3) to ensure the confidentiality and integrity of the command. After receiving the activation command, the light pole edge gateway decrypts and verifies it; if the verification is successful, the corresponding sub-smart agent is activated.

[0105] This embodiment provides an exemplary description of the complete process of the above-mentioned regional traffic signal cooperative control method based on multi-agent reinforcement learning in a practical application scenario.

[0106] Application Scenario 1: Normal traffic flow during morning rush hour The preset area is a central business district of a city, with 50 smart light poles distributed across 5 intersections and adjacent road sections. The current time is 8:15 AM, which is during the morning rush hour, and the scenario type is normal traffic.

[0107] In step S101, the information acquisition module 201 retrieves the light pole configuration information and the preset multi-agent collaborative strategy from the configuration server. The light pole configuration information records the light pole identifiers and locations of 50 smart light poles. The multi-agent collaborative strategy is configured as a location-first selection mechanism, and the location weight of each smart light pole has been pre-calculated based on historical traffic flow data.

[0108] In step S102, the hierarchy determination module 202, based on the current time 8:15 and the scene type "normal passage", queries the time period-scene type mapping table to determine that the number of light poles corresponding to the light pole hierarchy to be activated under the morning peak normal passage scene is 40.

[0109] In system execution step S103, the light pole selection module 203 selects 40 target smart light poles with higher position weights and non-adjacent positions from 50 smart light poles based on a position priority selection mechanism, according to position weight from high to low. Then, it sends activation commands to the edge gateways of the selected 40 smart light poles to activate the signal control sub-agent, the AI ​​event monitoring sub-agent, and the environmental perception sub-agent.

[0110] The 40 activated smart light poles begin operation: the signal control sub-agent optimizes signal timing at each intersection based on real-time traffic flow data; the AI ​​event monitoring sub-agent continuously monitors traffic events; and the environmental perception sub-agent collects and reports environmental data. The sub-agents within each light pole collaborate according to a hierarchical decision-making mechanism: latency-sensitive tasks are decided independently by each sub-agent at the real-time response layer (response time ≤ 100ms); tasks requiring multi-device collaboration are jointly negotiated by the light pole edge gateway at the collaborative optimization layer (response time ≤ 500ms); and tasks requiring cross-light pole collaboration are reported to the intersection coordinator at the global planning layer.

[0111] Application Scenario 2: Traffic Accidents During Evening Rush Hour The preset area is the same as above, and the current time is 18:00, which is during the evening rush hour. The AI ​​event monitoring sub-agent detected a traffic accident at a certain intersection, and the scene type changed to traffic accident.

[0112] The level determination module 202 queries the mapping table based on the current time 18:00 and the scenario type "traffic accident" to determine that the number of light poles corresponding to the level of light poles to be activated in the evening peak traffic accident scenario is 50 (all activated).

[0113] The light pole selection module 203 selects all 50 smart light poles as target smart light poles based on the location priority selection mechanism, and sends activation commands to all light poles to activate all types of sub-intelligent agents (lighting sub-intelligent agent, signal control sub-intelligent agent, environmental perception sub-intelligent agent, AI event monitoring sub-intelligent agent, information release sub-intelligent agent, one-click alarm sub-intelligent agent).

[0114] The smart streetlights around the accident site collaborate at the optimization layer: the lighting sub-agent increases the brightness of streetlights around the accident site; the signal control sub-agent adjusts signal timing around the accident site to alleviate traffic congestion; and the information dissemination sub-agent broadcasts accident warnings and detour information via radio and LED displays. Simultaneously, the global planning layer reports the accident information to the intersection coordinator, which coordinates with adjacent intersections to optimize regional signal timing, achieving coordinated control of regional traffic signals.

[0115] This embodiment further explains and clarifies the key technical terms and concepts involved in the above method.

[0116] Multi-agent reinforcement learning (MAL) is an extension of reinforcement learning in multi-agent systems, involving multiple agents learning optimal strategies through interaction with the environment in a shared environment. In this application, each sub-agent acts as an independent reinforcement learning agent. Its state space includes local perception data (such as traffic flow, ambient light, and event detection results), its action space includes device control commands (such as signal phase switching, light brightness adjustment, and information dissemination), and its reward function is designed based on a combination of traffic efficiency (such as average traffic delay and queue length) and energy efficiency (such as lighting power consumption). The sub-agents exchange information and make joint decisions through a light pole edge gateway, achieving multi-agent collaborative reinforcement learning.

[0117] A light pole edge gateway is an edge computing device deployed on the side of a smart light pole, possessing data acquisition, data processing, local decision-making, and communication forwarding functions. The light pole edge gateway connects all devices on the same smart light pole (lighting equipment, traffic signal equipment, environmental sensors, AI cameras, broadcasting equipment, LED displays, and one-button alarms), acting as the communication hub for each sub-agent, responsible for message routing, data fusion, and the organization and coordination of collaborative decision-making among the sub-agents.

[0118] An intersection coordinator is an edge computing device deployed at an intersection, connecting multiple smart light poles in the surrounding area as an edge gateway. The intersection coordinator is responsible for aggregating information reported by multiple light poles, making global traffic signal coordination and optimization decisions, and distributing the results to each light pole for execution. Intersection coordinators can communicate with each other via wired or wireless networks to achieve coordinated regional traffic signal control across intersections.

[0119] A consensus-based distributed decision-making algorithm is an algorithm that enables multiple nodes in a distributed system to reach a consensus on a certain value or decision. In the collaborative optimization layer of this application, the light pole edge gateway uses a consensus-based distributed decision-making algorithm to organize the sub-agents to conduct joint negotiation. Each sub-agent first independently calculates its own decision proposal and then sends the proposal to the light pole edge gateway. After collecting all proposals, the light pole edge gateway executes a consensus algorithm (such as weighted voting, Raft consensus, etc.) to reach a consensus among the proposals. Finally, the consensus result is distributed as the final collaborative decision to each sub-agent for execution. This algorithm ensures that even if there is a delay in the internal network communication of the light pole or some sub-agents are temporarily unavailable, a valid consensus decision can still be reached within a second preset time (such as 500ms).

[0120] This embodiment provides an exemplary description of the specific hardware implementation of the above system.

[0121] For example, a regional traffic signal cooperative control system based on multi-agent reinforcement learning can be deployed in a roadside edge computing node, which includes a processor, memory, network interface, and power module. The processor is an embedded processor with ARM or x86 architecture, possessing at least four cores and a clock speed of at least 1.5 GHz. The memory includes RAM and ROM; the RAM capacity is at least 4 GB, used to store runtime data and program instructions, and the ROM capacity is at least 32 GB, used to store light pole configuration information, multi-agent cooperative strategies, and system programs. The network interface includes a wired Ethernet interface (such as an RJ45 interface) and a wireless communication interface (such as a 4G / 5G cellular communication module, Wi-Fi module, or ZigBee module), used for communication with smart light poles, intersection coordinators, and the cloud platform.

[0122] The information acquisition module 201, the level determination module 202, and the light pole selection module 203 can be functional modules implemented by a processor executing computer program instructions stored in the memory, or they can be dedicated hardware modules implemented using hardware such as FPGA or ASIC.

[0123] This embodiment provides an exemplary description of the reinforcement learning training process of the sub-agents involved in the above method.

[0124] Each sub-agent (lighting sub-agent, signal control sub-agent, environmental perception sub-agent, AI event monitoring sub-agent, information dissemination sub-agent, one-click alarm sub-agent) needs to undergo multi-agent reinforcement learning training in a simulation environment before deployment.

[0125] Specifically, a traffic simulation environment is constructed, which includes a pre-defined regional road network, smart street light deployment, vehicle flow model, and pedestrian flow model. Each sub-agent is deployed as a reinforcement learning agent on the corresponding device in the simulation environment.

[0126] During training, each sub-agent collects state, action, and reward data by interacting with the environment. The state space includes: lamppost location, current time period, scene type, traffic flow, queue length, average vehicle speed, ambient light intensity, weather conditions, and event detection results. The action space includes: signal phase switching commands, light brightness levels, information release content categories, and alarm reporting triggers. The reward function adopts a multi-objective design, comprehensively considering traffic efficiency (such as reducing average travel delay and queue length) and energy efficiency (such as reducing lighting power consumption and reducing equipment idling time).

[0127] Each sub-agent is trained using a multi-agent reinforcement learning framework with centralized training and decentralized execution (CTDE). During the training phase, each sub-agent can access global state information and perform joint policy optimization through a shared reward function; during the execution phase, each sub-agent makes independent decisions based solely on local observation information, ensuring real-time response capabilities.

[0128] After training is completed, the trained sub-agent policy model is deployed to the edge gateway of each smart light pole for use in actual operation.

[0129] This embodiment provides a detailed description of a specific implementation of step S101 of the above method, which involves obtaining light pole configuration information and a preset multi-agent collaborative strategy.

[0130] For example, the specific process of obtaining light pole configuration information and preset multi-agent collaborative strategies includes: After the system starts up, the information acquisition module 201 sends a configuration request to the preset configuration server. The configuration request carries a system identifier and an authentication token, which are used by the configuration server for authentication.

[0131] Secondly, after the configuration server is verified, the corresponding light pole configuration information and multi-agent collaboration strategy are queried according to the system identifier, and the query results are returned to the information acquisition module 201 in JSON format.

[0132] Then, the information acquisition module 201 receives the returned JSON data and performs data parsing and format verification. If the data parsing is successful and the format is correct, the light pole configuration information and multi-agent collaboration strategy are stored in the local cache; if the data parsing fails or the format is incorrect, the information acquisition module 201 attempts to read the most recently successfully acquired configuration information from the local backup storage.

[0133] Finally, if no valid configuration information is found in the local backup storage, the information acquisition module 201 generates a configuration missing alarm and sends the alarm information to the operation and maintenance platform through the network interface, prompting the operation and maintenance personnel to perform manual configuration.

[0134] Optionally, the information acquisition module 201 can also periodically (e.g., every 30 minutes) send configuration update query requests to the configuration server to check if there are any updates to the light pole configuration information or multi-agent collaboration strategy. If there are updates, the content is incrementally synchronized to the local machine to ensure that the local configuration information remains consistent with the cloud in real time.

[0135] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. For those skilled in the art, other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all implementation methods here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A regional traffic signal cooperative control method based on multi-agent reinforcement learning, characterized in that, include: The system acquires light pole configuration information and a preset multi-agent collaborative strategy. The light pole configuration information records the light pole identifier and light pole position of each smart light pole in a preset area. Each smart light pole integrates multiple devices, and the multiple devices are abstracted into corresponding sub-agents. Based on the current time period and / or scene type, determine the required level of light poles to be activated in the preset area; Based on the multi-agent collaborative strategy, the target smart light pole corresponding to the light pole level is selected in the light pole configuration information, and all or some of the sub-agents on the target smart light pole are activated; The multi-agent collaborative strategy includes a hierarchical decision-making mechanism within the light pole, which includes: In the real-time response layer, latency-sensitive tasks are decided independently by each sub-agent, and the response time is less than or equal to the first preset time. In the collaborative optimization layer, for tasks requiring multi-device collaboration, the edge gateway of the light pole initiates joint negotiation among sub-agents, and a consensus-based distributed decision-making algorithm is used to reach a consensus within a time frame of less than or equal to the second preset time. At the global planning layer, tasks that require coordination with adjacent light poles or intersection coordinators are reported from the light pole edge gateway to the intersection coordinator.

2. The regional traffic signal cooperative control method based on multi-agent reinforcement learning according to claim 1, characterized in that, The step of determining the required light pole level to be activated in the preset area based on the current time period and / or scene type includes: When the current time is the set intersection coordination trigger time, the number of light poles to be activated in the next time period will be used as the number of light poles corresponding to the required light pole level.

3. The regional traffic signal cooperative control method based on multi-agent reinforcement learning according to claim 1, characterized in that, Before acquiring the light pole configuration information and the preset multi-agent collaborative strategy, the following steps are also included: The multi-agent cooperative strategy is set, which includes a fault-priority selection mechanism and a location-priority selection mechanism.

4. The regional traffic signal cooperative control method based on multi-agent reinforcement learning according to claim 3, characterized in that, When the multi-agent collaborative strategy is a fault-priority selection mechanism, the multi-agent collaborative strategy records the fault weight of each smart light pole. The step of selecting the target smart light pole corresponding to the light pole level based on the light pole configuration information according to the multi-agent collaborative strategy includes: Based on the pole location and the fault weight, the target smart pole corresponding to the pole level is selected in the pole configuration information.

5. The regional traffic signal cooperative control method based on multi-agent reinforcement learning according to claim 4, characterized in that, The step of selecting the target smart light pole corresponding to the light pole level in the light pole configuration information based on the light pole location and the fault weight includes: Based on the lamp post hierarchy, the same number of lamp post position groups are determined, and each lamp post position group includes multiple lamp post positions; In each group of light pole locations, the smart light pole with the highest fault weight is selected as the target smart light pole.

6. The regional traffic signal cooperative control method based on multi-agent reinforcement learning according to claim 3, characterized in that, When the multi-agent collaborative strategy is a location-first selection mechanism, the multi-agent collaborative strategy records the location weight of each smart light pole. The step of selecting the target smart light pole corresponding to the light pole level based on the light pole configuration information according to the multi-agent collaborative strategy includes: Based on the pole position and the position weight, the target smart pole corresponding to the pole level is selected in the pole configuration information.

7. The regional traffic signal cooperative control method based on multi-agent reinforcement learning according to claim 6, characterized in that, The step of selecting the target smart light pole corresponding to the light pole level in the light pole configuration information based on the light pole position and the position weight includes: Based on the position weight, smart light poles are selected sequentially from high to low for non-adjacent positions until the number of light poles corresponding to the light pole level is reached.

8. The regional traffic signal cooperative control method based on multi-agent reinforcement learning according to claim 1, characterized in that, The smart light pole integrates multiple devices including lighting equipment, traffic signal equipment, environmental sensors, AI cameras, broadcasting equipment, LED displays, and one-button alarms; the sub-intelligent agents include lighting sub-intelligent agents, signal control sub-intelligent agents, environmental perception sub-intelligent agents, AI event monitoring sub-intelligent agents, information dissemination sub-intelligent agents, and one-button alarm sub-intelligent agents.

9. A regional traffic signal cooperative control system based on multi-agent reinforcement learning, characterized in that, include: The information acquisition module is used to acquire light pole configuration information and preset multi-agent collaborative strategies. The light pole configuration information records the light pole identifier and light pole position of each smart light pole in the preset area. Each smart light pole integrates multiple devices, and the multiple devices are abstracted into corresponding sub-agents. The level determination module is used to determine the required level of light poles to be activated in the preset area based on the current time period and / or scene type. The light pole selection module is used to select the target smart light pole corresponding to the light pole level in the light pole configuration information based on the multi-agent collaborative strategy, and activate all or some of the sub-agents on the target smart light pole; The multi-agent collaborative strategy includes a hierarchical decision-making mechanism within the light pole, which includes: In the real-time response layer, latency-sensitive tasks are decided independently by each sub-agent, and the response time is less than or equal to the first preset time. In the collaborative optimization layer, for tasks requiring multi-device collaboration, the edge gateway of the light pole initiates joint negotiation among sub-agents, and a consensus-based distributed decision-making algorithm is used to reach a consensus within a time frame of less than or equal to the second preset time. At the global planning layer, tasks that require coordination with adjacent light poles or intersection coordinators are reported from the light pole edge gateway to the intersection coordinator.

Images