Dynamic speed limit optimization method and device for reconstruction and extension bottleneck road section of expressway
Through the near-end strategy optimization algorithm, the speed limit is dynamically adjusted in the bottleneck section of the highway renovation and expansion construction, which solves the problem of insufficient adaptability and stability of the speed limit optimization strategy in the existing technology, improves traffic flow and safety, and is suitable for complex Traffic environment.
Patent Information
- Application Number
- CN202411398821.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-09
- Publication Date
- 2024-11-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
There is a risk of traffic congestion and accidents during the construction period and subsequent operation of the bottleneck section of the expressway renovation and expansion. The existing technology speed limit optimization strategies are not targeted, insufficiently adaptable and unstable, making it difficult to effectively deal with dynamic changes in complex traffic environments.
Adopt the near-end strategy optimization (PPO) algorithm, by collecting and preprocessing traffic data, defining status representation and reward functions, optimizing the speed limit adjustment strategy, realizing dynamic speed limit adjustment of bottleneck sections of highway reconstruction and expansion, and using reinforcement learning algorithms to implement real-time Dynamic speed limit adjustments are made in traffic conditions.
It improves traffic flow and safety, ensures the stability and efficiency of the speed limiting strategy, is suitable for complex traffic environments and highway renovation and expansion scenarios, and achieves sustainable traffic management optimization.
Smart Images

Figure CN118917495A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of traffic technology, and in particular to a method and device for optimizing dynamic speed limit in a bottleneck section of a highway reconstruction and expansion project. Background Art
[0002] With the rapid development of social economy and the acceleration of urbanization, the number of vehicles and transportation demand carried by the highway system as the main traffic channel between cities is increasing. However, due to various factors such as traffic accidents, construction areas, vehicle congestion, etc., traffic mobility and safety on highways face many challenges. Therefore, the development of intelligent and dynamic traffic management systems has become one of the important ways to improve road use efficiency and ensure traffic safety.
[0003] During the operation of highways, there are often situations where reconstruction and expansion are needed, such as road widening and lane addition projects to increase traffic capacity or adapt to changes in traffic demand. However, reconstruction and expansion projects often cause the bottleneck sections of the reconstruction and expansion to become narrower during the construction process, thus forming bottleneck areas. These bottlenecks will significantly affect traffic flow and driving speed, and increase the risk of traffic accidents and congestion. Traditional static speed limit methods are often unable to effectively cope with these dynamically changing traffic conditions, resulting in serious traffic congestion and accidents in bottleneck sections of bottleneck reconstruction and expansion. Therefore, how to achieve dynamic speed limit adjustment through intelligent technology during the construction period of highway reconstruction and expansion and the subsequent operation stage has become a technical problem that urgently needs to be solved in the current traffic management field.
[0004] In the prior art, patents such as CN201610542934.7 and CN202310130498.2 proposed a technical solution for variable speed limit using reinforcement learning, but there are the following shortcomings: (1) Lack of optimization for bottleneck sections of highway reconstruction and expansion: Existing technologies mainly focus on speed limit optimization for ordinary bottleneck sections of reconstruction and expansion, but lack targeted optimization strategies for the special traffic conditions of bottleneck sections of reconstruction and expansion, resulting in poor results during construction and subsequent operation. (2) Insufficient algorithm adaptability: Existing reinforcement learning algorithms have performance bottlenecks when processing high-dimensional states and discrete action spaces, and are unable to fully adapt to complex traffic environments and real-time changing traffic conditions; (3) Insufficient stability and efficiency: The strategy optimization process of some existing technologies is prone to instability, which makes the speed limit adjustment strategy unstable and inefficient, affecting the actual application effect. Summary of the invention
[0005] The purpose of the present invention is to provide a method and device for optimizing dynamic speed limit of bottleneck section in highway reconstruction and expansion, which solves the problems existing in the prior art.
[0006] The present invention is achieved through the following technical solutions: On the one hand, the present invention provides a method for optimizing dynamic speed limit of bottleneck section in highway reconstruction and expansion, comprising: Collecting traffic data of each bottleneck section of the target expressway to be rebuilt or expanded, and preprocessing the traffic data to obtain preprocessed traffic data; Determine the state representation corresponding to each rebuilt and expanded bottleneck section according to the preprocessed traffic data, define the action according to the speed limit value, and determine the reward function corresponding to each rebuilt and expanded bottleneck section according to the state representation and the action, so as to obtain the state representation, action and reward function corresponding to each rebuilt and expanded bottleneck section; Based on the state representation, action and reward function corresponding to each bottleneck section of reconstruction and expansion, the proximal strategy optimization algorithm is used to optimize the speed limit adjustment strategy corresponding to each bottleneck section of reconstruction and expansion, so as to complete the dynamic speed limit optimization of the bottleneck section of highway reconstruction and expansion.
[0007] In a possible implementation, collecting traffic data of each bottleneck section of the target highway to be rebuilt or expanded includes: For each bottleneck section of the target highway that is undergoing reconstruction and expansion, the traffic flow, vehicle speed and lane occupancy rate corresponding to the bottleneck section are collected to obtain the traffic data corresponding to each bottleneck section that is undergoing reconstruction and expansion.
[0008] In a possible implementation manner, after preprocessing the traffic data, obtaining the preprocessed traffic data includes: The traffic data is normalized and / or denoised to obtain pre-processed traffic data.
[0009] In a possible implementation manner, determining the state representation corresponding to each reconstruction and expansion bottleneck section according to the preprocessed traffic data includes: Based on the pre-processed traffic data, determine the average vehicle speed and average lane occupancy rate corresponding to each bottleneck road section to be rebuilt or expanded at the decision time step; According to the average vehicle speed and average lane occupancy rate of each bottleneck section to be rebuilt and expanded at the decision time step, the state corresponding to each bottleneck section to be rebuilt and expanded is determined as follows: ;in, represents the average vehicle speed of the bottleneck section i at the decision time step t, represents the average lane occupancy rate of the bottleneck road section i at the decision time step t; , I represents the total number of bottleneck sections to be rebuilt or expanded in the target expressway.
[0010] In a possible implementation, defining an action with a speed limit value includes: A speed limit adjustment action space A is constructed using multiple speed limit values; Define actions for each bottleneck section to be renovated or expanded , which represents the speed limit value selected at time t for the expansion and reconstruction bottleneck section of speed limit control.
[0011] In a possible implementation manner, the reward function corresponding to each bottleneck section to be rebuilt or expanded is determined according to the state representation and the action: ;in, Indicates taking action Then the downstream traffic flow at time t+1.
[0012] In a possible implementation, based on the state representation, action and reward function corresponding to each bottleneck section of the reconstruction and expansion, the speed limit adjustment strategy corresponding to each bottleneck section of the reconstruction and expansion is optimized by using a proximal strategy optimization algorithm to complete the dynamic speed limit optimization of the bottleneck section of the highway reconstruction and expansion, including: For each bottleneck section to be rebuilt or expanded, the status is expressed as And actions As the input of the policy network, to obtain the state of the policy network output Take action Probability ; In the state indicated As the input of the value network, it evaluates the state representation at the decision time step t The value of ; According to the status The value of And the reward function , determine the advantage function ; According to the probability And the advantage function , determine the loss function ; The gradient descent method is used to adjust the policy network parameters and value network parameters Update to minimize the loss function , and take the action corresponding to the maximum probability output by the strategy network at each decision time step t as the corresponding speed limit at the next decision time step t, and obtain the optimized speed limit adjustment strategy; The optimized speed limit adjustment strategy is deployed in the dynamic speed limit system of the highway. The traffic data is updated at each decision time step t to obtain the corresponding speed limit at the next decision time step t, thereby realizing dynamic speed limit optimization of bottleneck sections in highway reconstruction and expansion.
[0013] In a possible implementation, according to the state representation The value of And the reward function , determine the advantage function for: ; in, represents the discount factor to balance the weights of current rewards and future rewards, Represents the state representation at decision time step t+1 value.
[0014] In a possible implementation, according to the probability And the advantage function , determine the loss function for: ; in, represents the expected value at the decision time step t; min represents the minimum value; Indicates the current policy parameters Next, status Next select action probability; Indicates that under the old policy parameters, the status Next select action probability; is a clipping function, which is used to limit the range of variation of the probability ratio to prevent instability caused by excessive updates; ϵ is the clipping coefficient.
[0015] On the other hand, the present invention provides a method for optimizing dynamic speed limit in bottleneck section of highway reconstruction and expansion, comprising: a data acquisition module, a data processing module and a dynamic speed limit optimization module; The data collection module is used to collect traffic data of each bottleneck section of the target highway to be rebuilt and expanded, and pre-process the traffic data to obtain the pre-processed traffic data; The data processing module is used to determine the state representation corresponding to each reconstruction and expansion bottleneck section according to the pre-processed traffic data, define an action with a speed limit value, and determine the reward function corresponding to each reconstruction and expansion bottleneck section according to the state representation and the action, so as to obtain the state representation, action and reward function corresponding to each reconstruction and expansion bottleneck section; The dynamic speed limit optimization module is used to optimize the speed limit adjustment strategy corresponding to each bottleneck section of the reconstruction and expansion based on the state representation, action and reward function corresponding to each bottleneck section of the reconstruction and expansion, and to complete the dynamic speed limit optimization of the bottleneck section of the highway reconstruction and expansion.
[0016] The present invention provides a method and device for optimizing dynamic speed limits in bottleneck sections of highway reconstruction and expansion. Through a reinforcement learning algorithm, the speed limit of the highway can be dynamically adjusted according to real-time traffic conditions and environmental conditions, thus avoiding the limitations of traditional fixed speed limit methods and improving traffic flow and safety. The proximal policy optimization algorithm can process high-dimensional states and discrete action spaces, and is suitable for various complex traffic environments and highway reconstruction and expansion scenarios, and has wide applicability. The proximal policy optimization algorithm introduces a shear objective function to limit the stride of each policy update, thereby ensuring the stability and efficiency of the policy optimization process and avoiding the common problem of unstable policy updates in other algorithms. In the actual operation process, data is continuously collected and processed, and the speed limit adjustment strategy is continuously updated and optimized to ensure that the system can adapt to long-term traffic changes and environmental changes and achieve sustainable traffic management optimization. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without creative work. In the drawings: Figure 1 A schematic diagram of a computer device provided by an embodiment of the present invention; Figure 2 A schematic flow chart of a method for optimizing dynamic speed limit on bottleneck sections of expressway reconstruction and expansion provided by an embodiment of the present invention; Figure 3 A schematic diagram of a road environment with dynamic speed limit at a bottleneck of a highway reconstruction and expansion provided by an embodiment of the present invention; Figure 4 A schematic diagram of a device for optimizing dynamic speed limit on bottleneck sections of expressway reconstruction and expansion provided by an embodiment of the present invention; Figure 5 A schematic diagram of the structure of a computing device provided by an embodiment of the present invention.
[0018] Marks and corresponding parts names in the attached drawings: 100. Computer device; 110. Processor; 111. Processor core; 112. Operating system; 120. Memory; 130. Memory controller; 140. I / O device; 150. Bus; 160. Data processing unit; 400. Dynamic speed limit optimization device for bottleneck section of highway reconstruction and expansion; 410. Data acquisition module; 420. Data processing module; 430. Dynamic speed limit optimization module; 500. Computing device; 510. Second processor; 520. Second bus; 530. Memory; 540. Communication interface; 550. Memory unit. DETAILED DESCRIPTION
[0019] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.
[0020] First, the terms involved in this application are explained: PPO (Proximal Policy Optimization) is a reinforcement learning algorithm used to train agents to perform specific tasks in a given environment. Its main features are easy implementation and adjustment, as well as excellent performance, and it is particularly suitable for optimization problems in discrete action spaces and high-dimensional state spaces.
[0021] Reconstruction and expansion projects often cause road sections to become narrower during construction, thus forming bottleneck areas. These bottlenecks will significantly affect traffic flow and driving speed, and increase the risk of traffic accidents and congestion. Traditional static speed limit methods are often difficult to effectively cope with these dynamically changing traffic conditions, resulting in serious traffic congestion and accidents in bottleneck sections. However, the dynamic speed limit of the prior art has technical problems of insufficient adaptability and insufficient stability. Therefore, the present invention provides a dynamic speed limit optimization method for highway reconstruction and expansion based on the proximal policy optimization (PPO) algorithm, aiming to improve traffic efficiency and safety through intelligent and dynamic speed limit adjustment. Specifically, the present invention collects real-time traffic flow, vehicle speed, lane occupancy rate and other data, and uses the proximal policy optimization (PPO) algorithm to optimize the speed limit adjustment strategy, so as to dynamically adjust the speed limit according to the actual traffic conditions, thereby improving the overall operation efficiency and safety level of the highway.
[0022] For ease of understanding, combined Figure 1 The system architecture to which the embodiments of the present invention are applicable is introduced.
[0023] Figure 1 A schematic diagram of a computer device provided by an embodiment of the present invention. Figure 1As shown, the computer device 100 includes a processor 110 , a memory 120 , and a memory controller 130 .
[0024] The processor 110 is the computing core and control core of the computer device 100. The processor 110 can be a very large scale integrated circuit. The operating system 112 and other software programs are installed in the processor 110, so that the processor 110 can access the memory 120 and various peripheral component interconnection standard (Peripheral Component Interconnect Express, PCIe) devices. The processor 110 includes one or more processor cores 111. The processor core 111 in the processor 110 is, for example, a central processing unit (CPU) or other specific integrated circuit (Application Specific Integrated Circuit, ASIC). The processor 110 can also be other general-purpose processors, digital signal processors (digital signal processing, DSP), application-specific integrated circuits (application-specific integrated circuits, ASIC), field programmable gate arrays (field programmable gate arrays, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. In practical applications, the computer device 100 may also include multiple processors.
[0025] In an embodiment of the present invention, the processor 110 is used to send a speed limit optimization request to the data processing unit 160. The processor 110 may also send a speed limit optimization request to other dedicated processors. A dedicated processor is a processor for a specific application or field, such as a graphics processing unit (GPU) for processing image data or a DSP for processing signals.
[0026] The memory 120 is the main memory of the computer device 100. The memory 120 is usually used to store various running software in the operating system 112, input and output data, and information exchanged with external memory. In order to improve the access speed of the processor 110, the memory 120 needs to have the advantage of fast access speed. In the computer system architecture, dynamic random access memory (DRAM) is used as the memory 120. The processor 110 can access the memory 120 at high speed through the memory controller 130, and perform read and write operations on any storage unit in the memory 120. In addition to DRAM, the memory 120 can also be other random access memories, such as static random access memory (SRAM). In addition, the memory 120 can also be a read-only memory (ROM). As for the read-only memory, for example, it can be a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), etc. This embodiment does not limit the number and type of memory 120. In addition, the memory 120 can be configured to have a power-saving function. The power-saving function means that when the system loses power and then powers on again, the data stored in the memory will not be lost. The memory 120 with the power-saving function is called a non-volatile memory.
[0027] The memory controller 130 is a bus circuit controller that controls the memory 120 inside the computer device 100 and is used to manage and plan data transmission from the memory 120 to the processor core 111. Through the memory controller 130, data can be exchanged between the memory 120 and the processor core 111. The memory controller 130 can be a separate chip and connected to the processor core 111 through the system bus. The memory controller 130 can also be integrated into the processor 110, or it can be built into the north bridge, or it can be an independent memory controller chip. The embodiment of the present invention does not limit the specific location and existence form of the memory controller. In actual applications, the memory controller 130 can control the necessary logic to write data to the memory 120 or read data from the memory 120. The memory controller 130 can be a memory controller in a processor system such as a general-purpose processor, a dedicated accelerator, a GPU, an FPGA, an embedded processor, etc.
[0028] The computer device 100 also includes various input / output (I / O) devices 140. The I / O device 140 refers to hardware for data transmission, and can also be understood as a device connected to an I / O interface. Common I / O devices include network cards, printers, keyboards, mice, etc. All external memories can also be used as I / O devices, such as hard disks, floppy disks, optical disks, etc.
[0029] The processor 110, the memory 120, the memory controller 130 and the I / O device 140 are connected via a bus 150. The bus 150 may include a path for transmitting information between the above components (such as the processor 110 and the memory 120). In addition to the data bus, the bus 150 may also include a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are marked as bus 150 in the figure. Bus 150 may be a PCIe bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a computer express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. For example, the processor 110 can access these I / O devices 140 via the PCIe bus. The processor 110 is connected to the memory 120 via a double data rate (DDR) bus. Here, different memories 120 may use different data buses to communicate with the processor 110, so the DDR bus may also be replaced by other types of data buses, and the embodiment of the present invention does not limit the bus type.
[0030] It should be noted that the computer device 100 may be referred to as a host. Figure 1 This is just a schematic diagram. The computer device 100 may also include other components, such as a hard disk, an optical drive, a power supply, a chassis, a cooling system, and other input and output controllers and interfaces. Figure 1 The embodiment of the present invention does not limit the number of processors, memories, and memory controllers included in the computer device 100.
[0031] The computer device 100 further includes a data processing unit 160, which can be connected to the processor 110 via a PCIe bus. The data processing unit 160 unloads applications related to artificial intelligence, storage, etc., which are run by other chips (such as the processor 110) of the computer device 100, thereby improving the data processing performance of the computer device 100 and reducing the load of the computer device 100.
[0032] Next, combine Figure 2 The process of dynamic speed limit optimization of the bottleneck section of the highway reconstruction and expansion run by the data processing unit 160 is described. Figure 2 As shown, the method for optimizing dynamic speed limit of bottleneck section of highway reconstruction and expansion provided by the embodiment of the present invention may include the following steps: S210: Data collection; (1) Traffic data collection: Install traffic flow monitoring radar equipment on various sections along the highway reconstruction and expansion route to collect real-time traffic data such as traffic flow (F), vehicle speed (V) and lane occupancy rate (D). In particular, more sensors are deployed at bottleneck locations where two lanes are reduced to a single lane to ensure the comprehensiveness and accuracy of the data.
[0033] In the process of collecting data, the traffic flow (F), vehicle speed (V) and lane occupancy (D) can be collected based on the preset data sampling frequency, and then based on the preset decision time step t, data is collected once and a decision is made every decision time step t, so that dynamic speed limit can be achieved.
[0034] (2) Data transmission and processing: The above data is transmitted to the central processing system through a wireless communication network (such as 5G). The central processing system pre-processes the data, including denoising, normalization and other operations, to ensure the quality and consistency of the data.
[0035] It is worth noting that, during the preprocessing process, only one operation can be selected for preprocessing, or other preprocessing operations can be added to meet different needs.
[0036] S220: status indication; The preprocessed data is converted into the state representation of the reinforcement learning model, including the average speed of each road section, lane occupancy rate, etc. These state representations should be able to fully reflect the current traffic conditions and provide a basis for subsequent decision-making. It can be defined as: ,in, and They represent the average vehicle speed and average occupancy of road section i at decision time step t respectively.
[0037] S230: Action definition; Define the speed limit adjustment action space A, including multiple speed limit value options (such as 40, 50, 60, 70, 80 km / h, the speed limit value options here are only for example, and other values can also be used as speed limit values). Represents the speed limit value selected for the speed limit control section of the expressway at time t.
[0038] S240: reward function; Designing the reward function , with the goal of increasing the maximum throughput downstream of the highway expansion bottleneck. The reward function can be defined as: ,in, Take action The downstream traffic flow at time t+1. The greater the downstream traffic flow, the higher the reward, and vice versa.
[0039] like Figure 3 As shown, the embodiment of the present invention explains the road environment of dynamic speed limit at the bottleneck of highway reconstruction and expansion, so as to better understand the technical solution described in the embodiment of the present invention.
[0040] The description of each section is as follows: (1) Upstream road section: Obtain upstream traffic status data, such as real-time collection of traffic flow, vehicle speed, lane occupancy rate and other data.
[0041] (2) Speed limit controlled sections: Obtain traffic status data of speed limit controlled areas and the layout of variable speed limit signs.
[0042] (3) Bottleneck sections: Obtain traffic status data on bottleneck sections of highway reconstruction and expansion.
[0043] (4) Downstream section: Obtain the number of vehicles passing downstream.
[0044] S250: Strategy optimization; The Proximal Policy Optimization (PPO) algorithm is used to optimize the speed limit adjustment strategy. The PPO algorithm updates the strategy through the following steps: (1) Policy Network : The policy network output is in state Take action The policy network uses a deep neural network structure to handle complex state space and action space.
[0045] (2) Value Network : Value network estimated state , helping to measure the potential benefits of the current state.
[0046] (3) Advantage function : It is used to measure the deviation between the actual action taken and the expected strategy. The calculation formula is: ; in, is a discount factor that balances the weights of current rewards and future rewards. Represents the state representation at decision time step t+1 value.
[0047] (4) Loss function: The goal of the PPO algorithm is to maximize the following loss function : ; in, Represents the expected value at time step t. min represents the minimum value. Indicates the current policy parameters Next, status Next select action probability. Indicates that under the old policy parameters, the status Next select action probability. is a clipping function that limits the range of probability ratio changes to prevent instability caused by excessive updates. ϵ is the clipping coefficient, usually a small positive number such as 0.1 or 0.2.
[0048] (5) Strategy update: Optimize the loss function through gradient descent method , update the policy network parameters and value network parameters After each update, the policy network becomes more likely to choose actions that have performed well in the past.
[0049] S260: Deployment and real-time adjustment; The optimized speed limit adjustment strategy is deployed to the dynamic speed limit system of the highway to monitor the traffic conditions in real time and dynamically adjust the speed limit value of the speed limit control section. The specific implementation method is to regularly update the strategy and adjust the speed limit according to the latest traffic data. The deployment process needs to ensure the reliability and response speed of the system to meet the real-time management needs of the highway.
[0050] Compared with the prior art, the present invention has the following advantages and effects: (1) Intelligent and dynamic speed limit adjustment: The present invention uses a reinforcement learning algorithm to dynamically adjust the speed limit on highways according to real-time traffic conditions and environmental conditions, avoiding the limitations of traditional fixed speed limit methods and improving traffic flow and safety.
[0051] (2) Wide applicability: The PPO algorithm can handle high-dimensional states and discrete action spaces. It is suitable for various complex traffic environments and highway reconstruction and expansion scenarios and has wide applicability.
[0052] (3) Efficient and stable strategy optimization: The PPO algorithm introduces a clipping objective function to limit the step size of each strategy update, thereby ensuring the stability and efficiency of the strategy optimization process and avoiding the unstable strategy update problem common in other algorithms.
[0053] (4) Long-term adaptability and optimization capability: During actual operation, the system continuously collects and processes data, and constantly updates and optimizes the speed limit adjustment strategy to ensure that the system can adapt to long-term traffic and environmental changes and achieve sustainable traffic management optimization.
[0054] In summary, the present invention provides a dynamic speed limit method for highway reconstruction and expansion based on reinforcement learning algorithm. Through intelligent and dynamic speed limit adjustment, it effectively improves the traffic flow and safety of the highway, and has broad application prospects and significant technical advantages.
[0055] It is understandable that in order to implement the functions in the above embodiments, the computing device includes hardware structures and / or software modules corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and method steps of each example described in the embodiments disclosed in the present invention, the present invention can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.
[0056] Combined with the above Figure 2 and Figure 3 The method for optimizing the dynamic speed limit of the bottleneck section of the highway reconstruction and expansion provided by this embodiment is described in detail. Figure 4 , describes the dynamic speed limit optimization device for bottleneck sections of highway reconstruction and expansion provided by this embodiment.
[0057] Figure 4 A schematic diagram of the structure of a possible device for optimizing the dynamic speed limit of a bottleneck section of a highway reconstruction and expansion provided in this embodiment. These devices for optimizing the dynamic speed limit of a bottleneck section of a highway reconstruction and expansion can be used to implement the functions of the processor in the above method embodiment, and thus can also achieve the beneficial effects of the above method embodiment. In this embodiment, the device for optimizing the dynamic speed limit of a bottleneck section of a highway reconstruction and expansion can be a separate data processing module, or a module (such as a chip) applied to a server.
[0058] like Figure 4 As shown, the dynamic speed limit optimization device 400 for bottleneck section of highway reconstruction and expansion includes a data acquisition module 410, a data processing module 420 and a dynamic speed limit optimization module 430; The data collection module 410 is used to collect traffic data of each bottleneck section of the target highway to be rebuilt and expanded, and pre-process the traffic data to obtain the pre-processed traffic data; The data processing module 420 is used to determine the state representation corresponding to each reconstruction and expansion bottleneck section according to the pre-processed traffic data, define an action with a speed limit value, and determine the reward function corresponding to each reconstruction and expansion bottleneck section according to the state representation and the action, so as to obtain the state representation, action and reward function corresponding to each reconstruction and expansion bottleneck section; The dynamic speed limit optimization module 430 is used to optimize the speed limit adjustment strategy corresponding to each bottleneck section of the reconstruction and expansion based on the state representation, action and reward function corresponding to each bottleneck section of the reconstruction and expansion, and to complete the dynamic speed limit optimization of the bottleneck section of the highway.
[0059] It should be understood that the apparatus 400 for optimizing the dynamic speed limit of the bottleneck section of the highway reconstruction and expansion according to the embodiment of the present invention can be implemented by the data processing unit 160. The apparatus 400 for optimizing the dynamic speed limit of the bottleneck section of the highway reconstruction and expansion according to the embodiment of the present invention can correspond to the method described in the embodiment of the present invention, and the above and other operations and / or functions of each unit in the apparatus 400 for optimizing the dynamic speed limit of the bottleneck section of the highway reconstruction and expansion are respectively to achieve Figure 2 For the sake of brevity, the corresponding processes of each method in are not repeated here.
[0060] Figure 5 A schematic diagram of the structure of a computing device 500 provided in this embodiment. As shown in the figure, the computing device 500 includes a second processor 510, a second bus 520, a memory 530, a memory unit 550 (also referred to as a main memory unit), and a communication interface 540. The second processor 510, the memory 530, the memory unit 550, and the communication interface 540 are connected via the second bus 520.
[0061] It should be understood that in this embodiment, the second processor 510 may be a CPU, and the second processor 510 may also be other general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0062] The communication interface 540 is used to implement communication between the computing device 500 and an external device or component. In this embodiment, the communication interface 540 is used to perform data exchange with other computing devices.
[0063] The second bus 520 may include a path for transmitting information between the above components (such as the second processor 510, the memory unit 550 and the memory 530). In addition to the data bus, the second bus 520 may also include a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are marked as the second bus 520 in the figure. The second bus 520 can be a Peripheral Component Interconnect Express (PCIe) bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a computer express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc.
[0064] As an example, computing device 500 may include multiple processors. The processor may be a multi-core (multi-CPU) processor. The processor here may refer to one or more devices, circuits, and / or computing units for processing data (such as computer program instructions).
[0065] It is worth mentioning that Figure 5 In the example, the computing device 500 includes one second processor 510 and one memory 530. Here, the second processor 510 and the memory 530 are respectively used to indicate a type of device or equipment. In a specific embodiment, the number of each type of device or equipment can be determined according to business requirements.
[0066] The memory unit 550 may correspond to the storage medium for storing information such as traffic data in the above method embodiment. The memory unit 550 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).
[0067] The memory 530 is used to store the new version of the first application program, and may be a solid state drive or a mechanical hard disk.
[0068] It should be understood that the computing device 500 may be a data processing unit 160. The computing device 500 according to this embodiment may correspond to the apparatus 400 for optimizing the dynamic speed limit of the bottleneck section of the highway reconstruction and expansion in this embodiment, and may correspond to the apparatus for executing the Figure 2 The corresponding subject in the above and other operations and / or functions of each module in the highway expansion bottleneck section dynamic speed limit optimization device 400 are respectively to achieve Figure 2 For the sake of brevity, the corresponding process in will not be repeated here.
[0069] The method steps in this embodiment can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a computing device. Of course, the processor and the storage medium can also exist as discrete components in a network device or a terminal device.
[0070] The present invention also provides a chip system, which includes a processor for implementing the functions of the data processing unit 160 in the above method. In a possible design, the chip system also includes a memory for storing program instructions and / or data. The chip system can be composed of a chip, or can include a chip and other discrete devices.
[0071] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present invention is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instruction may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, a hard disk, or a tape; it may also be an optical medium, such as a digital video disc (DVD); it may also be a semiconductor medium, such as a solid state drive (SSD).
[0072] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A method for optimizing dynamic speed limit in bottleneck sections of highway reconstruction and expansion, characterized in that: include: Collecting traffic data of each bottleneck section of the target expressway to be rebuilt or expanded, and preprocessing the traffic data to obtain preprocessed traffic data; Determine the state representation corresponding to each rebuilt and expanded bottleneck section according to the preprocessed traffic data, define the action according to the speed limit value, and determine the reward function corresponding to each rebuilt and expanded bottleneck section according to the state representation and the action, so as to obtain the state representation, action and reward function corresponding to each rebuilt and expanded bottleneck section; Based on the state representation, action and reward function corresponding to each bottleneck section of reconstruction and expansion, the proximal strategy optimization algorithm is used to optimize the speed limit adjustment strategy corresponding to each bottleneck section of reconstruction and expansion, so as to complete the dynamic speed limit optimization of the bottleneck section of highway reconstruction and expansion.
2. The method for optimizing dynamic speed limit in bottleneck section of highway reconstruction and expansion according to claim 1 is characterized in that: Collect traffic data for each bottleneck section of the target highway to be rebuilt or expanded, including: For each bottleneck section of the target highway that is undergoing reconstruction and expansion, the traffic flow, vehicle speed and lane occupancy rate corresponding to the bottleneck section are collected to obtain the traffic data corresponding to each bottleneck section that is undergoing reconstruction and expansion.
3. The method for optimizing dynamic speed limit of bottleneck section of highway reconstruction and expansion according to claim 1 is characterized in that: After the traffic data is preprocessed, the preprocessed traffic data is obtained, including: The traffic data is normalized and / or denoised to obtain pre-processed traffic data.
4. The method for optimizing dynamic speed limit of bottleneck section of expressway reconstruction and expansion according to claim 2 is characterized in that: Determining the state representation corresponding to each bottleneck road section to be rebuilt or expanded according to the pre-processed traffic data includes: Based on the pre-processed traffic data, determine the average vehicle speed and average lane occupancy rate corresponding to each bottleneck road section to be rebuilt or expanded at the decision time step; According to the average vehicle speed and average lane occupancy rate of each bottleneck section to be rebuilt and expanded at the decision time step, the state corresponding to each bottleneck section to be rebuilt and expanded is determined as follows: ;in, represents the average vehicle speed of the bottleneck section i at the decision time step t, represents the average lane occupancy rate of the bottleneck road section i at the decision time step t; , I represents the total number of bottleneck sections to be rebuilt or expanded in the target expressway.
5. The method for optimizing dynamic speed limit of bottleneck section of highway reconstruction and expansion according to claim 4 is characterized in that: Define actions based on speed limits, including: A speed limit adjustment action space A is constructed using multiple speed limit values; Define actions for each bottleneck section to be renovated or expanded , which represents the speed limit value selected for the speed limit controlled bottleneck section at time t.
6. The method for optimizing dynamic speed limit in bottleneck section of highway reconstruction and expansion according to claim 5 is characterized in that: According to the state representation and action, the reward function corresponding to each bottleneck section to be rebuilt and expanded is determined as follows: ;in, Indicates taking action Then the downstream traffic flow at time t+1.
7. The method for optimizing dynamic speed limit of bottleneck section of expressway reconstruction and expansion according to claim 6 is characterized in that: Based on the state representation, action and reward function corresponding to each bottleneck section of the reconstruction and expansion, the proximal strategy optimization algorithm is used to optimize the speed limit adjustment strategy corresponding to each bottleneck section of the reconstruction and expansion, and the dynamic speed limit optimization of the bottleneck section of the highway reconstruction and expansion is completed, including: For each bottleneck section to be rebuilt or expanded, the following status is used to represent And actions As the input of the policy network, to obtain the state of the policy network output Take action Probability ; In the state indicated As the input of the value network, it evaluates the state representation at the decision time step t The value of ; According to the status The value of And the reward function , determine the advantage function ; According to the probability And the advantage function , determine the loss function ; The gradient descent method is used to adjust the policy network parameters and value network parameters Update to minimize the loss function , and take the action corresponding to the maximum probability output by the strategy network at each decision time step t as the corresponding speed limit at the next decision time step t, and obtain the optimized speed limit adjustment strategy; The optimized speed limit adjustment strategy is deployed in the dynamic speed limit system of the highway. The traffic data is updated at each decision time step t to obtain the corresponding speed limit at the next decision time step t, thereby realizing dynamic speed limit optimization of bottleneck sections in highway reconstruction and expansion.
8. The method for optimizing dynamic speed limit in bottleneck section of highway reconstruction and expansion according to claim 7 is characterized in that: According to the status The value of And the reward function , determine the advantage function for: ; in, represents the discount factor to balance the weights of current rewards and future rewards, Represents the state representation at decision time step t+1 value.
9. The method for optimizing dynamic speed limit in bottleneck section of highway reconstruction and expansion according to claim 7 is characterized in that: According to the probability And the advantage function , determine the loss function for: ; in, represents the expected value at the decision time step t; min represents the minimum value; Indicates the current policy parameters Next, status Next select action The probability of Indicates that under the old policy parameters, the status Next select action The probability of is a clipping function, which is used to limit the range of variation of the probability ratio to prevent instability caused by excessive updates; ϵ is the clipping coefficient.
10. A dynamic speed limit optimization device for bottleneck sections of highway reconstruction and expansion, characterized in that: include: Data acquisition module, data processing module and dynamic speed limit optimization module; The data collection module is used to collect traffic data of each bottleneck section of the target expressway to be rebuilt and expanded, and pre-process the traffic data to obtain the pre-processed traffic data; The data processing module is used to determine the state representation corresponding to each reconstruction and expansion bottleneck section according to the pre-processed traffic data, define an action with a speed limit value, and determine the reward function corresponding to each reconstruction and expansion bottleneck section according to the state representation and the action, so as to obtain the state representation, action and reward function corresponding to each reconstruction and expansion bottleneck section; The dynamic speed limit optimization module is used to optimize the speed limit adjustment strategy corresponding to each bottleneck section of the reconstruction and expansion based on the state representation, action and reward function corresponding to each bottleneck section of the reconstruction and expansion, and to complete the dynamic speed limit optimization of the bottleneck section of the highway reconstruction and expansion.
Citation Information
Patent Citations
Variable speed-limit control method of expressway isolate bottleneck road section
CN106128095A
Traffic signal control method based on deep reinforcement learning
CN112216124A
Vehicle ramp entrance confluence control method based on deep reinforcement learning
CN116215532A
Signal lamp intersection passing method based on near-end strategy optimization
CN116884246A
Highway differential variable speed limit control method and device based on deep reinforcement learning and storage medium
CN117496721A
