Gravity center adjustable intelligent traffic cone based on deep reinforcement learning

Abstract

According to the gravity center adjustable intelligent traffic cone based on deep reinforcement learning provided by the invention, a deep reinforcement learning algorithm is adopted, so that the gravity center position of the traffic cone can be automatically adjusted according to the magnitude of wind power and the change of external impact force, and thus the stability is kept. The intelligent traffic cone can resist the influence of high-speed airflow in an expressway construction area through self-learning and adaptation, keeps stable in an area with large wind power, and meanwhile achieves rapid deployment and environment adaptation in urban road construction. In addition, the intelligent traffic cone has the advantages of reducing manual intervention, reducing maintenance cost, improving construction efficiency and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of road safety equipment, and in particular to a gravity-adjustable smart traffic cone based on deep reinforcement learning. Background Art

[0002] With the acceleration of urbanization and the continuous expansion of transportation networks, road construction safety has become a key link in modern traffic management. As an economical and practical road construction safety facility, traditional traffic cones have played an important role in traffic guidance and warning, but their lack of stability has become increasingly prominent in practical applications. Especially under the influence of external factors such as high-speed airflow, strong wind or impact, traditional traffic cones are prone to tipping over, which not only affects the construction progress, but also increases the risk of traffic accidents. In order to improve the safety of construction sites and reduce traffic accidents caused by tipping traffic cones, it is particularly important to invent a smart traffic cone that is highly adaptable, can cope with various complex environments, and intelligently adjust its own structure. Summary of the invention

[0003] In order to improve the safety of construction sites and reduce traffic accidents caused by accidental dumping of traffic cones, the present invention proposes a smart traffic cone with adjustable center of gravity based on deep reinforcement learning. The deep reinforcement learning algorithm is used to enable the traffic cone to automatically adjust its center of gravity position according to changes in wind force and external impact force, thereby maintaining stability. The smart traffic cone of the present invention can resist the influence of high-speed airflow in highway construction areas through self-learning and adaptation, remain stable in areas with strong winds, and achieve rapid deployment and environmental adaptation in urban road construction. In addition, the smart traffic cone of the present invention has the advantages of reducing manual intervention, reducing maintenance costs, and improving construction efficiency.

[0004] A gravity-adjustable intelligent traffic cone based on deep reinforcement learning, comprising: a sensor module, a control unit and a hydraulic system, wherein: The sensor module includes an inclination sensor, an accelerometer, a wind sensor and a position sensor; The control unit includes a deep reinforcement learning algorithm model; The hydraulic system includes a hydraulic pump, a control valve, a hydraulic cylinder, a liquid storage chamber and pipelines.

[0005] Furthermore, the sensor module monitors and collects the tilt angle of the traffic cone, acceleration in all directions, and surrounding wind speed and wind direction data in real time.

[0006] Furthermore, the tilt angle threshold determination method is used to determine in real time whether the traffic cone is stable. If it is stable, it is continuously monitored. If it is unstable, the center of gravity needs to be adjusted. The sensor module transmits the collected data to the control unit. Determine stability. Determined to be unstable, Indicated in The tilt angle of the traffic cone at the moment, The threshold angle for whether the traffic cone is stable.

[0007] Furthermore, the control unit processes the collected relevant data through a deep deterministic policy gradient algorithm and decides on the best center of gravity adjustment strategy. The control unit sends instructions to the control valve of the hydraulic system to start the movement of the liquid. The liquid is sucked from the liquid storage chamber and pushed to the hydraulic cylinders of various parts of the system, thereby affecting the center of gravity position of the traffic cone.

[0008] Furthermore, the sensor module continuously detects the adjusted state. After judgment by the tilt angle threshold determination method, if the judgment result indicates that the traffic cone is still in an unstable state, the sensor continues to feed back data to the control unit. The control unit has a built-in deep deterministic policy gradient algorithm that adjusts the action according to the feedback to ensure that the traffic cone reaches the expected stability state.

[0009] The system continuously cycles through the above steps, and continuously learns and adjusts through the DDPG algorithm, so that the traffic cones can adapt to various external environments and always remain stable.

[0010] The beneficial effects of the present invention are as follows: The present invention provides an integrated intelligent traffic cone, which monitors and collects changes in the external environment, such as wind or impact, and the state of the traffic cone itself, such as the tilt angle and center of gravity position and other related data in real time through a sensor module; then uses a tilt angle threshold determination method to determine in real time whether the traffic cone is stable. If the traffic cone remains stable, the system will continue to monitor. If the sensor module detects that the tilt angle of the traffic cone exceeds the preset stability threshold, the control unit adjusts the distribution of the liquid inside the traffic cone through the hydraulic system according to the learning results of the DDPG algorithm, so as to change the center of gravity position of the traffic cone and restore its stability. Through the coordinated work of the sensor module, the control unit and the hydraulic system, the present invention enables the intelligent traffic cone to adapt to various complex environmental conditions and automatically find the optimal center of gravity position, thereby improving its stability and safety. This not only helps to improve the safety level of road construction, but also reduces manual intervention, reduces maintenance costs, and improves construction efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a flow chart of the center of gravity adjustable smart traffic cone based on deep reinforcement learning of the present invention.

[0012] Figure 2 This is a diagram of the deep deterministic policy gradient (DDPG) algorithm in the control unit of the present invention. DETAILED DESCRIPTION

[0013] The following is a further detailed description of a gravity-adjustable intelligent traffic cone based on deep reinforcement learning of the present invention in conjunction with the accompanying drawings and specific embodiments: Example

[0014] The present invention proposes a Figure 1 The center-of-gravity adjustable intelligent traffic cone system based on deep reinforcement learning shown in the paper provides a new idea and method for research in the field of intelligent transportation.

[0015] S101. The sensor module includes an inclination sensor, an accelerometer, a wind sensor, and a position sensor.

[0016] The tilt sensor is used to measure the tilt angle of the traffic cone to the ground. When the traffic cone tilts due to wind or vehicle impact, the tilt sensor will immediately detect this change and provide the control unit with the precise angle of the traffic cone relative to the horizontal plane.

[0017] The accelerometer is used to capture the changes in the acceleration of the traffic cone in three spatial dimensions. It can detect not only the tilt of the traffic cone, but also any sudden movement or impact, such as a vehicle collision. By analyzing the acceleration data, the control unit is able to identify the dynamic behavior of the traffic cone and quickly make corresponding adjustments to maintain balance.

[0018] Wind sensors are used to monitor the wind speed and direction around traffic cones in real time. Since wind is one of the main external factors that cause instability of traffic cones, real-time monitoring of wind force is essential to maintain the stability of traffic cones. These data will be input into the control unit so that the center of gravity of the traffic cones can be adjusted in time.

[0019] Position sensor for determining the center of gravity of the traffic cone. In the present invention, an ultrasonic sensor is used to measure the position of the liquid (a part of the hydraulic system) inside the traffic cone, thereby indirectly determining the center of gravity of the entire traffic cone.

[0020] The sensor module monitors and collects relevant data such as inclination, acceleration, wind force and center of gravity in real time, and evaluates the stability of the traffic cone in real time through the inclination angle threshold judgment method. If the traffic cone remains stable, the system will continue to monitor; if it detects that the traffic cone is unstable, the system will start the adjustment program to ensure that the traffic cone remains stable in a changing environment.

[0021] S102. Deep reinforcement learning algorithm in control unit We use the deep deterministic policy gradient (DDPG) algorithm that can handle continuous problems for center of gravity adjustment to ensure the stability of traffic cones. Figure 2As shown in the figure, DDPG consists of an Actor network and a Critic network. The Actor network includes a policy network and a target policy network, and the Critic network includes a Q network and a target Q network.

[0022] Traffic cones, as intelligent agents, need to adjust their center of gravity according to the external environment state collected by sensors to ensure stability. The state space of the system is defined as ,in Indicated in The tilt angle of the traffic cone at the moment, Traffic cones indicate The total acceleration at that moment is Indicated in The wind force on the traffic cone at the moment, that is Indicates wind speed, Indicates the wind direction angle, Indicates wind force, Indicated in The center of gravity position of the traffic cone at the moment. Since DDPG can solve the problem of continuous action, the center of gravity adjustment of the traffic cone can change continuously, and the action can be expressed as a real value, representing the adjustment amount of the center of gravity. In my country, the standard size of traffic cones is generally 100 cm or 120 cm, so the action space is defined as , Indicated in The center of gravity position after adjustment at all times, The reward function is defined as is the adjusted traffic cone tilt angle, The threshold for whether the traffic cone is stable.

[0023] The goal of DDPG is to find an optimal strategy , so that the expected long-term return is maximized, that is ,in is the discount factor, is the weight parameter of the Q network.

[0024] DDPG adjusts the center of gravity of the traffic cone through the cooperation of the Actor network and the Critic network to improve its stability. Observe the local state of your surroundings , and then make a decision , get rewarded and environmental status In order to increase the exploration rate, a noise term is introduced into the output of the deterministic strategy ,Right now ,Bundle Stored in the experience replay pool In. From The number of random sampling batches is Multidimensional array of for training.

[0025] Next, Q Network The Q value is evaluated based on the current action. The Q network is then trained with the estimated Q value and the target Q value, and the policy network is updated based on the feedback from the Q network. At each step the following minimum loss function is used for updating:

[0026] in, For the policy network, the training goal is to make The expected value of is maximized, that is .parameter Update by sampling gradients, i.e.

[0027] The target policy network and target Q network are updated using the soft update method, that is,

[0028] in, is the learning rate.

[0029] S103, the hydraulic system adjusts the center of gravity of the traffic cone according to the command of the control unit. In the present invention, the hydraulic system changes the center of gravity of the traffic cone by moving the internal liquid, thereby achieving stability adjustment. The hydraulic system is composed of a hydraulic pump, a control valve, a hydraulic cylinder, a liquid storage chamber, a pipeline, etc.

[0030] The hydraulic pump, as the power core of the system, is responsible for providing the required pressure for the entire system; it is mainly responsible for sucking in the liquid stored in the liquid storage chamber and pushing it to other parts of the system.

[0031] Control valves are mainly used to regulate the flow path and flow rate of liquid in hydraulic systems. They open or close the channel to a specific hydraulic cylinder according to the instructions of the control unit.

[0032] The hydraulic cylinder, as the actuator of the system, plays a vital role. When the hydraulic pump presses the liquid into the hydraulic cylinder, the piston will move accordingly. This action can change the distribution of the liquid inside the traffic cone, thereby changing the center of gravity of the entire traffic cone.

[0033] The liquid storage chamber, as an important part of the system, is mainly responsible for storing the fluid required by the hydraulic system. Generally, the liquid stored in the storage chamber is a relatively heavy and non-volatile fluid, such as water or oil.

[0034] Pipes are used to connect various parts of the system and are responsible for transmitting liquids.

[0035] Through the design of the above invention, we can understand the adaptive adjustment effect of the proposed DDPG algorithm model. According to the performance and stability of the model, the hyperparameters of the model are adjusted to achieve the best adjustment effect and ensure the stability of the traffic cone.

[0036] It should be noted that the design and implementation of this algorithm model should be adjusted and optimized according to specific application scenarios and requirements. Different application scenarios may require different algorithms and technical means to achieve the best adaptive adjustment effect of traffic cones in different environments.

[0037] In summary, the present invention proposes an innovative smart traffic cone, which realizes the intelligent adjustment of the center of gravity of the traffic cone by integrating a sensor module, a control unit and a hydraulic system. The sensor monitors and collects the tilt of the traffic cone and the changes in the external environment, such as wind force or impact force, in real time, and determines whether the traffic cone is stable in real time through the tilt angle threshold judgment method. If it is stable, it will continue to monitor. If it is unstable, the data collected by the sensor will be transmitted to the control unit, which uses the deep deterministic policy gradient (DDPG) algorithm to decide the best center of gravity adjustment strategy. The control unit then commands the hydraulic system to restore balance by moving the internal fluid. The control valve of the hydraulic system adjusts the flow of fluid to a specific hydraulic cylinder according to the command, thereby pushing the piston to move and changing the center of gravity position of the traffic cone. For example, if the traffic cone tilts in one direction, the control unit may instruct the hydraulic system to transfer the fluid to a container in the opposite direction to offset the tilt and restore balance. This automated process not only reduces manual intervention and maintenance costs, but also significantly improves construction efficiency and safety, enabling smart traffic cones to adapt to various environmental conditions, including high-speed airflow in highway construction areas and rapid deployment requirements for urban road construction.

[0038] The examples of the present invention are described in detail above in conjunction with the embodiments, but the present invention is not limited to the above examples. Various changes can be made within the knowledge scope of ordinary technicians in the field without departing from the purpose of the present invention, and should also be regarded as the protection scope of the present invention.

Claims

1. A gravity-adjustable intelligent traffic cone based on deep reinforcement learning, characterized in that: include: Sensor modules, control units and hydraulic systems, whereby: The sensor module includes an inclination sensor, an accelerometer, a wind sensor and a position sensor; The control unit includes a deep reinforcement learning algorithm model; The hydraulic system includes a hydraulic pump, a control valve, a hydraulic cylinder, a liquid storage chamber and pipelines.

2. The center-of-gravity adjustable intelligent traffic cone based on deep reinforcement learning according to claim 1 is characterized in that: The sensor module monitors and collects the tilt angle of the traffic cone, acceleration in all directions, and surrounding wind speed and wind direction data in real time.

3. The center-of-gravity adjustable intelligent traffic cone based on deep reinforcement learning according to claim 2 is characterized in that: The tilt angle threshold method is used to determine in real time whether the traffic cone is stable. If it is stable, it is continuously monitored. If it is unstable, the center of gravity needs to be adjusted. The sensor module transmits the collected data to the control unit. Determine stability. Determined to be unstable, Indicated in The tilt angle of the traffic cone at the moment, The threshold angle for whether the traffic cone is stable.

4. The center-of-gravity adjustable intelligent traffic cone based on deep reinforcement learning according to claim 3 is characterized in that: The control unit processes the collected relevant data through a deep deterministic policy gradient algorithm and decides on the optimal center of gravity adjustment strategy. The control unit sends instructions to the control valve of the hydraulic system to start the movement of the liquid. The liquid is sucked from the liquid storage chamber and pushed to the hydraulic cylinders of various parts of the system, thereby affecting the center of gravity position of the traffic cone.

5. The center-of-gravity adjustable intelligent traffic cone based on deep reinforcement learning according to claim 4 is characterized in that: The sensor module continuously detects the adjusted state. After judgment by the tilt angle threshold determination method, if the judgment result shows that the traffic cone is still in an unstable state, the sensor continues to feed back data to the control unit. The control unit has a built-in deep deterministic policy gradient algorithm to adjust the action according to the feedback to ensure that the traffic cone reaches the expected stability state.

Images