Vehicle speed planning method under double traffic lights based on reinforcement learning
Through the vehicle speed planning method based on reinforcement learning, the vehicle's driving conditions at the intersection of the signal light and adjusts the vehicle speed, the passenger discomfort caused by the sudden acceleration or sudden braking of the autonomous vehicle is solved, and the riding comfort is improved.
Patent Information
- Application Number
- CN202211116672.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-14
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2042-09-14
AI Technical Summary
When autonomous vehicles are on the road, passengers are uncomfortable due to sudden acceleration or sudden braking, and the comfort level is low when riding inside the vehicle.
Using the vehicle speed planning method in the dual signal light state based on reinforcement learning, the vehicle speed is determined by obtaining traffic information on the vehicle's expected driving route and the current vehicle speed, and adjusting the vehicle speed according to the analysis model to maintain it. Stable state.
It improves the ride comfort of passengers inside the vehicle, avoids sudden acceleration or rapid deceleration, and ensures that the vehicle maintains stable driving at the intersection of the signal light.
Smart Images

Figure CN115447608B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of vehicle control, and in particular to a vehicle speed planning method under a dual signal light state based on reinforcement learning. Background Art
[0002] When an autonomous driving vehicle is driving on the road, it needs to keep driving or stop and wait according to the instructions of the traffic signal indicator. When the vehicle passes or reaches the traffic signal indicator, in order to avoid the vehicle missing the green light and passing through the intersection, the control system needs to control the vehicle to accelerate suddenly. There are also vehicles that use emergency braking to avoid running red lights.
[0003] When the vehicle is in a state of sudden acceleration or sudden braking, the passengers inside the vehicle will be shaken back and forth due to inertia, which will bring discomfort to the passengers inside the vehicle and result in low riding comfort for the passengers inside the vehicle. Summary of the invention
[0004] In view of this, the present application provides a vehicle speed planning method under a dual signal light state based on reinforcement learning, aiming to improve the riding comfort of passengers inside the vehicle.
[0005] To achieve the above object, the present application provides a vehicle speed planning method under a dual signal light state based on reinforcement learning, and the vehicle speed planning method under a dual signal light state based on reinforcement learning comprises the following steps:
[0006] Obtaining traffic information on the vehicle's expected travel route and obtaining the vehicle's current speed;
[0007] Based on the traffic information and the current vehicle speed, determining that the vehicle meets a driving condition when driving smoothly at a traffic light intersection;
[0008] Inputting the driving condition into the analysis model, so that the analysis model determines and outputs the speed change required when the vehicle enters the driving condition;
[0009] Based on the speed change, the speed of the vehicle is adjusted to control the vehicle to maintain stable driving.
[0010] Exemplarily, the determining, based on the traffic information and the current vehicle speed, that the vehicle meets the driving condition for stable driving at a traffic light intersection includes:
[0011] Based on the traffic information, determining status information of a preset number of traffic lights on the expected travel route, and determining traffic congestion conditions;
[0012] Based on the state information, determining the distance between each of the signal lights and the vehicle, and determining the signal switching information of each of the signal lights;
[0013] When the traffic congestion condition satisfies the normal passage of the vehicle, it is determined that the vehicle satisfies the driving condition for smooth driving at a traffic light intersection based on the distance, the signal switching information and the current vehicle speed.
[0014] Exemplarily, the driving condition includes a driving condition, and determining the driving condition when the vehicle is driving steadily at a signal light intersection based on the distance, the signal switching information, and the current vehicle speed includes:
[0015] Based on the distance and the current vehicle speed, calculating the time required for the vehicle to pass through the signal light intersection to be passed;
[0016] Based on the time and the signal switching information, determining whether the vehicle can pass through a first signal light intersection to be passed in a stable driving manner;
[0017] If the vehicle can pass through the first signal light intersection to be passed, determining the phase difference of the signal changes between the signal lights;
[0018] Determining the number of signal light intersections that the vehicle is expected to travel through based on the phase difference and the signal switching information;
[0019] Based on the quantities, the corresponding driving situation is determined.
[0020] Exemplarily, the driving condition includes a braking condition, and after determining whether the vehicle can pass through a first signal light intersection to be passed in a stable driving manner based on the time and the signal switching information, the method includes:
[0021] If the vehicle cannot pass through the first signal light intersection to be passed, calculating the deceleration required for the vehicle to brake smoothly;
[0022] Based on the deceleration, a corresponding braking condition is determined.
[0023] Exemplarily, determining the corresponding braking condition based on the deceleration includes:
[0024] If the deceleration is within a preset deceleration range, determining a corresponding braking condition based on the deceleration;
[0025] If the deceleration exceeds the preset deceleration range, determining the maximum deceleration allowed by the preset deceleration range;
[0026] Based on the maximum deceleration, a corresponding braking condition is determined.
[0027] Exemplarily, the calculating, based on the distance and the current vehicle speed, the time required for the vehicle to pass through the signal light intersection to be passed includes:
[0028] Based on the distance and the current vehicle speed, determining a speed range that satisfies the change in the current vehicle speed when the vehicle smoothly drives through a traffic light intersection;
[0029] If the preset speed limit range covers the speed range, the time required for the vehicle to pass the signalized intersection to be passed is calculated based on the speed range;
[0030] If the speed range is equal to the preset speed limit range, the time required for the vehicle to pass through the signal light intersection to be passed is calculated based on the preset speed limit range or the speed range.
[0031] Exemplarily, before inputting the driving condition into the analysis model, the process includes:
[0032] Obtain a data set consisting of multiple driving conditions and obtain reward indicators;
[0033] Inputting the data set into a learning model to train the learning model to record the driving conditions of the vehicle passing through the signal light intersection under different driving conditions;
[0034] Based on the reward index, determining a reward function that returns a reward value when the vehicle passes through the signalized intersection;
[0035] Based on the reward function and the trained learning model, an analysis model for analyzing different driving conditions is generated.
[0036] Exemplarily, to achieve the above purpose, the present application also provides a vehicle speed planning device under a dual signal light state based on reinforcement learning, the device comprising:
[0037] An acquisition module, used to acquire traffic information on the vehicle's expected travel route and to acquire the vehicle's current speed;
[0038] A determination module, configured to determine, based on the traffic information and the current vehicle speed, whether the vehicle satisfies a driving condition for stable driving at a traffic light intersection;
[0039] An input module, used for inputting the driving condition into the analysis model, so that the analysis model can determine and output the speed change required when the vehicle enters the driving condition;
[0040] The adjustment module is used to adjust the speed of the vehicle based on the speed change to control the vehicle to maintain stable driving.
[0041] Exemplarily, to achieve the above-mentioned purpose, the present application also provides a vehicle speed planning device under a dual signal light state based on reinforcement learning, the device comprising: a memory, a processor, and a vehicle speed planning program under a dual signal light state based on reinforcement learning stored on the memory and executable on the processor, the vehicle speed planning program under a dual signal light state based on reinforcement learning being configured to implement the steps of the vehicle speed planning method under a dual signal light state based on reinforcement learning as described above.
[0042] Exemplarily, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, on which is stored a vehicle speed planning program under a dual signal light state based on reinforcement learning, and when the vehicle speed planning program under a dual signal light state based on reinforcement learning is executed by a processor, the steps of the vehicle speed planning method under a dual signal light state based on reinforcement learning as described above are implemented.
[0043] Compared with the prior art, in which an autonomous driving vehicle may suddenly accelerate or brake according to the external environment when traveling on the road, thereby causing the passengers inside the vehicle to move relative to the vehicle due to inertia, resulting in low comfort for the passengers inside the vehicle when riding in the vehicle, in the present application, by obtaining traffic information on the vehicle's expected driving route and the vehicle's current speed, and determining the driving condition of the vehicle when it meets the requirements for stable driving at a traffic light intersection based on the traffic information and the current speed, the driving condition is input into the analysis model, the driving condition is analyzed by the model, and the driving condition is determined and output in combination with the vehicle's current speed. The speed change required when the vehicle enters the driving condition is adjusted according to the speed change, so that the vehicle maintains smooth driving at the traffic light intersection, that is, a corresponding analysis is performed on the conditions of the traffic light intersection that the vehicle will encounter on the expected driving route, the driving condition for the vehicle to drive smoothly at the traffic light intersection is determined, and the adjustment amount of the vehicle speed is determined according to the driving condition, so as to ensure the smooth driving of the vehicle without sudden acceleration or deceleration, thereby ensuring that the passengers inside the vehicle will not have excessive relative movement with the vehicle due to inertia, thereby improving the riding comfort of the passengers inside the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 It is a flowchart of the first embodiment of the vehicle speed planning method under the dual signal light state based on reinforcement learning of the present application;
[0045] Figure 2 This is a flow chart of a second embodiment of a vehicle speed planning method under a dual signal light state based on reinforcement learning in this application;
[0046] Figure 3This is a flow chart of a third embodiment of a vehicle speed planning method under a dual signal light state based on reinforcement learning of the present application;
[0047] Figure 4 This is a schematic diagram of the structure of the hardware operating environment involved in the embodiment of the present application.
[0048] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0049] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0050] This application provides a vehicle speed planning method under dual signal light conditions based on reinforcement learning, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the vehicle speed planning method under dual signal light conditions based on reinforcement learning in this application.
[0051] The embodiment of the present application provides an embodiment of a vehicle speed planning method under a dual signal light state based on reinforcement learning. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that here. For ease of description, the following omits the execution subject to describe the various steps of the vehicle speed planning method under a dual signal light state based on reinforcement learning. The vehicle speed planning method under a dual signal light state based on reinforcement learning includes:
[0052] Step S110: obtaining traffic information on the vehicle's expected travel route, and obtaining the vehicle's current speed;
[0053] The estimated driving route is the driving route planned for the vehicle, which is a route formulated accordingly based on the departure point, destination and road conditions. For example, through GPS (Global Positioning System) positioning or map navigation, the selected route is a road with smooth traffic and no congestion. At the same time, the energy consumption of the vehicle is taken into consideration, and the route selected is a route with low energy consumption.
[0054] The traffic information on the expected driving route is information on the road, including vehicles traveling on the road, the number of traffic light intersections, information on the color changes of traffic lights, positions of traffic lights, positions of vehicles on the road and the number of lanes, etc.
[0055] For example, when acquiring traffic information, the traffic information may be collected through the Internet of Vehicles or through related equipment carried by the vehicle itself.
[0056] Among them, when obtaining traffic information through the Internet of Vehicles, the vehicle exchanges information with other vehicles through the network, and obtains corresponding traffic information (detection of traffic roads by other vehicles and collected related data information) from other vehicles (usually vehicles on the expected driving route, which are in front of the vehicle).
[0057] The vehicle collects and detects the surrounding conditions of the vehicle through the radar, camera, sensor and other equipment carried by the vehicle, and determines the road information around the vehicle, the environment information around the vehicle, the information of other vehicles around the vehicle, etc. Among them, the road information includes the relevant information of the traffic lights.
[0058] The current speed of the vehicle is the real-time speed of the vehicle, which is monitored by detecting the axle speed and other methods.
[0059] Step S120: determining, based on the traffic information and the current vehicle speed, whether the vehicle meets the driving condition for stable driving at a traffic light intersection;
[0060] Based on the traffic information, the information of the traffic light intersection that the vehicle is about to pass and the driving information of other vehicles on the road are determined, and based on the current speed of the vehicle, it can be judged whether the vehicle can pass the traffic light intersection.
[0061] For example, when other vehicles on the road cause congestion, the judgment condition of this phenomenon is that the current speed of the vehicle is lower than a preset judgment threshold (a standard formulated according to actual conditions, which is calibrated in the vehicle system, for example, 10km / h, 15km / h and other low-speed driving conditions of vehicles), and other vehicles around the vehicle are also in a low-speed driving state (this state is monitored by radar, speed sensors and other equipment). At this time, the traffic light is monitored to be green, but the vehicle's driving speed is low, and the vehicle cannot pass the traffic light intersection before the green light ends (determined by calculating the time it takes for the vehicle to reach the traffic light intersection).
[0062] For example, the road condition is good and there is no congestion, but the vehicle is far away from the traffic light intersection and needs to accelerate to pass the traffic light intersection. At the same time, there is a street light state where the vehicle enters the acceleration state but still cannot catch up with the traffic light. The vehicle needs to brake and wait before driving to the current traffic light intersection. That is, the vehicle has a driving state in which it can pass the traffic light intersection and a driving state in which it cannot pass the traffic light intersection. The above two driving states are driving conditions.
[0063] During the driving process, the vehicle may produce sudden acceleration or deceleration, causing the vehicle to quickly pass through the traffic light intersection or stop in front of the zebra crossing to avoid pedestrians. The above situations will cause the vehicle to produce large movements, causing the passengers inside the vehicle to be shaken and feel uncomfortable. Therefore, the vehicle needs to be controlled to enter a stable driving state. The judgment conditions of this state are the current speed of the vehicle and the allowed acceleration and deceleration values. If the acceleration and deceleration are too large, the vehicle will cause unstable driving.
[0064] Therefore, the traffic information and the current speed of the vehicle are used to determine the traffic light conditions at the traffic light intersection ahead on the expected driving road. When the vehicle is driving smoothly (acceleration and deceleration do not exceed the allowable values), it is determined that the vehicle can accelerate through, the vehicle can decelerate through (the vehicle travels at a lower speed, which will reduce fuel consumption) or the vehicle cannot pass and needs to slow down and brake (the vehicle is waiting at the traffic light intersection). The driving conditions include the above two situations, smooth acceleration and smooth deceleration.
[0065] Exemplarily, the determining, based on the traffic information and the current vehicle speed, that the vehicle meets the driving condition for stable driving at a traffic light intersection includes:
[0066] Step a: Based on the traffic information, determining status information of a preset number of traffic lights on the expected driving route, and determining traffic congestion conditions;
[0067] The preset number is at least two or more such as 2 or 3, which is the number of traffic light information obtained when the vehicle exchanges information with other vehicles through the Internet of Vehicles.
[0068] The status information of the traffic light includes the color of the traffic light (red, yellow, green), the location of the traffic light, and the duration of each color of the traffic light (for example, red light 30 seconds, yellow light 10 seconds, green light 30 seconds, etc. Each traffic light intersection will have different duration parameters depending on the traffic flow).
[0069] The traffic congestion situation is determined based on the information of other vehicles, the number of lanes, the current speed of the vehicle, etc. in the traffic information, for example, the smoothness of the vehicle's driving (there are no other vehicles on the road and it can drive smoothly, or there are other vehicles on the road that are congested and cannot drive smoothly).
[0070] By obtaining the status information of a preset number of traffic lights and combining it with the traffic congestion situation, it can be determined whether the vehicle can smoothly pass through two or more traffic light intersections at one time.
[0071] Step b: based on the state information, determining the distance between each of the signal lights and the vehicle, and determining the signal switching information of each of the signal lights;
[0072] The status information includes the positions of the traffic lights, and the positions of a preset number of traffic lights (the preset number is 2 as an example, and when the preset number is 3, the judgment process is the same as when the preset number is 2) are determined respectively, and the two distance parameters between the two traffic lights and the vehicle are determined respectively.
[0073] The distance parameter, combined with the vehicle's current speed, can be used to determine the time required for the vehicle to reach the traffic light intersection.
[0074] At the same time, the signal switching information of the two traffic lights is determined, and the signal switching information includes the current color of the traffic light, the waiting time for the traffic light to switch colors, etc. The signal switching information can be used to determine the length of time (green light duration) and specific time left for vehicles to pass through the traffic light intersection (for example, if the current straight direction is red, it is necessary to wait for the traffic light state of the traffic lights in other directions at the intersection to end before the current straight direction will enter the green light state, and the waiting time is the specific time).
[0075] Step c: When the traffic congestion condition satisfies the normal passage of the vehicle, based on the distance, the signal switching information and the current vehicle speed, it is determined that the vehicle meets the driving condition for smooth driving at a signal light intersection.
[0076] Traffic congestion conditions include smooth traffic conditions, road congestion that prevents smooth traffic, or few vehicles on the road and no obstruction to driving. When the traffic congestion conditions meet the requirements for normal vehicle traffic (vehicles are driving smoothly on the road), it is necessary to determine the driving conditions for the vehicle to drive smoothly at the traffic light intersection based on the distance, signal switching information and current vehicle speed. The driving conditions include the vehicle continuing to drive through the traffic light intersection and the vehicle slowing down and braking and waiting at the traffic light intersection.
[0077] Step S130: inputting the driving condition into the analysis model, so that the analysis model determines and outputs the speed change required when the vehicle enters the driving condition;
[0078] The driving condition is input into the analysis model, and the analysis model is used to analyze the speed change required for the vehicle to meet smooth driving at a traffic light intersection and enter the driving condition (for example, accelerating through a traffic light or decelerating at a traffic light intersection to brake and wait).
[0079] The analysis model is a learning statistics model. The model is trained using relevant training data so that the analysis model can learn the benefits under different states. For example, the state of the vehicle being able to pass through the signal light intersection under different vehicle speeds, different road conditions, and different signal light conditions is taken as the benefit value. This benefit value corresponds to the driving state effect that can be achieved by adjusting the vehicle speed when the road state and signal light state are constant. That is, after analyzing the traffic information and the current speed of the vehicle through the analysis model, the benefit value when the vehicle can pass the signal when the road is smooth and the signal light is continuously green can be determined. This benefit value corresponds to the speed required for the vehicle to pass the signal light intersection. That is, a relationship analysis model between various information states and vehicle actions is constructed.
[0080] Among them, the benefit value will correspond to different action states. The benefit value and different action states are combined into a benefit value table. After the analysis model analyzes the driving condition and obtains the benefit value, the action state corresponding to the benefit value (vehicle speed adjustment, driving state adjustment, etc.) is determined from the benefit value table, thereby determining the speed change required when the vehicle enters the driving condition.
[0081] Exemplarily, the analysis model may be a Markov model.
[0082] Exemplarily, before inputting the driving condition into the analysis model, the process includes:
[0083] Step d: obtaining a data set consisting of multiple driving conditions and obtaining a reward index;
[0084] Driving conditions include a vehicle driving smoothly through a traffic light intersection and a vehicle braking and waiting at a traffic light intersection. It also includes the current state of the traffic light, for example, the traffic light has just turned red, the traffic light is in a yellow state and is about to turn red, the traffic light is about to turn green, or the traffic light is in a green state, etc. It also includes information such as the vehicle's current speed and traffic congestion conditions.
[0085] The data in the dataset include the driving conditions generated by the above different situations.
[0086] The reward index is an index that determines the selection tendency of the analysis model when performing the analysis action. For example, when analyzing the driving conditions, the rules for formulating the reward index are related to the vehicle driving state and the vehicle energy consumption state. When the analysis model analyzes the driving conditions and determines the benefit value, the vehicle speed and vehicle action corresponding to the benefit value should meet the principle that the vehicle can smoothly pass the traffic light intersection and meet the principle of low energy consumption of the vehicle.
[0087] Step e: inputting the data set into a learning model to train the learning model to record the driving conditions of the vehicle passing through the signal light intersection under different driving conditions;
[0088] The data set is input into the learning model, and the learning model is trained to record the driving conditions of vehicles passing through traffic light intersections under different driving conditions. The driving conditions are affected by traffic congestion, the current speed of the vehicle and the status of the traffic lights, and the speed of the vehicle under different conditions to achieve smooth driving is recorded.
[0089] Among them, the main influencing factors are the current speed of the vehicle, the remaining time of the green light of the traffic light, and the distance between the vehicle and the traffic light.
[0090] The current vehicle state can be determined by these three parameters, so the state set S used by the learning model is the current vehicle speed v, the remaining time of the traffic light t, and the distance s between the vehicle and the traffic light:
[0091] S={v,t,s}
[0092] In addition, to ensure the smooth driving of the vehicle, it is necessary to limit the range of changes in the acceleration and deceleration of the vehicle:
[0093] A={a}
[0094] -1.2m / s 2 ≤a≤1.2m / s 2
[0095] Among them, a is the acceleration. When a takes a negative value, it is the deceleration caused by vehicle braking.
[0096] Step f: Based on the reward index, determining a reward function that returns a reward value when the vehicle passes through the signal light intersection;
[0097] Based on the reward index, a reward function that returns a reward value when the vehicle passes through a traffic light intersection is determined. Through the determination of the reward function, the learning model will tend to select a benefit value that satisfies the vehicle's passing through a traffic light intersection when analyzing different driving conditions based on the reward function. The vehicle speed and vehicle action corresponding to this benefit value will smoothly pass through the traffic light intersection.
[0098] In addition, the state of the vehicle is the current speed of the vehicle, the distance from the traffic light, and the remaining time of the traffic light (green light end time). The action that the vehicle can take is acceleration or deceleration. The reward function is the fuel economy that the vehicle can obtain and the passability of the traffic light. Then, the problem of vehicles passing through traffic light intersections can be abstracted into a Markov model. The model is to solve how to make an intelligent agent pass a predetermined distance within a specified time range. Therefore, the speed planning of the vehicle is mapped to the solution process of the Markov decision model.
[0099] Therefore, the essence of solving a Q-learning problem is to find an optimal strategy (which satisfies the vehicle's energy consumption economy and the vehicle's smooth driving through traffic light intersections) so that the cumulative reward obtained by the intelligent agent in the process of reaching the final state is maximized.
[0100] Among them, the iterative update formula of the reward function is as follows:
[0101] q(S,a)←q(S,a)+α(R(S'|S,a)+γmax a' q(S',a')-q(S,a))
[0102] Where:
[0103] α — learning rate;
[0104] max a' q(S',a')—Select the largest q(S',a') from all the actions a' that the successor state S' can choose.
[0105] γ—discount factor, which expresses the degree to which the agent considers the future state. The closer to 1, the more far-sighted the agent is, and the closer to 0, the more current the agent is. In this paper, the closer to 1, the main goal is passability (the vehicle drives smoothly through the traffic light intersection), and the closer to 0, the main goal is economy (energy consumption economy).
[0106] R(S'|S,a) — describes the reward obtained by the agent when taking action a to switch from state S to state S';
[0107] The above is a conventional Q-learning algorithm iteration method. Since the iteration distance in this embodiment is too long and there is only a large reward value when passing the traffic light, the above method requires multiple iteration cycles to generalize the final result value to the initial value, which slows down the convergence speed of the Q table and prolongs the iteration time.
[0108] Therefore, the learning rate and discount factor need to be adjusted to adapt to the complex scenario of passing through multiple signal light intersections proposed in this embodiment. In order to increase the convergence speed, the learning rate should be set larger, but in order to ensure the accuracy of the results, the learning rate should be gradually reduced as the number of iterations increases. However, the learning rate cannot be set too small, otherwise the result will be inaccurate. As the vehicle moves forward, the discount factor should be gradually increased. The updated iteration formula is as follows:
[0109]
[0110] Where:
[0111] i—the i-th cycle;
[0112] T—number of iterations;
[0113] num(S)—The serial number of state S in the state set. For example, the serial number of the initial state in the state set is 1.
[0114] Step g: Based on the reward function and the trained learning model, an analysis model for analyzing different driving conditions is generated.
[0115] According to the reward function and the trained learning model, an analysis model for analyzing different driving conditions is generated. By using the analysis model to analyze the driving conditions, the benefit value of the vehicle adjusting the speed to be adjusted can be obtained.
[0116] Step S140: Based on the speed change, the speed of the vehicle is adjusted to control the vehicle to maintain a stable driving state.
[0117] According to the speed change obtained after analysis by the analysis model, the speed of the vehicle is adjusted to reduce or increase the current speed of the vehicle, so that the vehicle can decelerate smoothly and brake and wait at the traffic light intersection, or the vehicle can increase its speed and drive smoothly and continuously through multiple traffic light intersections.
[0118] Compared with the prior art, in which an autonomous driving vehicle may suddenly accelerate or brake according to the external environment when traveling on the road, thereby causing the passengers inside the vehicle to move relative to the vehicle due to inertia, resulting in low comfort for the passengers inside the vehicle when riding in the vehicle, in the present application, by obtaining traffic information on the vehicle's expected driving route and the vehicle's current speed, and determining the driving condition of the vehicle when it meets the requirements for stable driving at a traffic light intersection based on the traffic information and the current speed, the driving condition is input into the analysis model, the driving condition is analyzed by the model, and the driving condition is determined and output in combination with the vehicle's current speed. The speed change required when the vehicle enters the driving condition is adjusted according to the speed change, so that the vehicle maintains smooth driving at the traffic light intersection, that is, a corresponding analysis is performed on the conditions of the traffic light intersection that the vehicle will encounter on the expected driving route, the driving condition for the vehicle to drive smoothly at the traffic light intersection is determined, and the adjustment amount of the vehicle speed is determined according to the driving condition, so as to ensure the smooth driving of the vehicle without sudden acceleration or deceleration, thereby ensuring that the passengers inside the vehicle will not have excessive relative movement with the vehicle due to inertia, thereby improving the riding comfort of the passengers inside the vehicle.
[0119] For example, refer to Figure 2 , Figure 21 is a flow chart of a second embodiment of a vehicle speed planning method under a dual signal light state based on reinforcement learning of the present application. Based on the first embodiment of the vehicle speed planning method under a dual signal light state based on reinforcement learning of the present application, a second embodiment is proposed, and the method further includes:
[0120] Step S210: Calculating the time required for the vehicle to pass through the signal light intersection based on the distance and the current vehicle speed;
[0121] Based on the distance and current vehicle speed, the time required for the vehicle to pass through the traffic light intersection can be calculated.
[0122] This time has a variation range. Factors that affect the time it takes for a vehicle to pass through a traffic light intersection include the distance between the vehicle and the traffic light, and the current vehicle speed. When the current vehicle speed increases, the time will decrease accordingly, and when the current vehicle speed decreases, the time will increase accordingly. In addition, when the current vehicle speed is constant, the longer the distance, the longer the time.
[0123] At the same time, distance affects the acceleration or deceleration effect of the vehicle. For example, when traveling with the same acceleration or deceleration, the speed changes of the vehicle at long distances and short distances are different. The longer the distance, the higher the speed of the vehicle after the acceleration. Conversely, the shorter the distance, the lower the speed of the vehicle after the acceleration.
[0124] In addition, the speed of both vehicles should be controlled within the speed limit of the road. For example, the speed limit on expressways is generally 110-120km / h; the speed limit on first-class highways, such as national highways and provincial highways, is 80km / h; the speed limit on suburban roads is 70-80km / h; the speed limit on urban roads is 40-60km / h, etc.
[0125] Exemplarily, the calculating, based on the distance and the current vehicle speed, the time required for the vehicle to pass through the signal light intersection to be passed includes:
[0126] Step h: based on the distance and the current vehicle speed, determining a speed range in which the current vehicle speed changes when the vehicle smoothly drives through a traffic light intersection;
[0127] Based on the distance and current vehicle speed, as well as the acceleration and deceleration required for smooth driving of the vehicle (the acceleration and deceleration are limiting indicators, i.e., to ensure smooth driving of the vehicle and control the speed of vehicle speed adjustment, the acceleration and deceleration are limited, except for emergency situations, which include the sudden appearance of pedestrians in front of the vehicle, requiring emergency avoidance, emergency braking, or emergency acceleration to avoid external interference, etc., and its specific parameters have been explained in the above description), based on the above parameters, the speed of the vehicle when it arrives at the signal light intersection is determined, and the speed is a range value, for example, the vehicle passes the signal light intersection at a low speed, or the vehicle passes the signal light intersection at a high speed.
[0128] The range of acceleration and deceleration is certain. Therefore, the maximum acceleration will result in the maximum acceleration of the vehicle, and the maximum deceleration will result in the maximum deceleration of the vehicle. The vehicle is in a continuously changing state during driving, and the distance affects the time for the vehicle to accelerate or decelerate. For example, when the acceleration of the vehicles is the same, the final speed obtained by the vehicle accelerating on a 100-meter road must be less than the final speed obtained by the vehicle accelerating on a 200-meter road.
[0129] Therefore, when a vehicle drives smoothly through a traffic light intersection, different speeds will be generated according to different situations. At the same time, the speed change mainly depends on the distance between the vehicle and the traffic light. If the distance is long enough, the vehicle can be increased to the maximum speed. On the contrary, if the distance is short, the vehicle cannot be increased to the maximum speed. The maximum and minimum speeds of the vehicle are counted, and the speeds between the two are the speeds that the vehicle can reach, that is, the speed range in which the vehicle speed changes is obtained.
[0130] Step i: If the preset speed limit range covers the speed range, the time required for the vehicle to pass through the signalized intersection to be passed is calculated based on the speed range;
[0131] Step j: If the speed range is equal to the preset speed limit range, the time required for the vehicle to pass through the signalized intersection to be passed is calculated based on the preset speed limit range or the speed range.
[0132] The preset speed limit range is the speed limit standard of the road on which the vehicle is traveling. For example, the speed limit of the first-class highways mentioned above, such as national highways and provincial highways, is 80km / h. Therefore, even if the distance between the vehicle and the traffic light intersection satisfies the situation that the vehicle is always accelerating, it is restricted by the road speed limit standard. Even if the vehicle meets the conditions for accelerating to 100km / h over a long distance, the maximum speed of the vehicle is 80km / h, that is, the maximum speed and the minimum speed of the vehicle on the road are both the speed limit standards of the road, that is, the distance between the vehicle and the traffic light satisfies the situation that when the vehicle accelerates or decelerates, the maximum speed change generated by the vehicle is within the speed limit range.
[0133] Similarly, when the distance between the vehicle and the traffic light intersection does not allow the vehicle to accelerate / decelerate to the highest / lowest speed in the speed limit range, the speed range of the speed change generated by the vehicle must be smaller than the preset speed limit range.
[0134] That is, the preset speed limit range is equivalent to the maximum set of vehicle speed variation ranges, and the speed range is a set that produces variation fluctuations, which belongs to the set of preset speed limit ranges. Therefore, when calculating the time required for a vehicle to pass through a traffic light intersection to be passed, the minimum set of vehicle speeds is used as the speed reference standard, that is, the time required for the vehicle to smoothly pass through the traffic light intersection is calculated based on the speed range.
[0135] In addition, there is a situation where the preset speed limit range and the speed range are the same, that is, the vehicle can be reduced to the minimum speed and increased to the maximum speed. At this time, the preset speed limit range and the speed range can be used to calculate the time required for the vehicle to smoothly pass through the traffic light intersection. This time is a range value, corresponding to the speed range or the preset speed limit range.
[0136] Step S220: Based on the time and the signal switching information, determining whether the vehicle can pass through the first signal light intersection to be passed in a stable driving manner;
[0137] The signal switching information includes the time information when the signal light switches the signal color, and the duration of each color light state, etc. For example, when the signal light in the current straight direction switches to red, it is necessary to wait for the green light in other directions (other straight directions or diagonal directions) to end before the signal light in the current straight direction will switch to green. The time is fixed, and at the same time, the green light duration of the signal light is fixed.
[0138] According to the signal switching information and time, it can be determined that the vehicle can pass the first signal light intersection to be passed in a stable driving manner.
[0139] For example, the current traffic light at the intersection is green and there are 20 seconds left before the light switches to red. The shortest time for a vehicle to reach the intersection is 10 seconds, and the longest time is 15 seconds or 30 seconds, etc. Even if the longest time exceeds the remaining time of the green light, the vehicle can pass as long as the shortest time for the vehicle to reach the intersection is less than the remaining time of the green light.
[0140] Among them, the shortest time and the longest time required for the vehicle to travel to the intersection are both the time ranges calculated by the above-mentioned speed range or the preset speed limit range, and the time range is the range obtained when the vehicle is traveling steadily.
[0141] If the vehicle can pass through the first traffic light intersection to be passed, it can be further determined whether the vehicle can pass through the subsequent traffic light intersections to be passed at the same time. If the vehicle cannot pass through the first traffic light intersection to be passed, the vehicle must brake and stop at the current first traffic light intersection to be passed and wait for the traffic light to switch to the green light state.
[0142] Step S230: If the vehicle can pass through the first signal light intersection to be passed, then determine the phase difference of the signal change between the signal lights;
[0143] The phase difference of signal changes between the signal lights is the time difference when the signals are switched between the signal lights. For example, when the first signal light switches to the red light state, the second signal light will just switch to the green light state. For another example, when the first signal light switches to the green light state, the second signal light still remains in the red light state. That is, there is a switching time difference between the signal lights, which is the phase difference of the signal change.
[0144] The phase difference time affects whether a vehicle can pass through multiple traffic light intersections continuously.
[0145] Step S240: determining the number of signal light intersections that the vehicle is expected to pass based on the phase difference and the signal switching information;
[0146] The number of traffic light intersections that the vehicle is expected to pass is determined based on the phase difference and signal switching information, and this number affects the planning of the vehicle's driving conditions.
[0147] For the first traffic light, let the distance between the vehicle and the traffic light be s 0 , the passing time of the signal light is t start1 ~t end1 (refers to the duration of the green light, t start is the time from the start of the green light. If the current signal light is green, then t start is zero, t end The current urban road speed range (preset speed limit range) is v min ~v max Therefore, whether a vehicle can pass through a traffic light can be simply divided into two situations, as shown in formula (1). When formula (1) is established, the vehicle can pass.
[0148]
[0149] For the sake of convenience, assume that the two signal lights are of the same type and the duration of each phase is the same. Then the two signal lights only differ in phase difference, and the phase difference is t offset , assuming that the total time it takes for the vehicle to pass the first traffic light is t pass , then the passing time for the second signal light is t start1 +t offset -t pass ~t end1 +t offset -t pass , set it to t start2 ~tend2 , the distance between the two traffic lights is s 1 , then, the result of whether the second traffic light can be passed is as shown in formula (2). When formula (2) is established, the vehicle can pass.
[0150]
[0151] Since s in formula (2) 1 、v max 、v min The parameters are known, and t start2 ,t end2 t in start1 ,t end1 ,t offset The parameters such as t are also known, so whether the second signal light can pass directly depends on t pass , which is the passing time of the first signal light. Therefore, in order for the second signal light to pass, the passing time range of the first signal light is as shown in formula (3):
[0152]
[0153] Therefore, just Set to the updated t start1 ,Will Set to the updated t end1 , and then put it into formula (2) for calculation. When the formula is established, it means that both traffic lights can be passed. As shown in formula (4):
[0154]
[0155] When the first signal light cannot be passed, let the duration of the green light be t green The second signal light passes at a time t offset ~t green +t offset At this time, whether the second traffic light is passed or not has nothing to do with the first traffic light, but only with the initial conditions. Therefore, when the first traffic light cannot be passed, the result of whether the second traffic light can be passed is shown in formula (5). When formula (5) is established, the vehicle can pass.
[0156]
[0157] Table 1 Working condition analysis under dual signal light scenario
[0158] By situation Establishment conditions Both signal lights pass Formula 4 is established Only pass the first signal Formula 1 is valid, but Formula 4 is not valid. Only pass the second signal Formula 1 is not true, but Formula 5 is true No pass at both signal lights Formula 1 is not true, Formula 5 is not true
[0159] Step S250: Determine the corresponding driving condition based on the quantity.
[0160] When passing through intersections with different numbers of traffic lights, the required speed of the vehicle is different. For example, the vehicle passes through the first traffic light intersection at a low speed, but due to the low speed, it will miss the second traffic light intersection. If the vehicle needs to pass through two traffic light intersections at the same time, the vehicle needs to be controlled to increase a certain speed before it can pass through the two traffic light intersections continuously.
[0161] Exemplarily, when obtaining relevant information of traffic lights, information of at least two traffic lights will be obtained, that is, based on the change relationship between the two traffic lights, the current color status of the traffic lights, and the distance between the traffic lights, the state in which the vehicle can pass through the traffic light intersection will be determined, including passing at a low speed, passing at a high speed, being unable to pass, or only being able to pass through one traffic light (the one traffic light includes the first traffic light that the vehicle passes, or the vehicle is waiting at the first traffic light intersection and passes through the second traffic light).
[0162] Among them, when the vehicle passes through at least two traffic light intersections at a low speed, it is determined based on the distance and the current vehicle speed and the status information of the traffic lights that when the vehicle runs smoothly at a low speed to the traffic light intersection, the traffic lights at the traffic light intersection are in the green light or are about to turn green, and when the vehicle drives at a low speed to at least two traffic light intersections, they are all in this state, that is, the vehicle can smoothly pass through the traffic light intersection at a low speed. If the vehicle accelerates, it will affect the efficiency of the vehicle passing through the intersection. When the vehicle accelerates to the traffic light intersection, it is necessary to brake at the traffic light intersection and wait for a while, causing the vehicle to start and stop, which is not conducive to stability control driving.
[0163] Among them, when a vehicle passes through an intersection with at least two traffic lights at a high speed, the effect is similar to that when the vehicle passes through at a low speed, that is, when the vehicle travels to the traffic light intersection at the current speed, it will miss the green light state, or the green light is about to end. If the vehicle forces its way through the traffic light intersection, it will cause a red light run. Therefore, the vehicle needs to rush to the traffic light intersection at a high speed in advance so that the vehicle can pass the green light state before the traffic light intersection ends.
[0164] When passing through the intersection, the second intersection is in the same state. Therefore, the vehicle needs to increase the speed to pass through the traffic light intersection, and cannot pass at the current speed or the reduced speed. If the vehicle passes at the current speed or the reduced speed, it will cause the vehicle to brake and wait at the traffic light intersection, affecting driving efficiency.
[0165] Among them, when the vehicle can only pass through one traffic light intersection regardless of whether it is accelerating or decelerating, due to the distance between the traffic light and the vehicle, and the acceleration and deceleration limits of the vehicle's speed, the vehicle will not be able to pass through the traffic light intersection when the light is green. For example, after the vehicle passes the first traffic light intersection, the distance to the second traffic light intersection is 500 meters, but the second traffic light intersection has entered the 10-second countdown for the green light state, and the current speed of the vehicle is 50km / h. In 10 seconds, the vehicle can travel a maximum of about 130 meters (which will be reduced accordingly due to road conditions and the influence of other vehicles). Therefore, the vehicle cannot pass the second intersection, and vice versa. When the vehicle arrives at the first intersection, it brakes and waits, and can pass the second intersection at a certain speed.
[0166] In this embodiment, the time required for the vehicle to reach the traffic light intersection is calculated based on the current vehicle speed, the signal switching information of the traffic light, and the distance between the vehicle and the traffic light. In combination with the signal switching time of the traffic light, it is determined whether the vehicle can pass through the first traffic light intersection to be passed. When the vehicle meets the requirements for smooth driving through the first traffic light intersection to be passed, the signal conditions of multiple traffic light intersections are further analyzed to determine whether the vehicle can continuously and smoothly pass through multiple traffic light intersections, thereby determining the corresponding driving conditions.
[0167] For example, refer to Figure 3 , Figure 3 : is a flow chart of a third embodiment of a vehicle speed planning method under a dual signal light state based on reinforcement learning of the present application. Based on the first embodiment and the second embodiment of the vehicle speed planning method under a dual signal light state based on reinforcement learning of the present application, a third embodiment is proposed, and the method further includes:
[0168] Step S310: If the vehicle cannot pass through the first signal light intersection to be passed, calculating the deceleration required for smooth braking of the vehicle;
[0169] When the vehicle cannot pass through the first traffic light intersection to be passed, the vehicle must brake and stop at the traffic light intersection. At the same time, considering the smooth driving state of the vehicle, the deceleration required for smooth braking of the vehicle is calculated, and this deceleration is the deceleration of the vehicle braking.
[0170] Step S320: Determine the corresponding braking condition based on the deceleration.
[0171] According to the deceleration requirement, the corresponding braking condition is determined, that is, braking and waiting at the first traffic light intersection to be passed to wait for the traffic light to switch to the green light state.
[0172] Exemplarily, determining the corresponding braking condition based on the deceleration includes:
[0173] Step k: if the deceleration is within a preset deceleration range, determining a corresponding braking condition based on the deceleration;
[0174] The preset deceleration range is a range value that limits the vehicle to avoid sudden braking. When the deceleration of the vehicle's smooth braking is within the preset deceleration range, the deceleration is used to formulate the corresponding braking condition to achieve a smooth deceleration effect.
[0175] When the deceleration exceeds the preset deceleration range, considering the stability of the vehicle and the actual ability to brake and stop at a traffic light intersection, it is necessary to consider formulating corresponding braking conditions with other parameters.
[0176] Step 1: If the deceleration exceeds the preset deceleration range, determining the maximum deceleration allowed by the preset deceleration range;
[0177] Step m: Based on the maximum deceleration, determine the corresponding braking condition.
[0178] When the deceleration exceeds the preset deceleration range, if this deceleration is adopted when the current vehicle brakes, the vehicle's instantaneous speed will drop sharply, and the passengers inside the vehicle will fall forward due to inertia, resulting in poor riding comfort for the passengers of the vehicle. Therefore, the maximum deceleration allowed by the preset deceleration range is selected at this time, and the corresponding braking condition is generated with this maximum deceleration.
[0179] In addition, the calculated deceleration is the deceleration required for the vehicle to brake and stop at the traffic light intersection. Therefore, if the vehicle is braked and decelerated only by the maximum deceleration within the preset deceleration range, the vehicle cannot be braked and stopped at the traffic light intersection. Therefore, when generating the corresponding braking conditions, the maximum deceleration is used as the basis. When the vehicle has a braking effect, the maximum deceleration is gradually increased, so that the vehicle can still maintain stable driving during braking, and will not cause problems such as sudden speed drop, vehicle shaking and drifting due to excessive deceleration.
[0180] In this embodiment, in view of the situation that the vehicle cannot pass through the first traffic light intersection to be passed, the deceleration required for the vehicle to brake at the traffic light intersection is calculated, and the relationship between the deceleration and the preset deceleration range is determined. When the deceleration meets the preset deceleration range, the vehicle is braked directly at the magnitude of the deceleration. When the deceleration exceeds the preset deceleration range, the vehicle is braked at the maximum deceleration value in the preset deceleration range. During the braking process, the maximum deceleration is gradually increased to ensure that the vehicle maintains a stable driving state during braking and deceleration.
[0181] In addition, the present application also provides a vehicle speed planning device under a dual signal light state based on reinforcement learning, and the vehicle speed planning device under a dual signal light state based on reinforcement learning includes:
[0182] An acquisition module, used to acquire traffic information on the vehicle's expected travel route and to acquire the vehicle's current speed;
[0183] A determination module, configured to determine, based on the traffic information and the current vehicle speed, whether the vehicle satisfies a driving condition for stable driving at a traffic light intersection;
[0184] An input module, used for inputting the driving condition into the analysis model, so that the analysis model can determine and output the speed change required when the vehicle enters the driving condition;
[0185] The adjustment module is used to adjust the speed of the vehicle based on the speed change to control the vehicle to maintain stable driving.
[0186] Exemplarily, the determining module includes:
[0187] A first determination submodule, configured to determine, based on the traffic information, status information of a preset number of traffic lights on the expected travel route, and determine traffic congestion conditions;
[0188] A second determination submodule, configured to determine the distance between each of the signal lights and the vehicle based on the state information, and determine the signal switching information of each of the signal lights;
[0189] The third determination submodule is used to determine that the vehicle meets the driving condition for smooth driving at a traffic light intersection based on the distance, the signal switching information and the current vehicle speed when the traffic congestion condition meets the normal passage of the vehicle.
[0190] Exemplarily, the third determining submodule includes:
[0191] A calculation unit, configured to calculate the time required for the vehicle to pass through the signal light intersection to be passed based on the distance and the current vehicle speed;
[0192] A first determining unit, configured to determine whether the vehicle can pass through a first signal light intersection to be passed in a stable driving manner based on the time and the signal switching information;
[0193] A first judgment unit, configured to determine a phase difference between signal changes of the signal lights if the vehicle can pass through the first signal light intersection to be passed;
[0194] A second determining unit, configured to determine the number of vehicles expected to pass through the signal light intersection based on the phase difference and the signal switching information;
[0195] a third determining unit, configured to determine a corresponding driving condition based on the quantity;
[0196] a second judgment unit, configured to calculate the deceleration required for smooth braking of the vehicle if the vehicle cannot pass through the first signal light intersection to be passed;
[0197] The fourth determining unit is used to determine a corresponding braking condition based on the deceleration.
[0198] Exemplarily, the fourth determining unit includes:
[0199] A first judgment subunit is used to determine a corresponding braking condition based on the deceleration if the deceleration is within a preset deceleration range;
[0200] a second judgment subunit, configured to determine a maximum deceleration allowed by the preset deceleration range if the deceleration exceeds the preset deceleration range;
[0201] A generating subunit is used to determine a corresponding braking condition based on the maximum deceleration.
[0202] Exemplarily, the computing unit includes:
[0203] A determination subunit, configured to determine, based on the distance and the current vehicle speed, a speed range that satisfies a change in the current vehicle speed when the vehicle smoothly drives through a signal light intersection;
[0204] A third judgment subunit is used to calculate the time required for the vehicle to pass through the traffic light intersection to be passed within the speed range if the preset speed limit range covers the speed range;
[0205] The fourth judgment subunit is used to calculate the time required for the vehicle to pass through the signal light intersection to be passed based on the preset speed limit range or the speed range if the speed range is equal to the preset speed limit range.
[0206] Exemplarily, the input module includes:
[0207] The acquisition submodule is used to obtain a data set consisting of multiple driving conditions and to obtain reward indicators;
[0208] An input submodule, used for inputting the data set into the learning model, so as to train the learning model to record the driving conditions of the vehicle passing through the signal light intersection under different driving conditions;
[0209] A fourth determination submodule, configured to determine, based on the reward index, a reward function that returns a reward value when the vehicle passes through the signal light intersection;
[0210] A generation submodule is used to generate an analysis model for analyzing different driving conditions based on the reward function and the trained learning model.
[0211] The specific implementation of the vehicle speed planning device under the dual signal light state based on reinforcement learning in the present application is basically the same as the various embodiments of the vehicle speed planning method under the dual signal light state based on reinforcement learning mentioned above, and will not be repeated here.
[0212] In addition, the present application also provides a vehicle speed planning device under dual signal light conditions based on reinforcement learning. Figure 4 As shown, Figure 4 It is a structural diagram of the hardware operating environment involved in the embodiment of the present application.
[0213] For example, Figure 4 This is a structural diagram of the hardware operating environment of the vehicle speed planning device under the dual signal light state based on reinforcement learning.
[0214] like Figure 4 As shown, the vehicle speed planning device under the dual signal light state based on reinforcement learning may include a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402 and the memory 403 communicate with each other through the communication bus 404, and the memory 403 is used to store computer programs; the processor 401 is used to implement the steps of the vehicle speed planning method under the dual signal light state based on reinforcement learning when executing the program stored in the memory 403.
[0215] The communication bus 404 mentioned in the vehicle speed planning device under the dual signal light state based on reinforcement learning can be a peripheral component interconnect standard (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus 404 can be divided into an address bus, a data bus, and a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0216] The communication interface 402 is used for communication between the vehicle speed planning device under the dual signal light state based on reinforcement learning and other devices.
[0217] The memory 403 may include a random access memory (RMD) or a non-volatile memory (NM), such as at least one disk memory. Optionally, the memory 403 may also be at least one storage device located away from the processor 401.
[0218] The above-mentioned processor 401 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0219] The specific implementation of the vehicle speed planning device under the dual signal light state based on reinforcement learning in the present application is basically the same as the various embodiments of the vehicle speed planning method under the dual signal light state based on reinforcement learning mentioned above, and will not be repeated here.
[0220] In addition, an embodiment of the present application also proposes a computer-readable storage medium, on which is stored a vehicle speed planning program under a dual signal light state based on reinforcement learning. When the vehicle speed planning program under a dual signal light state based on reinforcement learning is executed by a processor, the steps of the vehicle speed planning method under a dual signal light state based on reinforcement learning as described above are implemented.
[0221] The specific implementation of the computer-readable storage medium of the present application is basically the same as the various embodiments of the vehicle speed planning method under the dual signal light state based on reinforcement learning mentioned above, and will not be repeated here.
[0222] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0223] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0224] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0225] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A vehicle speed planning method under dual signal light conditions based on reinforcement learning, characterized in that: The vehicle speed planning method under the dual signal light state based on reinforcement learning comprises the following steps: Obtaining traffic information on the vehicle's expected travel route and obtaining the vehicle's current speed; Based on the traffic information and the current vehicle speed, determining that the vehicle meets a driving condition when driving smoothly at a traffic light intersection; Inputting the driving condition into the analysis model, so that the analysis model determines and outputs the speed change required when the vehicle enters the driving condition; Based on the speed change, adjusting the speed of the vehicle to control the vehicle to maintain stable driving; The step of determining, based on the traffic information and the current vehicle speed, whether the vehicle meets the driving condition for stable driving at a traffic light intersection comprises: Based on the traffic information, determining status information of a preset number of traffic lights on the expected travel route, and determining traffic congestion conditions; Based on the state information, determining the distance between each of the signal lights and the vehicle, and determining the signal switching information of each of the signal lights; When the traffic congestion condition satisfies the normal passage of the vehicle, determining, based on the distance, the signal switching information and the current vehicle speed, that the vehicle satisfies the driving condition for smooth driving at a signal light intersection; Wherein, the driving condition includes a driving condition, and the step of determining the driving condition when the vehicle is driving steadily at a signal light intersection based on the distance, the signal switching information and the current vehicle speed includes: Based on the distance and the current vehicle speed, calculating the time required for the vehicle to pass through the signal light intersection to be passed; Based on the time and the signal switching information, determining whether the vehicle can pass through a first signal light intersection to be passed in a stable driving manner; If the vehicle can pass through the first signal light intersection to be passed, determining the phase difference of the signal changes between the signal lights; Determining the number of signal light intersections that the vehicle is expected to travel through based on the phase difference and the signal switching information; Based on the quantity, determining a corresponding driving condition; Wherein, the driving condition includes a braking condition, and after the step of determining whether the vehicle can pass through the first signal light intersection to be passed in a stable driving manner based on the time and the signal switching information, the step further includes: If the vehicle cannot pass through the first signal light intersection to be passed, calculating the deceleration required for the vehicle to brake smoothly; Based on the deceleration, determining a corresponding braking condition; Wherein, before the step of inputting the driving condition into the analysis model, the method further includes: Obtain a data set consisting of multiple driving conditions and obtain reward indicators; Inputting the data set into a learning model to train the learning model to record the driving conditions of the vehicle passing through the signal light intersection under different driving conditions; Based on the reward index, determining a reward function that returns a reward value when the vehicle passes through the signalized intersection; Based on the reward function and the trained learning model, an analysis model for analyzing different driving conditions is generated.
2. The vehicle speed planning method under dual signal light state based on reinforcement learning as claimed in claim 1, characterized in that: The determining of the corresponding braking condition based on the deceleration includes: If the deceleration is within a preset deceleration range, determining a corresponding braking condition based on the deceleration; If the deceleration exceeds the preset deceleration range, determining the maximum deceleration allowed by the preset deceleration range; Based on the maximum deceleration, a corresponding braking condition is determined.
3. The vehicle speed planning method under dual signal light state based on reinforcement learning as claimed in claim 1, characterized in that: The calculating, based on the distance and the current vehicle speed, the time required for the vehicle to pass through the signal light intersection to be passed comprises: Based on the distance and the current vehicle speed, determining a speed range that satisfies the change in the current vehicle speed when the vehicle smoothly drives through a traffic light intersection; If the preset speed limit range covers the speed range, the time required for the vehicle to pass the signalized intersection to be passed is calculated based on the speed range; If the speed range is equal to the preset speed limit range, the time required for the vehicle to pass through the signal light intersection to be passed is calculated based on the preset speed limit range or the speed range.
4. A vehicle speed planning device under dual signal light state based on reinforcement learning, characterized in that: The vehicle speed planning device under the dual signal light state based on reinforcement learning includes: An acquisition module, used to acquire traffic information on the vehicle's expected travel route and to acquire the vehicle's current speed; A determination module, configured to determine, based on the traffic information and the current vehicle speed, whether the vehicle satisfies a driving condition for stable driving at a traffic light intersection; An input module, used for inputting the driving condition into the analysis model, so that the analysis model can determine and output the speed change required when the vehicle enters the driving condition; An adjustment module, used for adjusting the speed of the vehicle based on the speed change, so as to control the vehicle to maintain stable driving; Wherein, the determination module includes: A first determination submodule, configured to determine, based on the traffic information, status information of a preset number of traffic lights on the expected travel route, and determine traffic congestion conditions; A second determination submodule, configured to determine the distance between each of the signal lights and the vehicle based on the state information, and determine the signal switching information of each of the signal lights; A third determination submodule is used to determine, when the traffic congestion condition satisfies the normal passage of the vehicle, based on the distance, the signal switching information and the current vehicle speed, whether the vehicle satisfies the driving condition for stable driving at a signal light intersection; Wherein, the third determination submodule includes: A calculation unit, configured to calculate the time required for the vehicle to pass through the signal light intersection to be passed based on the distance and the current vehicle speed; A first determining unit, configured to determine whether the vehicle can pass through a first signal light intersection to be passed in a stable driving manner based on the time and the signal switching information; A first judgment unit, configured to determine a phase difference between signal changes of the signal lights if the vehicle can pass through the first signal light intersection to be passed; A second determining unit, configured to determine the number of vehicles expected to pass through the signal light intersection based on the phase difference and the signal switching information; a third determining unit, configured to determine a corresponding driving condition based on the quantity; a second judgment unit, configured to calculate the deceleration required for smooth braking of the vehicle if the vehicle cannot pass through the first signal light intersection to be passed; a fourth determining unit, configured to determine a corresponding braking condition based on the deceleration; Wherein, the input module includes: The acquisition submodule is used to obtain a data set consisting of multiple driving conditions and to obtain reward indicators; An input submodule, used for inputting the data set into the learning model, so as to train the learning model to record the driving conditions of the vehicle passing through the signal light intersection under different driving conditions; A fourth determination submodule, configured to determine, based on the reward index, a reward function that returns a reward value when the vehicle passes through the signal light intersection; A generation submodule is used to generate an analysis model for analyzing different driving conditions based on the reward function and the trained learning model.
5. A vehicle speed planning device under dual signal light state based on reinforcement learning, characterized in that: The device includes: a memory, a processor, and a vehicle speed planning program under a dual signal light state based on reinforcement learning, which is stored in the memory and can be run on the processor. The vehicle speed planning program under a dual signal light state based on reinforcement learning is configured to implement the steps of the vehicle speed planning method under a dual signal light state based on reinforcement learning as described in any one of claims 1 to 3.
6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a vehicle speed planning program under a dual signal light state based on reinforcement learning. When the vehicle speed planning program under a dual signal light state based on reinforcement learning is executed by a processor, the steps of the vehicle speed planning method under a dual signal light state based on reinforcement learning are implemented as described in any one of claims 1 to 3.
Citation Information
Patent Citations
Intersection auxiliary driving method, vehicle and computer readable storage medium
CN113830105A
Data-driven vehicle speed planning method used in signal lamp scene
CN114261392A
Vehicle control method and control device
US20200317224A1
