Vehicle control method and device

Abstract

The present invention discloses a vehicle control method and device, and relates to the field of intelligent driving technology. A specific implementation of the method includes: determining multiple lanes in an intersection control area, one or more conflict partitions corresponding to the multiple lanes, and multiple vehicles to be controlled; according to the number of multiple vehicles to be controlled and the identification of the target vehicle, using the Monte Carlo tree search algorithm, determine the passage order of the multiple vehicles to be controlled corresponding to the intersection control area; the target vehicle is a vehicle in the lane closest to the conflict partition, and / or a vehicle with the shortest arrival time for the same conflict partition; according to the passage order and the position of one or more conflict partitions in the intersection control area, determine the driving speed of the vehicle to be controlled; according to the passage order and the driving speed, control the vehicle to be controlled to pass through the intersection control area. This implementation reduces the calculation time of the search tree, improves the efficiency of determining the passage order, and thus improves the efficiency of vehicle passage in the intersection control area.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a vehicle control method and device. Background Art

[0002] The existing right-of-way allocation at urban intersections is to obtain the optimal traffic order by finding the optimal node on the search tree.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0004] The existing road right allocation method needs to exhaustively traverse all nodes in the search tree and calculate the objective function value corresponding to each node to find the optimal passage order corresponding to the optimal node. When the traffic volume at the intersection is large and there are multiple lanes, the calculation time will increase exponentially with the increase in the number of vehicles, resulting in the inability to traverse all nodes to find the optimal node within a limited time, thus failing to meet the real-time requirements. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a vehicle control method and device, which can use a target vehicle that meets the conditions as a heuristic strategy for a Monte Carlo tree search algorithm, determine the passage order of multiple vehicles to be controlled in an intersection control area according to the Monte Carlo tree search algorithm, and determine the driving speed of the vehicles to be controlled, so as to control the vehicles to be controlled to pass through the intersection control area; thus, combined with the Monte Carlo tree search algorithm and the heuristic strategy, the traversal direction of the search tree can be quickly and accurately guided, avoiding the situation where all nodes need to be exhaustively traversed to obtain the optimal passage order, thereby improving the efficiency of the tree search; it also avoids the problem of excessive calculation time due to the increase in vehicles and lanes, meets the real-time requirements of vehicle passage, and thus improves the efficiency of vehicle passage in the intersection control area.

[0006] Furthermore, the travel speed of the vehicle to be controlled is determined according to the constructed time fitting model and speed fitting model, and the vehicle to be controlled is controlled according to the travel speed, thereby achieving reasonable control of the vehicle and improving the efficiency of vehicle traffic in the intersection control area.

[0007] To achieve the above object, according to a first aspect of an embodiment of the present invention, a vehicle control method is provided, comprising:

[0008] Determine a plurality of lanes within a control area of ​​an intersection, one or more conflicting zones corresponding to the plurality of lanes, and a plurality of vehicles to be controlled;

[0009] According to the number of the plurality of vehicles to be controlled and the identification of the target vehicle, a Monte Carlo tree search algorithm is used to determine the passage order of the plurality of vehicles to be controlled corresponding to the intersection control area; the target vehicle is a vehicle in a lane closest to the conflict partition and / or a vehicle with the shortest arrival time for the same conflict partition;

[0010] Determine the travel speed of the vehicle to be controlled according to the passage order and the position of the one or more conflicting partitions in the intersection control area;

[0011] According to the traffic order and the travel speed, the vehicle to be controlled is controlled to pass through the intersection control area.

[0012] Optionally, the using of a Monte Carlo tree search algorithm to determine the passage order of the plurality of vehicles to be controlled corresponding to the intersection control zone comprises:

[0013] According to the identifications of the multiple vehicles to be controlled and the number of vehicles, a root node and first-level child nodes of a search tree are constructed; one first-level child node corresponds to one vehicle identification;

[0014] According to the identification of the target vehicle, the root node of the search tree is expanded using the Monte Carlo tree search algorithm to generate one or more leaf nodes; the leaf nodes indicate the passage order corresponding to the multiple vehicles to be controlled;

[0015] According to the passage sequence indicated by the leaf node, the passage sequence of the multiple vehicles to be controlled corresponding to the intersection control area is determined.

[0016] Optionally, the step of expanding the root node of the search tree to generate one or more leaf nodes includes:

[0017] The following steps are executed repeatedly until the preset time is reached:

[0018] According to the UCB algorithm of Monte Carlo tree search, the first-level child node with the largest UCB value is determined as the current child node;

[0019] The following steps are executed repeatedly until the lower-level node is determined to be a leaf node:

[0020] The Monte Carlo tree search algorithm is used to determine the target node as a subordinate node of the current child node according to the UCB value and / or the identifier of the target vehicle, and the subordinate node is used as the current child node.

[0021] Optionally, determining the target node as a subordinate node of the current child node according to the UCB value and / or the identifier of the target vehicle includes:

[0022] Determine whether there is a target node in the first-level child node corresponding to the target vehicle, and the target node is not the parent node of the current child node;

[0023] If yes, determine the subordinate node of the current child node from the target node.

[0024] Optionally, in the case where there are multiple target nodes,

[0025] According to the UCB values ​​of the multiple target nodes, a target node with the largest UCB value is determined as a subordinate node of the current child node.

[0026] Optionally, after determining that the lower-level node is a leaf node, the method further includes:

[0027] According to the objective function value corresponding to the leaf node, the statistical information corresponding to multiple parent nodes of the leaf node is reversely updated using the back propagation algorithm of the Monte Carlo tree search, and the statistical information includes the accumulated score and the number of visits.

[0028] Optionally, the determining, according to the UCB algorithm of the Monte Carlo tree search, the first-level child node with the largest UCB value as the current child node includes:

[0029] For each first-level child node: determining the UCB value of the first-level child node according to the statistical information of the first-level child node, the statistical information of the root node, and a weight parameter;

[0030] The first-level child node with the largest UCB value is used as the current child node.

[0031] Optionally, the method further comprises:

[0032] The vehicle that is in the same lane and closest to the conflicting partition is taken as the target vehicle;

[0033] and / or,

[0034] From a plurality of vehicles to be controlled corresponding to the same conflict partition, the corresponding vehicle with the shortest fastest arrival time is used as the target vehicle; the fastest arrival time indicates the time when the vehicle arrives at the conflict partition at the maximum speed and acceleration.

[0035] Optionally, determining the passage order of the plurality of vehicles to be controlled corresponding to the intersection control zone includes:

[0036] Determine an optimal traversal path based on statistical information of each node of the search tree;

[0037] The target leaf node corresponding to the optimal traversal path is determined, and the passage order corresponding to the target leaf node is used as the passage order of the intersection control area.

[0038] Optionally, the determining the driving speed of the vehicle to be controlled includes:

[0039] For multiple vehicles to be controlled corresponding to the same conflict partition: determining the estimated arrival time of the multiple vehicles to be controlled at the conflict partition according to the position of the conflict partition in the intersection control area and the traffic order;

[0040] The driving speeds of the multiple vehicles to be controlled are determined according to the estimated arrival time, the vehicle state function and the double integral dynamics model.

[0041] Optionally, determining the estimated arrival time of the plurality of vehicles to be controlled at the conflicting zone includes:

[0042] Constructing a time fitting model and a plurality of first constraints; the first constraints include a fastest time constraint, a collision constraint and a conflict interval constraint;

[0043] According to the first constraint condition, the time fitting model is used to fit the estimated arrival times of the multiple vehicles to be controlled arriving at the conflicting partitions.

[0044] Optionally, determining the driving speeds of the plurality of vehicles to be controlled according to the estimated arrival time, the vehicle state function and the double integral dynamics model includes:

[0045] Construct velocity fitting model;

[0046] For each of the vehicles to be controlled: determining the state parameters of the vehicle to be controlled; the state parameters include: vehicle driving time, vehicle driving position, vehicle maximum speed, vehicle minimum speed, vehicle constant speed, vehicle maximum acceleration and vehicle minimum acceleration;

[0047] determining the vehicle state function according to the state parameter;

[0048] The driving speed of the vehicle to be controlled is fitted according to the speed fitting model, the estimated arrival time, the vehicle state function, and the double-integral dynamics model.

[0049] According to a second aspect of an embodiment of the present invention, a vehicle control device is provided, comprising: a determination module, a search module and a control module; wherein:

[0050] The determination module is used to determine a plurality of lanes in the intersection control area, one or more conflicting zones corresponding to the plurality of lanes, and a plurality of vehicles to be controlled;

[0051] The search module is used to determine the passage order of the multiple vehicles to be controlled corresponding to the intersection control area by using a Monte Carlo tree search algorithm according to the number of the multiple vehicles to be controlled and the identification of the target vehicle; the target vehicle is a vehicle in a lane closest to the conflict partition and / or a vehicle with the shortest arrival time for the same conflict partition;

[0052] The control module is used to determine the driving speed of the vehicle to be controlled according to the traffic order and the position of the one or more conflicting partitions in the intersection control area; and control the vehicle to be controlled to pass through the intersection control area according to the traffic order and the driving speed.

[0053] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, including:

[0054] one or more processors;

[0055] a storage device for storing one or more programs,

[0056] When the one or more programs are executed by the one or more processors, the one or more processors implement any method described in the vehicle control method provided in the first aspect above.

[0057] According to a fourth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, any method described in a vehicle control method provided in the first aspect is implemented.

[0058] An embodiment of the above invention has the following advantages or beneficial effects: it can use the target vehicle that meets the conditions as the heuristic strategy of the Monte Carlo tree search algorithm, determine the passage order of multiple vehicles to be controlled in the intersection control area according to the Monte Carlo tree search algorithm, and determine the driving speed of the vehicles to be controlled, so as to control the vehicles to be controlled to pass through the intersection control area; thus, combined with the Monte Carlo tree search algorithm and the heuristic strategy, the traversal direction of the search tree can be guided quickly and accurately, avoiding the situation where all nodes need to be traversed exhaustively to obtain the optimal passage order, thereby improving the efficiency of the tree search; it also avoids the problem of long calculation time caused by the increase in vehicles and lanes, meets the real-time requirements of vehicle passage, and thus improves the efficiency of vehicle passage in the intersection control area.

[0059] Furthermore, the travel speed of the vehicle to be controlled is determined according to the constructed time fitting model and speed fitting model, and the vehicle to be controlled is controlled according to the travel speed, thereby achieving reasonable control of the vehicle and improving the efficiency of vehicle traffic in the intersection control area.

[0060] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0062] Figure 1 is a flow chart of a vehicle control method provided by an embodiment of the present invention;

[0063] Figure 2 is a schematic diagram of a multi-lane intersection control area provided by an embodiment of the present invention;

[0064] Figure 3 is a schematic diagram of a control area for a single lane intersection provided by an embodiment of the present invention;

[0065] Figure 4 is a schematic diagram of a tree representation of a pass order provided by an embodiment of the present invention;

[0066] Figure 5 is a schematic diagram of an iterative process of a Monte Carlo tree search provided by an embodiment of the present invention;

[0067] Figure 6 is a flow chart of a vehicle control method provided by another embodiment of the present invention;

[0068] Figure 7 is a structural schematic diagram of a vehicle control device provided by an embodiment of the present invention;

[0069] Figure 8 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0070] Fig. 9 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0071] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0072] It should be pointed out that the embodiments of the present invention and the technical features therein may be combined with each other without conflict.

[0073] like Figure 1 As shown, an embodiment of the present invention provides a vehicle control method, which may include the following steps S101 to S104:

[0074] Step S101: Determine a plurality of lanes in a road intersection control area, one or more conflicting zones corresponding to the plurality of lanes, and a plurality of vehicles to be controlled.

[0075] Step S102: According to the number of the multiple vehicles to be controlled and the identification of the target vehicle, the Monte Carlo tree search algorithm is used to determine the passage order of the multiple vehicles to be controlled corresponding to the intersection control area; the target vehicle is the vehicle in the lane closest to the conflict partition, and / or the vehicle with the shortest arrival time for the same conflict partition.

[0076] Step S103: Determine the travel speed of the vehicle to be controlled according to the passage order and the position of the one or more conflicting partitions in the intersection control area.

[0077] Step S104: According to the traffic sequence and the travel speed, the vehicle to be controlled is controlled to pass through the intersection control area.

[0078] One application scenario of the present invention is the scenario of multiple autonomous driving vehicles cooperating at urban intersections. In a complex urban intersection with multiple lanes, multiple autonomous driving vehicles can use wireless communication and sensing technology to interact with surrounding vehicles and roadside equipment, share vehicle status information (such as vehicle location, vehicle speed, etc.), and inform the system of the vehicle's planned route; through the intelligent vehicle-road cooperative system, the passage order of multiple autonomous driving vehicles at the urban intersection is calculated to obtain the optimal passage order; then, according to the optimal passage order, the multiple autonomous driving vehicles are controlled to pass through the urban intersection by controlling the vehicle's driving speed.

[0079] Among them, a schematic diagram of a multi-lane urban intersection can be as follows Figure 2 As shown, the intersection control area includes multiple lanes, and multiple vehicles traveling in the intersection control area are vehicles to be controlled. Vehicles in different directions may collide when passing through multiple intersection areas formed by multiple lanes in the figure, and these multiple intersection areas are also multiple conflict partitions. Figure 2 There are four directions and three lanes in each direction. The lanes can be marked as lane 1, lane 2, lane 3... to lane 12. Therefore, the 36 conflict zones formed by these 12 lanes can be divided into conflict zone 1, conflict zone 2... conflict zone 36 according to their positions in the intersection control area.

[0080] In addition, the number of vehicles to be controlled in the intersection control area can be determined in real time through sensing technology, and these vehicles to be controlled can be numbered to obtain the vehicle identification of the vehicle in the system. Of course, the key information of the vehicle (such as the license plate number, etc.) obtained by wireless communication technology can also be used as the identification of the vehicle.

[0081] Since the traffic efficiency of multiple vehicles in the control area of ​​a city intersection mainly depends on the traffic order of multiple vehicles to be controlled, multiple traffic orders can constitute a solution space of a search tree. Each node of the search tree represents a feasible traffic order. Among them, the traffic order represents the priority between vehicles passing through the same conflict partition. For example, if two vehicles pass through a conflict partition at the same time, the vehicle with the first traffic order can pass first. Figure 3 Taking the single intersection scenario shown as an example, the control area includes lanes 1, 2, 3, and 4, and contains five vehicles with vehicle identifications of A, B, C, D, and E. Among them, the traffic order BD means that when passing through conflict partition 2, vehicle D needs to give way to the vehicle and adjust its own arrival time in the conflict partition according to the time when vehicle B arrives at conflict partition 2. However, when the traffic order is BC, since there is no conflict between vehicle C and vehicle B, vehicle C can still enter conflict partition 1 with the fastest arrival time. In other words, the objective function values ​​corresponding to the traffic orders BC and CB are the same. Each vehicle will try its best to determine the fastest time to arrive at the conflict partition according to the traffic order. Therefore, two non-conflicting vehicles can pass through the intersection at the same time, and a vehicle at the back of the traffic order may pass through the intersection before another vehicle that does not conflict with it. In addition, if a traffic order does not include all vehicles in the intersection control area, this type of order is called a partial traffic order. For example, in Figure 3 There are a total of 5 vehicles in the traffic order, and the traffic order BC belongs to a partial traffic order.

[0082] by Figure 3 Taking the traffic scene in the middle as an example, the tree expression of the traffic order can be as follows Figure 4 As shown. First, set the root node Root to empty, then each first-level child node of the root node will consist of one character (A, B, C, D), representing the first car in the passage sequence. Then, based on this, the third-level node will be a string consisting of two characters (such as CA, CB, CD, CE). Similarly, each child node expands its own subordinate child nodes (such as CBA, CBD, CBE, CBDA, CBDE, etc.) until it reaches the deepest depth of the search tree, that is, reaches the bottom leaf node (such as CBDAE, CBDEA, etc.). The order of the string corresponding to a leaf node (such as CBDAE) also corresponds to a complete passage sequence. Therefore, all leaf nodes of the search tree represent all feasible passage sequences.

[0083] It is understandable that the number of nodes in such a search tree will grow exponentially as the number of vehicles increases. In order to avoid relying entirely on exhaustive traversal to traverse the search tree to find the optimal passage order and improve traversal efficiency, the following method provided by an embodiment of the present invention can be used to expand the search tree: construct a root node and a first-level child node of the search tree according to the identification of the multiple vehicles to be controlled and the number of vehicles; one of the first-level child nodes corresponds to a vehicle identification; according to the identification of the target vehicle, the root node of the search tree is expanded using the Monte Carlo tree search algorithm to generate one or more leaf nodes; the leaf node indicates the passage order corresponding to the multiple vehicles to be controlled; according to the passage order indicated by the leaf node, determine the passage order of the multiple vehicles to be controlled corresponding to the intersection control area.

[0084] The process of the Monte Carlo tree search algorithm can be described as follows: Figure 5 As shown. Each iteration process may include four parts: selection, expansion, simulation and back propagation. The specific implementation method is explained in detail in the subsequent embodiments.

[0085] It can be understood that in order to improve the efficiency of the Monte Carlo tree search algorithm, the following method provided by an embodiment of the present invention can be adopted as a heuristic strategy for the Monte Carlo tree search algorithm: the vehicle that is in the same lane and closest to the conflict partition is taken as the target vehicle; and / or, from multiple vehicles to be controlled corresponding to the same conflict partition, the corresponding vehicle with the shortest fastest arrival time is taken as the target vehicle; the fastest arrival time indicates the time it takes for the vehicle to arrive at the conflict partition at the maximum speed and acceleration.

[0086] The target vehicle can be a vehicle in the same lane closest to the conflict partition, that is, a vehicle in the same lane and closest to the conflict partition. By using this heuristic strategy, unreasonable sequences can be quickly pruned. For example, if the vehicles in the same lane pass through the conflict partition in sequence, Figure 3 Some unreasonable passing orders, such as (EC, ED, EB, EA, etc.) will be pruned, thus Figure 4 There will be no situation where E is a first-level child node.

[0087] In addition, the target vehicle may also be: a vehicle in a plurality of lanes closest to the conflicting partition, or a vehicle with the shortest arrival time among a plurality of vehicles to be controlled corresponding to the same conflicting partition. In one embodiment of the present invention, the heuristic strategy may be Figure 5The simulation strategy in the simulation process can more accurately find the target vehicle and add it to the traffic order, which can avoid random sampling to generate a large number of different types of leaf nodes, thereby resulting in too many traffic orders. It improves the efficiency of Monte Carlo tree search and thus improves the real-time performance of traffic in the intersection control area.

[0088] The specific process of expanding the root node of the search tree and generating one or more leaf nodes can be as follows: looping the following steps until a preset time is reached: according to the UCB algorithm of the Monte Carlo tree search, determining that the first-level child node with the largest UCB value is the current child node; looping the following steps until the determined lower-level node is a leaf node: using the Monte Carlo tree search algorithm, according to the UCB value and / or the identification of the target vehicle, determining the target node as the lower-level node of the current child node, and using the lower-level node as the current child node.

[0089] It is understood that, in one embodiment of the present invention, the passage order represented by the lower node is a passage order string formed by appending a vehicle identification represented by a target node to the end of the passage order indicated by the current child node. For example, the lower node string of C is CB, the lower node string of CB is CBD, and so on.

[0090] In each iteration of the Monte Carlo tree search, the root node is used as the starting point, and the search continues to expand downward until the bottom of the search tree is reached, that is, the leaf node is generated. When determining the subordinate node of the root node, it is also determined which first-level child node of the root node to start expanding downward. At this time, the first-level child node with the largest UCB value can be used as the subordinate node of the root node.

[0091] When Monte Carlo Tree Search constructs a search tree, each node has its own statistical information, namely the cumulative score and the number of visits. The cumulative score refers to the cumulative score of the current node after multiple visits, and the number of visits refers to the number of times the node is selected as the current child node.

[0092] Specifically, when determining the first-level child node with the largest UCB value in the search tree, the following method provided by an embodiment of the present invention can be adopted: for each first-level child node: determine the UCB value of the first-level child node based on the statistical information of the first-level child node, the statistical information of the root node and the weight parameter; and use the first-level child node with the largest UCB value as the current child node.

[0093] The UCB value is calculated using the following UCB formula:

[0094]

[0095] Among them, i is a child node, Q iis the cumulative score in the statistical information of the child node, n i is the number of visits to the child node, n is the cumulative score of the parent node of the child node, and C is the weight parameter, which can be set to a constant, such as 2, or

[0096] by Figure 3 Take the intersection traffic scenario as an example. When starting from the root node, the root node is Root, and the first-level child nodes generated by it can be as follows Figure 4 As shown, they are A, B, C, and D. Among them, E cannot be used as a first-level child node due to the principle of the order of the same lane. At this time, the UCB values ​​of the four first-level child nodes A, B, C, and D can be calculated. The node with the largest UCB value is used as the next lower-level node to be expanded from the root node. The process can be regarded as Figure 5 For example, in this round of selection, if C has the largest UCB value, then C will be selected as the first-level child node of the root node, thus entering Figure 5 in the expansion phase.

[0097] For new nodes that have never been visited (i.e., nodes that are not currently in the search tree but may be added to the search tree), their visit count is 0 and their cumulative score is also 0. As the number of iterations increases, after multiple leaf nodes are generated, the Monte Carlo tree search backpropagation algorithm can be used to update the statistical information of most nodes, and then the above UCB formula can be used to calculate the UCB value of the node itself.

[0098] It is understandable that each traversal of the search tree starts from the root node and goes downwards in sequence. In the first iteration for the root node, the selected first-level child nodes A, B, C, and D have not been visited, so their corresponding access times are all 0, and the access times of their root nodes are 0. According to the UCB formula, the UCB values ​​of the current child nodes A, B, C, and D are all ∞ (infinity). At this time, a vehicle identification can be randomly selected as a lower-level node, for example, C is selected as the lower-level node to be expanded, and then C is further expanded.

[0099] When expanding the first-level child node C, the following method provided by an embodiment of the present invention may be adopted: determine whether there is a target node in the first-level child node corresponding to the target vehicle, and the target node is not the upper node of the current child node; if so, determine the lower node of the current child node from the target node. In the case of multiple target nodes, according to the UCB values ​​of the multiple target nodes, determine the target node with the largest UCB value as the lower node of the current child node.

[0100] The target node is determined from the first-level child nodes. The pass order corresponding to the target node is to append a first-level child node that is not included in the pass order string s to the end of the current child node, thereby forming a lower-level node, such as Figure 4 As shown. If the subordinate node is a leaf node, then the vehicle identification not included in the string s may be one or more, that is, the subordinate nodes of the non-leaf nodes represent part of the pass order. It is understandable that these newly constructed subordinate nodes may be nodes that have not been traversed in the search tree. For example, when selecting a subordinate node for CB, CB has formed a subordinate node CBA, then CBA can be selected to continue traversing downward, or it can be a subordinate node formed based on other passable orders CBD and CBE.

[0101] During the expansion process, the target vehicle (i.e., the heuristic strategy) can be used to continuously perform simulations until a leaf node is generated. Figure 3 Take the traffic scenario of Figure 5 The expansion process shown indicates that C is expanded. Among them, C is the current child node, and the target node is selected according to the target vehicle of the heuristic strategy among the four vehicles A, B, D, and E (first-level child nodes), such as Figure 5 In the example, B is the vehicle closest to the conflicting partition, so the target node can be B, and the corresponding partial traffic order CB after selecting B is selected as the subordinate node of C. If it is found that there are multiple target nodes at this time, such as B and A both meet the conditions of the heuristic strategy, then the UCB values ​​of CB and CA can be calculated, and the one with the largest UCB value can be used as the subordinate node. If CB and CA are both new nodes that may be added, since they have never been visited, their UCB values ​​are both ∞ (infinity), and you can randomly select one at this time. In addition, if the UCB values ​​are equal, you can randomly select one.

[0102] Therefore, after the simulation process of the Monte Carlo tree search is completed, that is, after the lower-level node is determined to be a leaf node, the statistical information of multiple nodes can be updated in the following manner provided by an embodiment of the present invention: According to the objective function value corresponding to the leaf node, the back propagation algorithm of the Monte Carlo tree search is used to reversely update the statistical information corresponding to multiple upper-level nodes of the leaf node, and the statistical information includes the cumulative score and the number of visits.

[0103] The objective function value corresponding to the leaf node can be generated by an objective function, and the objective function is not limited here. The objective function value is used to evaluate the potential of the parent node corresponding to the leaf node to become the optimal node. Figure 5 The back propagation in the above is to use the back propagation algorithm to update the statistical information of multiple nodes on the path in the opposite direction of the traversal path, that is, the cumulative score Q i and the number of visits ni .

[0104] The multiple upper-level nodes may be all or part of the upper-level nodes on the traversal path from the leaf node to the root node.

[0105] The process of using the Monte Carlo tree search algorithm can be a process of traversing and searching on an already constructed search tree, that is, the full set of search trees has been constructed, and the Monte Carlo tree search method is used to find the optimal traversal path. Then, during back propagation, the statistical information of all nodes on the traversal path can be updated until the root node.

[0106] The multiple upper nodes may also be part of the upper nodes in the traversal path. It is understandable that the process of using the Monte Carlo tree search algorithm may also be a process of building a search tree while finding the optimal traversal path. That is, each iteration can determine an expandable node, and a leaf node is generated based on the node for simulation. This simulation process may be a virtual construction process. The multiple lower nodes and leaf nodes generated by the simulation process may not be stored in the current search tree. The possibility of the expandable node and its multiple upper nodes becoming the optimal node may be evaluated only according to the objective function value of the leaf node; thus, the statistical information of the multiple upper nodes may be updated in reverse from the expandable node until the root node. Since the multiple nodes between the lower node and the leaf node of the expansion node are not stored in the current search tree, they do not need to be evaluated, that is, the statistical information of the multiple nodes below the lower node may not be updated. After the evaluation is completed, the nodes generated in the simulation process may be deleted to save storage space. In this way, the process of traversing the search while building the search tree is realized. The purpose of the traversal is to obtain the statistical information of each node, so that the current optimal traversal path can be continuously found dynamically according to the statistical information of the existing nodes.

[0107] For example, using Figure 5 Through the simulation process, we get a leaf node (CBDAE) with an objective function value of 10, then we can update Figure 5 The cumulative scores Q corresponding to the CB node, the upper node C and the root node selected in the middle expansion phase i and the number of visits n i , add 10 to the Q of these nodes i At the same time, the number of visits to these nodes is increased by 1. For example, Q i Updated from 20 to 30, and the number of CB visits n i Add 1.

[0108] It is understandable that in the early iteration process of the Monte Carlo tree search algorithm, most nodes are unvisited nodes, and the UCB values ​​are all ∞ (infinity). When expanding a current child node, its selectable lower-level nodes can be determined based on the heuristic strategy first; if the lower-level nodes do not meet the heuristic strategy, an unvisited node is selected for expansion, so that as the number of iterations increases, most nodes are visited, and the cumulative scores and visit times of most nodes are updated using the back propagation method. In the later iterations, since the UCB values ​​of most nodes can be calculated, the UCB values ​​and heuristic strategies can be better used to select nodes worth exploring, which can better guide the traversal direction of the search tree, so as to obtain a better traversal path when the estimated time is reached.

[0109] In one embodiment of the present invention, when the traffic order of the intersection control area is traversed using the Monte Carlo tree search, the current optimal traffic order will change as the number of iterations increases. After multiple leaf nodes have been determined or the preset time has been reached, if the current Monte Carlo tree search iteration is stopped, an optimal traversal path can be determined based on the states of each node of the currently constructed search tree, thereby determining a more reasonable and efficient traffic order. The traffic order of the multiple vehicles to be controlled corresponding to the intersection control area can be determined in the following manner provided by an embodiment of the present invention: determine the optimal traversal path based on the statistical information of each node of the search tree; determine the target leaf node corresponding to the optimal traversal path, and use the traffic order corresponding to the target leaf node as the traffic order of the intersection control area.

[0110] The optimal traversal path can be understood as: when selecting nodes at each level, select the node with the most visits until reaching the leaf node. For example, Figure 4 In the example, after the search tree is constructed using the Monte Carlo tree search algorithm, among the first-level child nodes, C has the most visits, so C is selected; among the next-level nodes CA, CB, CD, and CE of C, CB has the most visits, so CB is selected; among the next-level nodes CBDA and CBDE of CB, CBDA has the most visits, so CBDA is selected, and finally a leaf node CBDAE is selected. At this time, the passage order corresponding to the leaf node can be used as the passage order of the intersection control area.

[0111] According to the combination of Monte Carlo tree search algorithm and heuristic strategy, unreasonable traffic sequences (i.e., child nodes) can be quickly removed when traversing the search tree, and the potential of the child nodes (UCB value) and the heuristic strategy can be combined to guide the traversal direction of the search tree, thereby improving the time for searching the optimal traffic sequence, meeting the real-time requirements of intersection traffic, and improving the traffic efficiency of the intersection control area.

[0112] After determining the traffic order of the intersection control area, in one embodiment of the present invention, the expected arrival time of multiple vehicles to be controlled at the corresponding conflict partition can be determined in the following manner: based on the position of the conflict partition in the intersection control area and the traffic order, the expected arrival time of the multiple vehicles to be controlled at the conflict partition is determined.

[0113] In one embodiment of the present invention, the estimated arrival time can be first determined in the following manner: construct a time fitting model and multiple first constraints; the first constraints include the fastest time constraint, the collision constraint and the conflict interval constraint; according to the first constraints, the time fitting model is used to fit the estimated arrival times corresponding to the multiple vehicles to be controlled arriving at the conflict partitions.

[0114] It can be understood that the goal of traffic at an intersection is to maximize traffic efficiency, and accordingly, the goal can be positioned as minimizing the traffic delay time of multiple vehicles to be controlled. Therefore, in one embodiment of the present invention, a time fitting model can be constructed to minimize the sum of the traffic delay time of multiple vehicles to be controlled.

[0115] When constructing the time fitting model, each vehicle entering the intersection control area will be assigned a unique number V i , indicating that it is the i-th vehicle entering the control area. At the same time, Z i For vehicle V i The conflicting partitions will be passed. For example, Z i ={4,1} represents vehicle V i The vehicle will sequentially pass through conflict zone 4 and conflict zone 1. The positions of multiple conflict zones in the intersection control area can be determined based on the division of the conflict zones in the intersection control area and the road geometry structure of multiple lanes.

[0116] In one embodiment of the present invention, for the sake of simplicity, the following conditions can be set for the time fitting model: each vehicle shares its own status information (such as position, speed, etc.) and route information with surrounding vehicles and roadside equipment through wireless communication technology, and the communication process is completed instantaneously without problems such as packet loss and delay; to ensure safety, vehicles in the control area are prohibited from changing lanes, that is, vehicles are prohibited from switching between lanes in the same direction when passing through the control area; the speed of the vehicle remains unchanged during the process of passing through the conflict partition.

[0117] Before building a time fitting model, you can build the following constraints: fastest time constraint, collision constraint, and conflict interval constraint.

[0118] Due to physical constraints such as speed constraints and acceleration constraints, the time it takes for a vehicle to arrive at a conflicting partition must be greater than or equal to the fastest arrival time. Therefore, the fastest time constraint can be expressed as follows:

[0119] t a,i,z ≥t m,i,z

[0120] Among them, t a,i,z is the vehicle V i Estimated arrival time at conflicting partition Z, t m,i,z is the vehicle V i The fastest arrival time to conflict zone Z at maximum vehicle speed and acceleration.

[0121] In order to avoid a collision between two vehicles on the same lane, it is required that a certain safe headway be maintained between the front and rear vehicles. Therefore, in one embodiment of the present invention, a collision constraint may be constructed as follows:

[0122] t a,i,z -t a,j,z ≥Δ j,a

[0123] Among them, V i It is the rear car, V j It is the car in front of you in the same lane, Δ j,a is the minimum safe headway and is related to the action of the vehicle in front. The value of a represents the vehicle V j The time interval between left turns and right turns is 0, which indicates the safe driving direction of the vehicle.

[0124] It can be understood that the collision constraint requires that a vehicle can enter the corresponding conflict zone only after the other conflicting vehicle leaves the conflict zone.

[0125] Based on the condition that the speed of the vehicle remains unchanged when passing through the conflict partition, the time it takes for the vehicle to travel from one conflict partition to another can be simply determined based on the road geometry. At this time, define Δt i,z,z′ Indicates vehicle V i The time interval from the conflict zone Z to the conflict zone Z′ can be constructed as follows:

[0126] t a,i,z +Δt i,z,z′ =t a,i,z′

[0127] After constructing the fastest time constraint, the collision constraint, and the conflict interval constraint, in one embodiment of the present invention, a time fitting model is constructed by introducing binary variables, as shown below:

[0128]

[0129] subject to a,i,z ≥t m,i,z

[0130] t a,i,z -t a,j,z ≥Δ j,a

[0131] t a,k,z -t a,l,z +M·b k,l ≥Δ l,a

[0132] t a,l,z -t a,k,z +M·(1-b k,l )≥Δ k,a

[0133] t a,i,z +Δt i,z,z′ =t a,i,z′

[0134] b k,l ∈{0,1}

[0135] Where n is the total number of vehicles in the control area, M is a large constant, and b k,l is a binary number. k,l When it is equal to 0, the vehicle V l First arrive at the conflicting partition, otherwise, V k Arrive at the conflicting partition first. After a traffic order is obtained according to the Monte Carlo tree search algorithm, any two vehicles V can be determined according to the traffic order. l With V k b k, l value. Z i (1) is the set Z i The first element in is the vehicle V i The time to arrive at the first conflict zone. Since the vehicle's speed is constant after arriving at the first conflict zone, the time to pass other conflict zones is also fixed. The time fitting model can minimize the sum of the delay times of multiple vehicles arriving at their corresponding first conflict zones. According to the time fitting model, the estimated arrival time of multiple vehicles to be controlled at the corresponding first conflict zones is obtained by fitting.

[0136] After the estimated arrival time is determined, the vehicle can be controlled to arrive at the corresponding conflict partition within the estimated arrival time according to a certain driving speed. In one embodiment of the present invention, the following method can be used: determine the driving speed of the multiple vehicles to be controlled according to the estimated arrival time, the vehicle state function and the double integral dynamics model. Specifically, the driving speed can be determined as follows: construct a speed fitting model; for each of the vehicles to be controlled: determine the state parameters of the vehicle to be controlled; the state parameters include: vehicle driving time, vehicle driving position, vehicle maximum speed, vehicle minimum speed, vehicle constant speed, vehicle maximum acceleration and vehicle minimum acceleration; determine the vehicle state function according to the state parameters; fit the driving speed of the vehicle to be controlled according to the speed fitting model, the estimated arrival time, the vehicle state function and the double integral dynamics model.

[0137] According to the vehicle dynamics equation, the vehicle state function can be expressed as:

[0138]

[0139] x i (t) = [p i (t),v i (t)]

[0140] Among them, x i (t) is the vehicle V i The state function at time t has state parameters v i (t) and p i (t) is determined. Among them, v i (t) is the vehicle V i The velocity at time t, p i (t) is the vehicle V i The position at time t. For x i The first derivative of i (t) is the vehicle V i The acceleration at time t.

[0141] The double-integral kinetic model used is as follows:

[0142]

[0143]

[0144] in, For p i The first derivative of (t) is equal to v i (t), For p i The second derivative of (t) is equal to u i(t).

[0145] The speed fitting model constructed by one embodiment of the present invention may be as follows:

[0146]

[0147] subject to you min ≤u i (t)≤u max

[0148] v min ≤v i (t)≤v max

[0149]

[0150]

[0151]

[0152]

[0153]

[0154] in, is the initial moment, is the initial position, is the initial velocity, v c is the constant speed within the conflict zone. max and v min is the maximum speed and minimum speed of the vehicle, u max and u min are the maximum acceleration of the vehicle and the minimum acceleration of the vehicle.

[0155] According to the speed fitting model, the acceleration of each vehicle to be controlled at each moment is fitted, so as to determine the driving speed of each vehicle to be controlled, and then the vehicle to be controlled can be controlled to arrive at the corresponding first conflict zone Z at the estimated arrival time through the driving speed. i (1), subsequent Z i For other conflicting partitions in the set, the constant driving speed v c Just go through them in order.

[0156] The time fitting model and the speed fitting model are used to achieve orderly passage according to the optimal passage sequence, reducing the passage delay time of vehicles to be controlled, thereby improving the efficiency of vehicle passage in the intersection control area.

[0157] Take a traffic scene in a traffic control area as an example. Figure 6A vehicle control method provided by an embodiment of the present invention is described in detail, and specific steps may include:

[0158] Step S601: Determine a plurality of conflicting zones and a plurality of vehicles to be controlled in a road intersection control zone.

[0159] Step S602: construct a search tree according to the number and identification of multiple vehicles to be controlled.

[0160] The root node of the search tree is Root, which corresponds to an empty node and generates multiple first-level child nodes.

[0161] Next, the Monte Carlo tree search algorithm can be used to expand the root node.

[0162] Step S603: Determine the first-level child node with the largest UCB value in the search tree as the current child node.

[0163] Step S604: Determine the target node as a subordinate node of the current child node according to the UCB value and / or the identifier of the target vehicle.

[0164] Step S605: Determine whether the lower-level node is a leaf node; if so, execute step S606; if not, return to execute step S604.

[0165] The lower-level node is used as the current child node, and S604 is continued to be executed.

[0166] Step S606: According to the objective function value of the leaf node, the statistical information of multiple upper-level nodes is reversely updated.

[0167] Step S607: Determine whether the preset time has been reached; if yes, execute step S608; if no, return to execute step S603.

[0168] Of course, if it is determined that multiple leaf nodes have been generated before the preset time is reached, the iteration process can also be stopped. It is understandable that, if time permits, multiple iterations of the Monte Carlo tree search can be performed as much as possible, which can effectively improve the accuracy of determining the optimal pass order.

[0169] Step S608: Determine the target leaf node corresponding to the optimal traversal path based on the statistical information of each node in the search tree.

[0170] Step S609: Determine the estimated arrival time according to the passage sequence corresponding to the target leaf node.

[0171] For each control vehicle, the estimated arrival time of the first conflicting partition it will pass through is determined.

[0172] Step S610: Determine the travel speed of each vehicle to be controlled according to the estimated arrival time.

[0173] Step S611: Control the vehicle to be controlled to pass through the intersection control area according to the driving speed and the traffic order.

[0174] According to an embodiment of the present invention, a vehicle control method is provided, which can use a target vehicle that meets the conditions as a heuristic strategy of a Monte Carlo tree search algorithm, determine the passage order of multiple vehicles to be controlled in an intersection control area according to the Monte Carlo tree search algorithm, and determine the driving speed of the vehicles to be controlled, so as to control the vehicles to be controlled to pass through the intersection control area; thus, in combination with the Monte Carlo tree search algorithm and the heuristic strategy, the traversal direction of the search tree can be quickly and accurately guided, avoiding the situation where all nodes need to be traversed exhaustively to obtain the optimal passage order, thereby improving the efficiency of the tree search; it also avoids the problem of excessive calculation time due to the increase in vehicles and lanes, meets the real-time requirements of vehicle passage, and thus improves the efficiency of vehicle passage in the intersection control area.

[0175] Furthermore, the travel speed of the vehicle to be controlled is determined according to the constructed time fitting model and speed fitting model, and the vehicle to be controlled is controlled according to the travel speed, thereby achieving reasonable control of the vehicle and improving the efficiency of vehicle traffic in the intersection control area.

[0176] like Figure 7 As shown, the embodiment of the present invention provides a vehicle control device 700, including: a determination module 701, a search module 702 and a control module 703; wherein,

[0177] The determination module 701 is used to determine a plurality of lanes in the intersection control area, one or more conflicting zones corresponding to the plurality of lanes, and a plurality of vehicles to be controlled;

[0178] The search module 702 is used to determine the passage order of the multiple vehicles to be controlled corresponding to the intersection control area by using a Monte Carlo tree search algorithm according to the number of the multiple vehicles to be controlled and the identification of the target vehicle; the target vehicle is a vehicle in a lane closest to the conflict partition and / or a vehicle with the shortest arrival time for the same conflict partition;

[0179] The control module 703 is used to determine the driving speed of the vehicle to be controlled according to the traffic order and the position of the one or more conflicting partitions in the intersection control area; and control the vehicle to be controlled to pass through the intersection control area according to the traffic order and the driving speed.

[0180] In one embodiment of the present invention, the determination module 701 is used to select a vehicle that is in the same lane and closest to the conflict partition as the target vehicle; and / or, from multiple vehicles to be controlled corresponding to the same conflict partition, the corresponding vehicle with the shortest fastest arrival time is used as the target vehicle; the fastest arrival time indicates the time when the vehicle arrives at the conflict partition at the maximum speed and acceleration.

[0181] In one embodiment of the present invention, the search module 702 is used to construct a root node and a first-level child node of a search tree according to the identifications of the multiple vehicles to be controlled and the number of vehicles; one of the first-level child nodes corresponds to a vehicle identification; according to the identification of the target vehicle, the root node of the search tree is expanded using the Monte Carlo tree search algorithm to generate one or more leaf nodes; the leaf nodes indicate the passage order corresponding to the multiple vehicles to be controlled; according to the passage order indicated by the leaf nodes, the passage order of the multiple vehicles to be controlled corresponding to the intersection control area is determined.

[0182] In one embodiment of the present invention, the search module 702 is used to loop the following steps until a preset time is reached: according to the UCB algorithm of the Monte Carlo tree search, determine that the first-level child node with the largest UCB value is the current child node; loop the following steps until the determined lower-level node is a leaf node: using the Monte Carlo tree search algorithm, according to the UCB value and / or the identification of the target vehicle, determine the target node as the lower-level node of the current child node, and use the lower-level node as the current child node.

[0183] In one embodiment of the present invention, the search module 702 is used to determine whether there is a target node in the first-level sub-node corresponding to the target vehicle, and the target node is not the upper node of the current sub-node; if so, determine the lower node of the current sub-node from the target node.

[0184] In one embodiment of the present invention, the search module 702 is used to determine the target node with the largest UCB value as the subordinate node of the current child node according to the UCB values ​​of the multiple target nodes when there are multiple target nodes.

[0185] In one embodiment of the present invention, the search module 702 is used to reversely update the statistical information corresponding to multiple upper-level nodes of the leaf node according to the objective function value corresponding to the leaf node by using the back-propagation algorithm of the Monte Carlo tree search after determining that the lower-level node is a leaf node, and the statistical information includes the cumulative score and the number of visits.

[0186] In one embodiment of the present invention, the search module 702 is used to determine, for each first-level child node: the UCB value of the first-level child node based on the statistical information of the first-level child node, the statistical information of the root node and the weight parameter; and take the first-level child node with the largest UCB value as the current child node.

[0187] In one embodiment of the present invention, the control module 703 is used to determine the optimal traversal path based on the statistical information of each node of the search tree; determine the target leaf node corresponding to the optimal traversal path, and use the passage order corresponding to the target leaf node as the passage order of the intersection control area.

[0188] In one embodiment of the present invention, the control module 703 is used to determine, for multiple vehicles to be controlled corresponding to the same conflict partition: according to the position of the conflict partition in the intersection control area and the traffic order, the estimated arrival time of the multiple vehicles to be controlled to arrive at the conflict partition; and determine the driving speed of the multiple vehicles to be controlled according to the estimated arrival time, the vehicle state function and the double integral dynamics model.

[0189] In one embodiment of the present invention, the control module 703 is used to construct a time fitting model and multiple first constraints; the first constraints include the fastest time constraint, the collision constraint and the conflict interval constraint; according to the first constraints, the time fitting model is used to fit the estimated arrival times corresponding to the multiple vehicles to be controlled arriving at the conflict partitions.

[0190] In one embodiment of the present invention, the control module 703 is used to construct a speed fitting model; for each of the vehicles to be controlled: determine the state parameters of the vehicle to be controlled; the state parameters include: vehicle driving time, vehicle driving position, vehicle maximum speed, vehicle minimum speed, vehicle constant speed, vehicle maximum acceleration and vehicle minimum acceleration; determine the vehicle state function according to the state parameters; fit the driving speed of the vehicle to be controlled according to the speed fitting model, the estimated arrival time, the vehicle state function and the double integral dynamics model.

[0191] A vehicle control device provided according to an embodiment of the present invention can use a target vehicle that meets the conditions as a heuristic strategy for a Monte Carlo tree search algorithm, determine the passage order of multiple vehicles to be controlled in an intersection control area according to the Monte Carlo tree search algorithm, and determine the driving speed of the vehicles to be controlled, thereby controlling the vehicles to be controlled to pass through the intersection control area; thus, in combination with the Monte Carlo tree search algorithm and the heuristic strategy, the traversal direction of the search tree can be quickly and accurately guided, avoiding the situation where all nodes need to be exhaustively traversed to obtain the optimal passage order, thereby improving the efficiency of the tree search; it also avoids the problem of excessive calculation time due to the increase in vehicles and lanes, meets the real-time requirements of vehicle passage, and thereby improves the efficiency of vehicle passage in the intersection control area.

[0192] Furthermore, the travel speed of the vehicle to be controlled is determined according to the constructed time fitting model and speed fitting model, and the vehicle to be controlled is controlled according to the travel speed, thereby achieving reasonable control of the vehicle and improving the efficiency of vehicle traffic in the intersection control area.

[0193] Figure 8 An exemplary system architecture 800 is shown in which a vehicle control method or a vehicle control device according to an embodiment of the present invention can be applied.

[0194] like Figure 8 As shown, the system architecture 800 may include terminal devices 801, 802, 803, a network 804, and a server 805. The network 804 is used to provide a medium for communication links between the terminal devices 801, 802, 803 and the server 805. The network 804 may include various connection types, such as wired, wireless communication links, vehicle networks, or optical fiber cables, etc.

[0195] Users can use terminal devices 801, 802, 803 to interact with server 805 through network 804 to receive or send messages, etc.

[0196] Terminal devices 801, 802, and 803 may be vehicles equipped with sensor devices, wireless communication devices, or autonomous driving control systems.

[0197] The server 805 may be a server that provides various services, for example, a server that uses wireless communication technology, sensor technology, etc. to share and interact with terminal devices 801, 802, and 803; or a server that obtains the location, speed, and route data of the vehicle and uses these data to perform big data processing and vehicle-road cooperative control. The server can analyze and process the vehicle data and the data of the intersection control area, and control and interact with the vehicle according to the processing results.

[0198] It should be noted that the vehicle control method provided in the embodiment of the present invention is generally executed by the server 805 , and accordingly, the vehicle control device is generally arranged in the server 805 .

[0199] It should be understood that Figure 8 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0200] Reference below Fig. 9 , which shows a schematic diagram of the structure of a computer system 900 of a terminal device suitable for implementing an embodiment of the present invention. Fig. 9 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0201] like Fig. 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the system 900 are also stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0202] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage section 908 as needed.

[0203] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above-mentioned functions defined in the system of the present invention are executed.

[0204] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0205] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0206] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes a determination module, a search module, and a control module. The names of these modules do not constitute a limitation on the modules themselves in some cases, for example, the determination module may also be described as a "module for determining a vehicle".

[0207] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: determining multiple lanes in the intersection control area, one or more conflict partitions corresponding to the multiple lanes, and multiple vehicles to be controlled; according to the number of the multiple vehicles to be controlled and the identification of the target vehicle, using the Monte Carlo tree search algorithm, determine the passage order of the multiple vehicles to be controlled corresponding to the intersection control area; the target vehicle is the vehicle in the lane closest to the conflict partition, and / or the vehicle with the shortest arrival time for the same conflict partition; according to the passage order and the position of the one or more conflict partitions in the intersection control area, determine the driving speed of the vehicle to be controlled; according to the passage order and the driving speed, control the vehicle to be controlled to pass through the intersection control area.

[0208] According to the technical solution of the embodiment of the present invention, the target vehicle that meets the conditions can be used as the heuristic strategy of the Monte Carlo tree search algorithm, and the passage order of multiple vehicles to be controlled in the intersection control area is determined according to the Monte Carlo tree search algorithm, and the driving speed of the vehicles to be controlled is determined, so as to control the vehicles to be controlled to pass through the intersection control area; thus, in combination with the Monte Carlo tree search algorithm and the heuristic strategy, the traversal direction of the search tree can be guided quickly and accurately, avoiding the situation where all nodes need to be traversed exhaustively to obtain the optimal passage order, thereby improving the efficiency of the tree search; it also avoids the problem of excessive calculation time due to the increase in vehicles and lanes, meets the real-time requirements of vehicle passage, and thus improves the efficiency of vehicle passage in the intersection control area.

[0209] Furthermore, the travel speed of the vehicle to be controlled is determined according to the constructed time fitting model and speed fitting model, and the vehicle to be controlled is controlled according to the travel speed, thereby achieving reasonable control of the vehicle and improving the efficiency of vehicle traffic in the intersection control area.

[0210] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A vehicle control method, characterized in that: include: Determine a plurality of lanes within a control area of ​​an intersection, one or more conflicting zones corresponding to the plurality of lanes, and a plurality of vehicles to be controlled; According to the number of the plurality of vehicles to be controlled and the identification of the target vehicle, using a Monte Carlo tree search algorithm, determining the passage order of the plurality of vehicles to be controlled corresponding to the intersection control area; The target vehicle is a vehicle in a lane closest to the conflicting partition, and / or a vehicle with a shortest arrival time for the same conflicting partition; Determine the travel speed of the vehicle to be controlled according to the traffic order and the position of the one or more conflicting partitions in the intersection control area; wherein, the determining the travel speed of the vehicle to be controlled includes: for multiple vehicles to be controlled corresponding to the same conflicting partition: determine the estimated arrival time of the multiple vehicles to be controlled to the conflicting partition according to the position of the conflicting partition in the intersection control area and the traffic order; determine the travel speed of the multiple vehicles to be controlled according to the estimated arrival time, the vehicle state function and the double integral dynamics model; According to the traffic order and the travel speed, the vehicle to be controlled is controlled to pass through the intersection control area.

2. The method according to claim 1, characterized in that: The method of using a Monte Carlo tree search algorithm to determine the passage order of the plurality of vehicles to be controlled corresponding to the intersection control area includes: According to the identifications of the multiple vehicles to be controlled and the number of the vehicles, a root node and first-level child nodes of a search tree are constructed; one first-level child node corresponds to one vehicle identification; According to the identification of the target vehicle, the root node of the search tree is expanded using the Monte Carlo tree search algorithm to generate one or more leaf nodes; the leaf nodes indicate the passage order corresponding to the multiple vehicles to be controlled; According to the passage sequence indicated by the leaf node, the passage sequence of the multiple vehicles to be controlled corresponding to the intersection control area is determined.

3. The method according to claim 2, characterized in that The step of expanding the root node of the search tree to generate one or more leaf nodes includes: The following steps are executed repeatedly until the preset time is reached: According to the UCB algorithm of Monte Carlo tree search, the first-level child node with the largest UCB value is determined as the current child node; The following steps are executed repeatedly until the lower-level node is determined to be a leaf node: The Monte Carlo tree search algorithm is used to determine the target node as a subordinate node of the current child node according to the UCB value and / or the identifier of the target vehicle, and the subordinate node is used as the current child node.

4. The method according to claim 3, characterized in that The step of determining the target node as a subordinate node of the current child node according to the UCB value and / or the identifier of the target vehicle includes: Determine whether there is a target node in the first-level child node corresponding to the target vehicle, and the target node is not the parent node of the current child node; If yes, determine the subordinate node of the current child node from the target node.

5. The method according to claim 4, characterized in that In the case of multiple target nodes, According to the UCB values ​​of the multiple target nodes, a target node with the largest UCB value is determined as a subordinate node of the current child node.

6. The method according to claim 3, characterized in that After determining that the lower-level node is a leaf node, the following steps are also included: According to the objective function value corresponding to the leaf node, the statistical information corresponding to multiple parent nodes of the leaf node is reversely updated using the back propagation algorithm of the Monte Carlo tree search, and the statistical information includes the accumulated score and the number of visits.

7. The method according to claim 6, characterized in that The method of determining the first-level child node with the largest UCB value as the current child node according to the UCB algorithm of the Monte Carlo tree search includes: For each first-level child node: determining the UCB value of the first-level child node according to the statistical information of the first-level child node, the statistical information of the root node, and a weight parameter; The first-level child node with the largest UCB value is used as the current child node.

8. The method according to claim 1, characterized in that The vehicle that is in the same lane and closest to the conflicting partition is taken as the target vehicle; and / or, From a plurality of to-be-controlled vehicles corresponding to the same conflict partition, the corresponding vehicle with the shortest arrival time is selected as the target vehicle; The fastest arrival time indicates the time when the vehicle arrives at the conflict zone at the maximum vehicle speed and acceleration.

9. The method according to claim 2, characterized in that: The determining the passage order of the plurality of vehicles to be controlled corresponding to the intersection control area includes: Determine an optimal traversal path based on statistical information of each node of the search tree; The target leaf node corresponding to the optimal traversal path is determined, and the passage order corresponding to the target leaf node is used as the passage order of the intersection control area.

10. The method according to claim 1, characterized in that The determining of the estimated arrival time of the plurality of vehicles to be controlled at the conflicting zone comprises: Constructing a time fitting model and a plurality of first constraints; the first constraints include a fastest time constraint, a collision constraint and a conflict interval constraint; According to the first constraint condition, the time fitting model is used to fit the estimated arrival times of the multiple vehicles to be controlled arriving at the conflicting partitions.

11. The method according to claim 10, characterized in that The determining the driving speeds of the plurality of vehicles to be controlled according to the estimated arrival time, the vehicle state function and the double integral dynamics model comprises: Construct velocity fitting model; For each of the vehicles to be controlled: determining the state parameters of the vehicle to be controlled; the state parameters include: vehicle driving time, vehicle driving position, vehicle maximum speed, vehicle minimum speed, vehicle constant speed, vehicle maximum acceleration and vehicle minimum acceleration; determining the vehicle state function according to the state parameter; The driving speed of the vehicle to be controlled is fitted according to the speed fitting model, the estimated arrival time, the vehicle state function, and the double-integral dynamics model.

12. A vehicle control device, characterized in that: include: Determination module, search module and control module; wherein, The determination module is used to determine a plurality of lanes in the intersection control area, one or more conflicting zones corresponding to the plurality of lanes, and a plurality of vehicles to be controlled; The search module is used to determine the passage order of the multiple vehicles to be controlled corresponding to the intersection control area by using a Monte Carlo tree search algorithm according to the number of the multiple vehicles to be controlled and the identification of the target vehicle; the target vehicle is a vehicle in a lane closest to the conflict partition and / or a vehicle with the shortest arrival time for the same conflict partition; The control module is used to determine the travel speed of the vehicle to be controlled according to the traffic order and the position of the one or more conflict partitions in the intersection control area; and control the vehicle to be controlled to pass through the intersection control area according to the traffic order and the travel speed; wherein, the determination of the travel speed of the vehicle to be controlled includes: for multiple vehicles to be controlled corresponding to the same conflict partition: determining the estimated arrival time of the multiple vehicles to be controlled to arrive at the conflict partition according to the position of the conflict partition in the intersection control area and the traffic order; determining the travel speed of the multiple vehicles to be controlled according to the estimated arrival time, the vehicle state function and the double integral dynamics model.

13. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 11.

14. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images