Unmanned vehicle coordination control method, device, equipment, medium and program product

Abstract

The invention discloses an unmanned vehicle coordination control method and device, equipment, a medium and a program product, and the method comprises the steps: employing an unmanned vehicle passing right coordination technology based on deep learning based on the states of a plurality of to-be-coordinated unmanned vehicles corresponding to a target intersection and an intersection map corresponding to the target intersection; according to the method, the passing right priorities of the multiple unmanned vehicles to be coordinated are optimized, the multiple unmanned vehicles to be coordinated are controlled based on the passing right priorities, orderly passing of the multiple unmanned vehicles at intersections of different topological structures of a mine can be coordinated, and the problem that a method based on passing right optimization has high requirements for computing power can be solved. Therefore, the calculation real-time requirement for landing of the technical scheme in engineering is met, especially for the working condition that the number of unmanned vehicles to be coordinated is large.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of automatic driving, and particularly relates to a method and device for coordinating and controlling unmanned vehicles, a medium, and a program product. BACKGROUND

[0002] Automatic driving is increasingly attracting attention from the academic and industrial communities because it can solve various traffic challenges related to safety, congestion, energy saving, and the like. For example, in the mining field, mine trucks (referred to as mine trucks) as important transport tools in mines have large load capacity and high fuel consumption. How to balance the transportation efficiency and energy consumption cost of mine trucks is a major problem in the field of mine truck transportation. The application of unmanned vehicles, which take automatic driving as the core technology, brings great changes to the mining field, improves production efficiency, enhances safety, and increases resource utilization. With the continuous progress of automatic driving technology, unmanned vehicles are expected to become an important standard configuration in the future mining industry, promoting the entire industry to develop in the direction of intelligentization and high efficiency.

[0003] However, the mine road environment is complex, the shape of the intersection in the mine area is irregular, and there is no signal light for centralized control, which increases the difficulty of controlling unmanned vehicles and is prone to collision accidents, seriously affecting the transportation efficiency and safety of the mine area. Therefore, there is an urgent need for a solution that can coordinate the safe and fast passage of unmanned vehicles at intersections without signal lights. SUMMARY

[0004] The embodiments of the present application provide a method and device for coordinating and controlling unmanned vehicles, a medium, and a program product, which can control the safe and fast passage of unmanned vehicles at intersections without signal lights.

[0005] In a first aspect, the embodiments of the present application provide a method for coordinating and controlling unmanned vehicles, comprising:

[0006] determining a plurality of unmanned vehicles to be coordinated corresponding to a target intersection;

[0007] generating a state space corresponding to the target intersection based on state information of the plurality of unmanned vehicles to be coordinated;

[0008] inputting the state space corresponding to the target intersection and an intersection map into a pre-trained deep learning model to obtain a priority of the plurality of unmanned vehicles to be coordinated at the target intersection output by the deep learning model;

[0009] controlling the plurality of unmanned vehicles to be coordinated to pass through the target intersection in turn based on the priority of the plurality of unmanned vehicles to be coordinated at the intersection;

[0010] wherein the intersection map is generated based on a topological structure of the target intersection.

[0011] The deep learning model is trained based on multiple sets of training data. Each set of training data includes a sample state space, a sample intersection map, and label data corresponding to the sample state space. The sample state space is generated based on the states of multiple sample unmanned vehicles corresponding to a sample intersection. The sample intersection map is generated based on the topological structure of the sample intersection. The label data is used to indicate the priority of the multiple sample unmanned vehicles at the sample intersection.

[0012] In a second aspect, an embodiment of the present application provides an unmanned vehicle coordination control device, comprising:

[0013] A determination module is configured to determine multiple unmanned vehicles to be coordinated corresponding to a target intersection.

[0014] A state space generation module is configured to generate a state space corresponding to the target intersection based on state information of the multiple unmanned vehicles to be coordinated.

[0015] A right-of-way prediction module is configured to input the state space corresponding to the target intersection and an intersection map into a pre-trained deep learning model to obtain a priority of the multiple unmanned vehicles to be coordinated at the target intersection output by the deep learning model.

[0016] A control module is configured to control the multiple unmanned vehicles to be coordinated to pass through the target intersection in sequence based on the priority of the multiple unmanned vehicles to be coordinated at the intersection.

[0017] The intersection map is generated based on the topological structure of the target intersection.

[0018] The deep learning model is trained based on multiple sets of training data. Each set of training data includes a sample state space, a sample intersection map, and label data corresponding to the sample state space. The sample state space is generated based on the states of multiple sample unmanned vehicles corresponding to a sample intersection. The sample intersection map is generated based on the topological structure of the sample intersection. The label data is used to indicate the priority of the multiple sample unmanned vehicles at the sample intersection.

[0019] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory storing computer program instructions.

[0020] The processor executes the computer program instructions to implement the unmanned vehicle coordination control method of the first aspect.

[0021] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores computer program instructions. When the computer program instructions are executed by a processor, the unmanned vehicle coordination control method of the first aspect is implemented.

[0022] In a fifth aspect, an embodiment of the present application provides a computer program product, instructions in the computer program product being executed by a processor of an electronic device to cause the electronic device to perform the unmanned vehicle coordination control method according to the first aspect.

[0023] The unmanned vehicle coordination control method, device, equipment, medium and program product provided by the embodiments of the present application are based on the states of the plurality of unmanned vehicles to be coordinated corresponding to the target intersection and the intersection map corresponding to the target intersection, adopt the unmanned vehicle right-of-way coordination technology based on deep learning to optimize the right-of-way priority of the plurality of unmanned vehicles to be coordinated, and control the plurality of unmanned vehicles to be coordinated based on the right-of-way priority, so as to realize the orderly passing of the plurality of unmanned vehicles at different topological structure intersections in the mine, and overcome the problem of high computing power requirement of the right-of-way optimization-based method, thereby meeting the computing real-time requirement of the landing of the technical solution in engineering, especially for the working condition in which the number of unmanned vehicles to be coordinated is large. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments of the present application will be briefly introduced below. Those skilled in the art can also obtain other drawings according to these drawings without creating any creative labor.

[0025] Figure 1 is a flowchart of the unmanned vehicle coordination control method provided by the embodiments of the present application;

[0026] Figure 2 is a topological structure diagram of an intersection provided by the embodiments of the present application;

[0027] Figure 3 is a schematic diagram of an intersection map provided by the embodiments of the present application;

[0028] Figure 4 is a schematic diagram of the division of an intersection map area provided by the embodiments of the present application;

[0029] Figure 5 is a flowchart of the deep learning model training method provided by the embodiments of the present application;

[0030] Figure 6 is a model processing logic diagram provided by the embodiments of the present application;

[0031] Figure 7 is a model structure diagram provided by the embodiments of the present application;

[0032] Figure 8 is a structure diagram of the unmanned vehicle coordination control device provided by the embodiments of the present application;

[0033] Figure 9 FIG. 1 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0034] The features and exemplary embodiments of various aspects of the present application will be described below in detail, in order to make the purposes, technical solutions and advantages of the present application more clear and apparent, the present application will be further described in detail below in combination with the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, but not to limit the present application. The present application can be implemented without some of these specific details by those skilled in the art. The following description of the embodiments is only to provide a better understanding of the present application by showing examples of the present application.

[0035] It should be noted that, in this paper, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the elements defined by the statement "include" do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0036] Autonomous driving is attracting more and more attention from academia and industry, because it can solve many traffic challenges related to safety, congestion, energy saving, etc. In the field of mining, the application of unmanned mine trucks and other unmanned vehicles, which take autonomous driving as the core technology, is undoubtedly a disruptive technological innovation.

[0037] Firstly, unmanned vehicles can achieve efficient transportation. For example, in the field of mining, after the ore is mined, it needs to be transported to the crushing station or other destinations by mine trucks, and its transportation link is very long and the road conditions are very poor, which causes the mine truck drivers to have a very high work intensity. Therefore, mine truck drivers must adopt a shift system to ensure rest. The shift of mine truck drivers inevitably leads to a decrease in transportation efficiency and mine production capacity. Autonomous driving technology enables unmanned vehicles to navigate and travel autonomously in the mine. In this way, unmanned vehicles can operate 24 hours a day without interruption, greatly improving transportation efficiency and mine production capacity.

[0038] Secondly, unmanned vehicles can improve safety. The mining industry often faces safety hazards such as explosions and landslides. Unmanned vehicles do not require mine truck drivers to work on site, thereby ensuring their personal safety.

[0039] Furthermore, unmanned vehicles can improve resource utilization. Based on real-time data analysis, they can achieve more precise collection and delivery of ores and materials, thereby increasing resource utilization efficiency and reducing costs.

[0040] Autonomous vehicles will bring tremendous changes to the mining industry. They improve production efficiency, enhance safety, and increase resource utilization. With the continuous advancement of autonomous driving technology, autonomous vehicles are expected to become an essential standard in the future mining industry, driving the entire industry towards intelligent and efficient development.

[0041] However, due to the varied routes, lengths, and intersections of unmanned vehicles operating in mining areas, the installation and relocation of traffic lights inevitably leads to repetitive investments of manpower, materials, and financial resources. Furthermore, traffic light signal transmission range is limited, highly susceptible to weather conditions, and easily obstructed. Traffic lights can only be installed at intersections, preventing unmanned vehicles from slowing down before reaching them. Using computer vision for traffic light detection and recognition places high demands on the perception algorithms and computing power of unmanned vehicles, and is prone to recognition errors and delays. Fixed traffic light timing strategies lack flexibility, cannot dynamically adjust according to actual traffic flow, and cannot guarantee optimal overall control. Even if there are no vehicles at an intersection, fixed timing strategies still require vehicles to wait, impacting production efficiency. Therefore, there is an urgent need for a solution that can coordinate the safe and rapid passage of unmanned vehicles at intersections without traffic lights.

[0042] To address the problems of existing technologies, embodiments of this application provide a method, apparatus, device, medium, and program product for coordinated control of unmanned vehicles. The coordinated control method for unmanned vehicles provided in this application embodiment will be described first below.

[0043] The unmanned vehicle coordination control method provided in this application can be applied to autonomous driving scenarios such as mines, for coordinating and controlling the right-of-way of unmanned vehicles at intersections without traffic lights. The inventive concept of this unmanned vehicle coordination control method is based on optimizing the right-of-way for coordinating unmanned vehicles. Considering that the optimization-based right-of-way coordination method requires high computing power, especially when there are a large number of unmanned vehicles to be coordinated at the intersection, the real-time computing requirements in engineering limit the application of the optimization-based right-of-way coordination method. Therefore, based on the optimization-based right-of-way coordination method, this application further proposes a deep learning-based method. Compared to the optimization-based right-of-way coordination method, this method has the advantage of significantly improved computational efficiency, making it more suitable for engineering applications. Because using deep learning to model the right-of-way coordination problem is equivalent to transforming it into a matrix operation problem, and with the help of GPU (graphics processing unit) computing architecture optimization for matrix operations, computational efficiency can be improved by one to three orders of magnitude.

[0044] The unmanned vehicle coordination control method provided in this application embodiment can be executed by a processor or a controller, wherein the processor or controller can be set in the cloud or on the local end, and the local end includes but is not limited to the vehicle terminal.

[0045] See Figure 1 This is a flowchart illustrating a coordinated control method for unmanned vehicles provided in an embodiment of this application. Figure 1 As shown, the method includes at least the following steps S11-S14, which will be described in detail below.

[0046] S11. Identify the multiple unmanned vehicles to be coordinated at the target intersection.

[0047] In this embodiment, the target intersection can be any intersection in the unmanned vehicle operation scenario. Unmanned vehicle operation scenarios include, but are not limited to, mining scenarios.

[0048] In this embodiment, the unmanned vehicle to be coordinated corresponding to the target intersection refers to the unmanned vehicle that needs to be coordinated and controlled at the target intersection location.

[0049] In some embodiments of this application, taking a mining scenario as an example, there may be multiple unmanned mining trucks that need to pass through a target intersection. Due to the complex road environment and frequent unexpected situations in mines, these unmanned mining trucks are very likely to collide or cause congestion, thereby affecting mine production efficiency. In this case, to avoid collisions or congestion at the target intersection, multiple unmanned mining trucks can be coordinated and controlled. The unmanned mining trucks that need to be coordinated and controlled are the unmanned vehicles corresponding to the target intersection.

[0050] In some embodiments of this application, unmanned vehicles located within the coordination area corresponding to the target intersection and traveling towards the center point of the target intersection can be designated as unmanned vehicles to be coordinated at the target intersection. The coordination area is the area where coordination and control of the unmanned vehicles are required. The coordination area can be set according to actual conditions. Based on this, when determining unmanned vehicles to be coordinated, the location information and driving direction of each unmanned vehicle within the area corresponding to the target intersection can be obtained. The obtained location information of the unmanned vehicles can be compared with the location information corresponding to the preset coordination area to determine whether the unmanned vehicle is within the coordination area. Finally, unmanned vehicles located within the coordination area and traveling towards the center point of the target intersection are designated as mining trucks to be coordinated. In this way, unmanned vehicles to be coordinated can be quickly identified.

[0051] In some embodiments of this application, to avoid frequent switching of the right-of-way for unmanned vehicles, a fixed right-of-way zone can be set for the target intersection. Once an unmanned vehicle enters the fixed right-of-way zone, its right-of-way will no longer be switched. Based on this, before performing step S11 above, the area to be coordinated corresponding to the target intersection can be determined through the following steps S111-S115:

[0052] S111. Obtain the topology of the target intersection.

[0053] Here, the topology of the target intersection includes, but is not limited to, the following: Figure 2 The examples shown are crossroads (a), X-shaped intersections (b), Y-shaped intersections (c), T-shaped intersections (d), and star-shaped intersections (f).

[0054] In some embodiments of this application, the topology of each intersection in the unmanned vehicle operation scenario can be stored in advance in the local memory of the processor or controller or in an accessible remote memory. Based on this, the pre-stored topology of the target intersection can be obtained directly from the local server or the remote server.

[0055] S112. Based on the topology, generate the intersection map corresponding to the target intersection.

[0056] In some embodiments of this application, the intersection map can be represented as an occupancy grid with multiple channels. The intersection map includes, but is not limited to, the following channels: drivable area channels, elevation channels, etc. The drivable area channel indicates the drivable and non-drivable areas corresponding to the target intersection. Each grid node in the drivable area channel can have a value of 0 or 1, where 0 represents a non-drivable area and 1 represents a drivable area. The elevation channel indicates the elevation information corresponding to the target intersection. Each grid node in the elevation channel displays the current local road elevation. For example, taking a crossroads intersection as an example, its corresponding intersection map can be represented as follows: Figure 3 As shown, Figure (A) on the left represents the drivable area passage, and Figure (B) on the right represents the elevation passage. For the drivable area passage, white represents a value of 0, and gray represents a value of 1. For the elevation passage, the darker the color, the higher the altitude.

[0057] In addition, after obtaining the intersection map corresponding to the target intersection, the association between the intersection map and the target intersection can be established and stored so that the intersection map corresponding to the target intersection can be directly obtained later.

[0058] S113. The area on the intersection map that is the first distance from the center point of the target intersection is designated as the conflict zone.

[0059] Here, the first distance can be dynamically adjusted according to the actual situation so as to include all unmanned vehicles near the target intersection.

[0060] S114. The area on the intersection map that is at the second distance from the center point of the target intersection is designated as the fixed right-of-way area, where the second distance is less than the first distance.

[0061] Here, the second distance can be set according to the actual situation, but it is necessary to ensure that the second distance is less than the first distance.

[0062] S115. Areas outside the fixed right-of-way zones within the conflict zone shall be designated as areas to be coordinated.

[0063] In this embodiment, the fixed right-of-way areas are removed from the conflict areas on the intersection map corresponding to the target intersection, and the remaining conflict areas are designated as areas to be coordinated. For example, taking a cross intersection as an example, ... Figure 4 As shown, where L 1 L represents the first distance. 2 This indicates the second distance, and the corresponding conflict area on the intersection map is... Figure 4The fixed right-of-way area encompassed by the Grand Circle is: Figure 4 The area to be coordinated is the entire region encompassed by the smaller and medium-sized rings. The area to be coordinated is the region remaining after subtracting the area encompassed by the smaller ring from the area encompassed by the larger ring; that is, the area between the two rings. It should be noted that L for each path... 1 and L 2 It can be set to different values; the specific settings can be adjusted according to specific needs.

[0064] Once the area to be coordinated is identified, the unmanned vehicles to be coordinated can be quickly determined based on that area.

[0065] S12. Based on the state information of multiple unmanned vehicles to be coordinated, generate the state space corresponding to the target intersection.

[0066] In some embodiments of this application, the state information of the unmanned vehicle to be coordinated is information related to its state. The state information of each unmanned vehicle to be coordinated includes, but is not limited to, information such as position, speed, acceleration, attitude, and destination.

[0067] In some embodiments of this application, after identifying multiple unmanned vehicles to be coordinated corresponding to the target intersection, the state information of all unmanned vehicles to be coordinated forms the unmanned vehicle state space to be coordinated.

[0068] S13. Input the state space and the intersection map corresponding to the target intersection into the pre-trained deep learning model to obtain the priority of the right-of-way for multiple unmanned vehicles to be coordinated at the target intersection, as output by the deep learning model.

[0069] In this embodiment, the deep learning model is a pre-trained model used to predict the right-of-way priority of the mining trucks to be coordinated. The deep learning model is trained based on multiple sets of training data. Each set of training data includes a sample state space, a sample intersection map, and label data corresponding to the sample state space. The sample state space is generated based on the states of multiple sample unmanned vehicles corresponding to the sample intersection. The sample intersection map is generated based on the topology of the sample intersection. The label data is used to indicate the right-of-way priority of multiple sample unmanned vehicles at the sample intersection. The sample intersection includes the target intersection and / or intersections belonging to the same scenario or the same category as the target intersection. Based on this, after obtaining the state space corresponding to the target intersection, this state space and the pre-constructed intersection map corresponding to the target intersection can be combined to form the input data. The input data is then fed into the deep learning model to obtain the right-of-way priority of multiple unmanned vehicles to be coordinated, as output by the deep model.

[0070] In some embodiments of this application, to accommodate multimodal input requirements, the state space can be integrated with the intersection location. Figure 1Similarly, an occupancy grid with multiple channels is used. Different positions of the unmanned trucks to be coordinated can correspond to different grid positions, while state information such as velocity, acceleration, and attitude can be placed on different channels. Based on this, before inputting the state space and the intersection map corresponding to the target intersection into the deep learning model, the intersection map and the state space can be tensor-concatenated along the channel dimension to obtain the input data, which is then fed into the deep learning model.

[0071] S14. Based on the priority of the right-of-way of multiple unmanned vehicles to be coordinated at the intersection, control the multiple unmanned vehicles to be coordinated to pass through the intersection in sequence.

[0072] In this embodiment, after obtaining the right-of-way priority of multiple unmanned vehicles to be coordinated at the target intersection, the multiple unmanned vehicles to be coordinated can be coordinated and controlled based on the right-of-way priority, so that the multiple unmanned vehicles to be coordinated can pass through the target intersection safely and quickly.

[0073] In some embodiments of this application, for a target intersection, it can be determined every set period whether all unmanned vehicles to be coordinated in the current period have entered the fixed right-of-way area. If not all unmanned vehicles to be coordinated in the current period have entered the fixed right-of-way area, it is detected whether any new unmanned vehicles have entered the conflict area corresponding to the target intersection. If any new unmanned vehicles have entered the conflict area corresponding to the target intersection, the process returns to step S11 above and coordination control is performed again. In this way, newly entering unmanned vehicles and unmanned vehicles to be coordinated that have not yet entered the fixed right-of-way area can be coordinated in a timely manner, so that all unmanned vehicles in the coordination area can pass safely and quickly, reducing the occurrence of conflicts. For unmanned vehicles that have already entered the fixed right-of-way area, they can be controlled according to the right-of-way last determined before entering the fixed right-of-way area, without having to re-determine their right-of-way priority, reducing the adverse effects of frequently switching right-of-way priorities.

[0074] The unmanned vehicle coordination control method provided in this application, based on the status of multiple unmanned vehicles to be coordinated at a target intersection and the intersection map corresponding to the target intersection, adopts a deep learning-based unmanned vehicle right-of-way coordination technology to optimize the right-of-way priority of multiple unmanned vehicles to be coordinated. Based on the right-of-way priority, the method controls multiple unmanned vehicles to be coordinated, which can realize the orderly passage of multiple unmanned vehicles at intersections with different topological structures. It can also overcome the problem of high computing power requirements of the right-of-way optimization method, thereby meeting the real-time computing requirements for the implementation of the technical solution in engineering, especially for working conditions with a large number of unmanned vehicles to be coordinated.

[0075] In some embodiments, before performing step S13 above, such as Figure 5 As shown, the following steps S51-S54 can be used to train the deep learning model, thereby obtaining a trained deep learning model.

[0076] S51. Obtain multiple sets of sample data. Each set of sample data includes a sample intersection map and a sample state space corresponding to a sample intersection.

[0077] In this embodiment, when acquiring sample data, sample intersections can be selected first, and then the corresponding sample intersection map and sample state space can be obtained. One sample intersection can correspond to multiple sample state spaces, and different sample state spaces can correspond to different times. The method for obtaining the sample state space involves sampling the state information of the unmanned vehicles to be coordinated corresponding to the sample intersection. Sampling methods include, but are not limited to, uniform sampling, random sampling, and Latin hypercube sampling. Then, multiple sets of sample data are constructed according to the requirement that each set of sample data includes one sample intersection map and one sample state space.

[0078] In some embodiments of this application, the trained deep learning model can be deployed using different deployment methods. Based on this, sample intersections can be selected according to the deployment method of the deep learning model. The deployment methods of the deep learning model can include the following three:

[0079] The first deployment method is to train the same deep learning model for all intersections in the same scenario and then deploy the model at all intersections.

[0080] The second deployment method is to cluster all intersections in the same scenario according to their topological structure. For example, cross intersections and X-shaped intersections are grouped into one category. The same deep learning model is trained for the same type of intersections and then deployed in the same type of intersections.

[0081] The third deployment method is to train a deep learning model for each intersection and deploy a deep learning model at each intersection.

[0082] When training the model, you can first choose the required deployment method based on actual needs, and then select the corresponding sample intersections based on the deployment method. For example, when using the first deployment method, you can select multiple intersections belonging to the same scenario as the target intersection as multiple sample intersections; when using the second deployment method, you can select multiple intersections belonging to the same scenario and category as the target intersection as multiple sample intersections; when using the third deployment method, you can use the target intersection as the sample intersection.

[0083] S52. For each set of sample data, based on the sample state space and the preset multi-agent path planning algorithm, with the shortest sum of travel time and the least braking of the unmanned vehicles to be coordinated in the sample as the optimization objective, solve the priority of the right-of-way of each unmanned vehicle to be coordinated in the sample data at the sample intersection.

[0084] In this embodiment, the multi-agent path planning algorithm is a type of multi-vehicle path-trajectory planning algorithm. Based on the initial state of the multiple agents and a given destination, it provides the driving routes and conflict-free trajectories for the agents, and optimizes one or more overall objectives, such as the sum of the convoy's driving time or minimizing the convoy's braking. Its implementation algorithms include, but are not limited to, distributed algorithms and centralized algorithms. Distributed algorithms include, but are not limited to, multi-vehicle motion coordination control algorithms, multi-vehicle heuristic search AlphaGo algorithms, and PAS (Push and Swap) algorithms. Centralized algorithms include, but are not limited to, linear programming, ICTS (Increasing Cost Tree Search) algorithms, and CBS (Collision Based Search) algorithms.

[0085] Because the main characteristic of multi-agent path planning algorithms lies in their potentially high computational complexity, the computation time required can increase exponentially with the increase in the number of unmanned vehicles. Therefore, to address the problems of high complexity and long computation time when using multi-agent path planning algorithms for multi-vehicle coordination, this application provides an embodiment that trains a deep learning model with multi-agent path planning capabilities to plan the right-of-way priorities of multiple vehicles. To enable the deep learning model to possess multi-agent path planning capabilities, the training data is labeled using multi-agent path planning algorithms, i.e., label data is generated in the training data.

[0086] In some embodiments of this application, in order to improve the quality of the constructed training data, for each set of sample data, multiple multi-agent path planning algorithms can be used to solve the trajectory of the unmanned vehicle to be coordinated in the sample data, and finally the trajectory with the best overall optimization objective is selected as the optimal trajectory. After obtaining the optimal trajectory, the timestamp of the unmanned vehicle to be coordinated in the sample data passing through the sample intersection is recorded according to the optimal trajectory, so as to obtain the right-of-way priority of the unmanned vehicle to be coordinated in the sample data.

[0087] In one possible implementation, when using multiple multi-agent path planning algorithms to solve for the trajectories of unmanned vehicles to be coordinated in the sample data, the multiple algorithms can be computed in parallel.

[0088] In another possible implementation, when using multiple multi-agent path planning algorithms to solve the trajectory of the unmanned vehicle to be coordinated in the sample data, the trajectory result of one algorithm can be used as input, and other algorithms can be used for subsequent optimization.

[0089] In another possible implementation, when using multiple multi-agent path planning algorithms to solve the trajectory of the unmanned vehicles to be coordinated in the sample data, the multiple unmanned vehicles to be coordinated in the sample data can be grouped, and then different algorithms can be used to solve the trajectory for different groups.

[0090] S53. Based on multiple sets of sample data and the solved passage priority, construct multiple sets of training data.

[0091] In this embodiment, each set of training data includes sample input data and sample input data label data. The sample input data consists of a sample intersection map and a sample state space from a set of sample data. The label data corresponding to the sample input data is the right-of-way priority corresponding to the set of sample data obtained through the above step S52.

[0092] In some embodiments of this application, both the sample intersection map and the sample state space are represented as a multi-channel occupancy grid. The channels of the sample intersection map are consistent with the channels of the intersection map corresponding to the target intersection, and the channels of the sample state space are consistent with the channels of the state space corresponding to the target intersection. Based on this, similar to combining the state space and intersection map corresponding to the target intersection into input data, the sample intersection map and sample state space in the same set of sample data can be tensor-concatenated along the channel dimension to obtain the sample input data.

[0093] S54. Use multiple sets of training data to iterate the initial deep learning model multiple times until the training termination condition is met, then stop training and obtain the trained deep learning model.

[0094] In this embodiment, after obtaining multiple sets of training data, these sets can be divided into multiple batches, each containing a set number of training data sets. Then, following a supervised training method, the initial deep learning model is iteratively trained multiple times based on the training data from these multiple batches until the training termination condition is met. Training is then stopped, and the initial deep learning model obtained after training stops is taken as the trained deep learning model. The training termination condition can be set according to actual conditions, including but not limited to the initial deep learning model's loss value being less than a set loss threshold, or the number of iterations reaching a preset threshold.

[0095] In some embodiments of this application, the following steps S541-S543 may be performed in each round of training.

[0096] S541. Input the sample state space and sample intersection map from each group of training data corresponding to this round of training into the initial deep learning model to obtain the right-of-way priority prediction value corresponding to each group of training data output by the initial deep learning model. The right-of-way priority prediction value is used to indicate the right-of-way priority of multiple sample unmanned vehicles to be coordinated at the sample intersection in the training data.

[0097] In some embodiments of this application, the initial deep learning model employs an artificial neural network model, which includes an encoder and a decoder. Unlike existing encoder-decoder architectures where the encoder and decoder are composed of the same type of artificial neural network, in this embodiment, the encoder is composed of a deep convolutional neural network, while the decoder is composed of a deep recurrent neural network. The reason for adopting this network architecture is that the deep learning model ultimately trained in this application is used to predict the priority of passage rights for multiple mining trucks to be coordinated. That is, the deep learning model ultimately needs to output a sequence composed of the priority of passage rights for multiple mining trucks to be coordinated; therefore, a network architecture based on a deep recurrent neural network is adopted. Based on this, as... Figure 6 As shown, for each set of training data, the sample state space and sample intersection map in the training data are stitched together to form sample input data, which is then input into the encoder of the initial deep learning model. The deep convolutional neural network in the encoder performs convolutional encoding to obtain an encoding vector, also known as a context vector. The encoding vector is then input into the decoder, where the deep recurrent network in the decoder maps the encoding vector output by the encoder to the right-of-way priority sequence of the sample mining trucks to be coordinated. The right-of-way priority sequence is then output as the right-of-way priority prediction value corresponding to the training data. This right-of-way priority sequence contains the right-of-way priorities of multiple sample unmanned vehicles to be coordinated at the sample intersections in the training data.

[0098] In some embodiments of this application, such as Figure 7As shown, the decoder may include at least one convolutional layer responsible for extracting features from the input data and a pooling layer responsible for reducing the spatial dimension of the convolutional layer output to reduce computation and prevent overfitting. The convolutional layer may include K convolutional kernels, with the value of K set according to the specific situation. Based on this, after the sample input data is input into the encoder of the initial deep learning model, features of the sample input data can be extracted through at least one convolutional layer in the encoder to obtain a feature map. The pooling layer then processes the feature map output by the convolutional layer to obtain the encoded vector.

[0099] In some embodiments of this application, in order to enable the trained deep learning model to be used to predict the right-of-way priority for different numbers of unmanned vehicles to be coordinated, such as... Figure 7 As shown, the decoder may include linear layers, fully connected layers, and normalization layers. The linear layers may include multiple cells, each with a structure including, but not limited to, LSTM (Long Short Term Memory) and GRU (Gate Control Recurrent Unit). The fully connected layers include a mask matrix, where the first N rows are all 1s and the remaining elements are all 0s, where N corresponds to the number of unmanned vehicles to be coordinated input into the model. Since the priority sequences of right-of-way are not continuous, the task of the deep learning model designed in this embodiment belongs to the classification category. Therefore, the output of the fully connected layers is passed to the normalization layer for normalization. The normalization layer may include a SoftMax activation function. Before passing the output of the fully connected layers to the normalization layer, it needs to be masked using a mask vector, where the first N elements are 1s and the remaining elements are 0s. Based on this, after inputting the encoded vector into the decoder, a linear layer is used to map the encoded vector to a traffic priority sequence to obtain an initial traffic priority sequence. This initial traffic priority sequence is then input into a fully connected layer, where a mask matrix in the fully connected layer performs a first masking process to obtain a first mask sequence. This first masking process involves setting all elements in the initial traffic priority sequence except for the first N rows to 0, where N is the number of unmanned vehicles to be coordinated. A second masking process is then performed on each row of the first mask sequence using a mask vector in the fully connected layer to obtain a second mask sequence. This second masking process involves setting all elements in each row except for the first N elements to 0, where N is the number of unmanned vehicles to be coordinated. Finally, the second mask sequence is input into a normalization layer, where the normalization process normalizes the second mask sequence to obtain the traffic priority sequence corresponding to the training data.

[0100] Furthermore, in some embodiments of this application, the encoder may include a graph convolutional neural network (GCN) in addition to a deep convolutional neural network, thereby improving the feature extraction capability of the encoder.

[0101] S542. Based on the label data and the corresponding right-of-way priority prediction value in each set of training data, calculate the value of the loss function to obtain the loss value of the initial deep learning model. The loss function includes the cross-entropy loss function and the focus loss function.

[0102] In this embodiment, since the technical problem solved by deep learning is essentially a classification problem, the cross-entropy loss function is used as the basic form of the loss function for training. Considering the potential class imbalance problem, a focal loss function is further introduced on the basis of the cross-entropy loss function.

[0103] S543. If the training termination condition is not met, adjust the parameters of the initial deep learning model based on the loss value and proceed with the next round of training.

[0104] In this embodiment, an optimization solver can be used to adjust the parameters of the initial deep learning model. The optimization solver includes, but is not limited to, Adam (short for Adaptive Moment Estimation).

[0105] In some embodiments of this application, two training modes can be used to train the initial deep model: Auto Regressive Mode and Teaching Forcing Mode. Auto Regressive Mode uses the predicted output of the decoder at the previous sequence position as the input for the prediction of the next sequence position. Teaching Forcing Mode uses the true value of the previous sequence position as the input for the prediction of the current sequence position. Using both training models combines the advantages of both, improving model training stability and accelerating model convergence.

[0106] The deep learning model trained in the above manner integrates multimodal information into its input, namely the state space and the intersection map, and its output has great flexibility, adapting to the complex environment of the mine and the dynamic needs of coordinating different numbers of unmanned mining trucks under different conditions.

[0107] Once the trained deep learning model is obtained, it can be deployed according to the selected deployment method. Then, the deployed deep learning model can be used to predict the right-of-way priority of multiple unmanned vehicles to be coordinated at the target intersection.

[0108] In some embodiments, the network structure of the trained deep learning model is consistent with the network structure of the initial deep model. The deep learning model may include an encoder and a decoder, where the encoder includes a deep convolutional neural network and the decoder includes a deep recurrent neural network. Based on this, the state space and intersection map corresponding to the target intersection are input into the pre-trained deep learning model to obtain the priority of the right-of-way for multiple unmanned vehicles to be coordinated at the target intersection, as output by the deep learning model, including:

[0109] The state space and the intersection map are stitched together to obtain the input data;

[0110] Input data is fed into the encoder, and the deep convolutional neural network in the encoder performs convolutional encoding on the input data to obtain the encoded vector;

[0111] The encoded vector is input into the decoder, and the deep recurrent neural network in the decoder maps the encoded vector into a right-of-way priority sequence of multiple unmanned vehicles to be coordinated. The right-of-way priority sequence includes the right-of-way priority of each of the multiple unmanned vehicles to be coordinated at the target intersection.

[0112] Furthermore, the decoder may include linear layers, fully connected layers, and normalization layers, with the fully connected layers comprising a mask matrix and a mask vector. Based on this, mapping the encoded vectors to a priority sequence of right-of-way for multiple unmanned vehicles to be coordinated via a deep recurrent neural network in the decoder can include:

[0113] By using a linear layer, the encoded vector is mapped to the passage priority sequence to obtain the initial passage priority sequence;

[0114] The initial right-of-way priority sequence is input into the fully connected layer. The initial right-of-way priority sequence is then subjected to a first masking process through the mask matrix in the fully connected layer to obtain the first masking sequence. The first masking process includes setting all elements in the initial right-of-way priority sequence except for the first N rows to 0, where N is the number of unmanned vehicles to be coordinated.

[0115] The second mask sequence is obtained by performing a second masking process on each row of the first mask sequence using the mask vector in the fully connected layer. The second masking process includes setting all elements except the first N elements in each row to 0, where N is the number of unmanned vehicles to be coordinated.

[0116] The second mask sequence is input into the normalization layer, and the normalization layer normalizes the second mask sequence to obtain the right-of-way priority sequence of multiple unmanned vehicles to be coordinated.

[0117] Based on the unmanned vehicle coordinated control method provided in the above embodiments, this application also provides specific implementation methods of the unmanned vehicle coordinated control device. Please refer to the following embodiments.

[0118] See Figure 8 This is a schematic diagram of the unmanned vehicle coordination and control device provided in the embodiments of this application, as shown below. Figure 8 As shown, the device 800 includes the following modules:

[0119] Module 801 is used to determine multiple unmanned vehicles to be coordinated at the target intersection.

[0120] The state space generation module 802 is used to generate the state space corresponding to the target intersection based on the state information of multiple unmanned vehicles to be coordinated.

[0121] The right-of-way prediction module 803 is used to input the state space and intersection map corresponding to the target intersection into a pre-trained deep learning model to obtain the right-of-way priority of multiple unmanned vehicles to be coordinated at the target intersection, as output by the deep learning model.

[0122] The control module 804 is used to control multiple unmanned vehicles to be coordinated to pass through the target intersection in sequence based on the priority of the right-of-way of multiple unmanned vehicles to be coordinated at the intersection.

[0123] The intersection map is generated based on the topology of the target intersection;

[0124] The deep learning model is trained based on multiple sets of training data. Each set of training data includes a sample state space, a sample intersection map, and label data corresponding to the sample state space. The sample state space is generated based on the states of multiple sample unmanned vehicles corresponding to the sample intersection. The sample intersection map is generated based on the topology of the sample intersection. The label data is used to indicate the priority of the right-of-way of multiple sample unmanned vehicles at the sample intersection.

[0125] The unmanned vehicle coordination and control device of this application embodiment, based on the status of multiple unmanned vehicles to be coordinated at the target intersection and the intersection map corresponding to the target intersection, adopts unmanned vehicle right-of-way coordination technology based on deep learning to optimize the right-of-way priority of multiple unmanned vehicles to be coordinated, and controls multiple unmanned vehicles to be coordinated based on the right-of-way priority. It can realize the orderly passage of multiple unmanned vehicles at intersections with different topological structures in the mine, and can overcome the problem of high computing power requirements of the right-of-way optimization method, thereby meeting the real-time computing requirements for the implementation of technical solutions in engineering, especially for working conditions with a large number of unmanned vehicles to be coordinated.

[0126] In some embodiments, the determining module 801 is configured to:

[0127] Unmanned vehicles located in the area to be coordinated corresponding to the target intersection and moving towards the center point of the target intersection are considered as unmanned vehicles to be coordinated corresponding to the target intersection. The area to be coordinated is the area where unmanned vehicles need to be coordinated and controlled.

[0128] In some embodiments, the apparatus 800 further includes: a region determination module, configured to:

[0129] Obtain the topology of the target intersection;

[0130] Based on the topology, generate an intersection map corresponding to the target intersection;

[0131] The area on the intersection map that is the first distance from the center point of the target intersection is designated as the conflict zone;

[0132] The area on the intersection map that is second in distance from the center point of the target intersection is designated as the fixed right-of-way area, where the second distance is less than the first distance.

[0133] Areas outside of the fixed right-of-way zones within the conflict zone will be designated as areas to be coordinated.

[0134] In some embodiments, the deep learning model includes an encoder and a decoder, the encoder including a deep convolutional neural network and the decoder including a deep recurrent neural network;

[0135] Right-of-way prediction module 803 is used for:

[0136] The state space and the intersection map are stitched together to obtain the input data;

[0137] Input data is fed into the encoder, and the deep convolutional neural network in the encoder performs convolutional encoding on the input data to obtain the encoded vector;

[0138] The encoded vector is input into the decoder, and the deep recurrent neural network in the decoder maps the encoded vector into a right-of-way priority sequence of multiple unmanned vehicles to be coordinated. The right-of-way priority sequence includes the right-of-way priority of each of the multiple unmanned vehicles to be coordinated at the target intersection.

[0139] In some embodiments, the decoder includes a linear layer, a fully connected layer, and a normalized layer, wherein the fully connected layer includes a mask matrix and a mask vector;

[0140] Right-of-way prediction module 803 is used for:

[0141] By using a linear layer, the encoded vector is mapped to the passage priority sequence to obtain the initial passage priority sequence;

[0142] The initial right-of-way priority sequence is input into the fully connected layer. The initial right-of-way priority sequence is then subjected to a first masking process through the mask matrix in the fully connected layer to obtain the first masking sequence. The first masking process includes setting all elements in the initial right-of-way priority sequence except for the first N rows to 0, where N is the number of unmanned vehicles to be coordinated.

[0143] The second mask sequence is obtained by performing a second masking process on each row of the first mask sequence using the mask vector in the fully connected layer. The second masking process includes setting all elements except the first N elements in each row to 0, where N is the number of unmanned vehicles to be coordinated.

[0144] The second mask sequence is input into the normalization layer, and the normalization layer normalizes the second mask sequence to obtain the right-of-way priority sequence of multiple unmanned vehicles to be coordinated.

[0145] In some embodiments, the apparatus 800 further includes: a training module, configured to:

[0146] Acquire multiple sets of sample data. Each set of sample data includes a sample intersection map and a sample state space corresponding to a sample intersection. The sample intersections include the target intersection and / or intersections that belong to the same scene or the same category as the target intersection.

[0147] For each set of sample data, based on the sample state space and the preset multi-agent path planning algorithm, with the shortest sum of travel time and the least braking of the unmanned vehicles to be coordinated in the sample as the optimization objective, the priority of the right-of-way of the unmanned vehicles to be coordinated in the sample data at the sample intersection is solved.

[0148] Multiple sets of training data are constructed based on multiple sets of sample data and the solved passage priority.

[0149] The initial deep learning model is trained iteratively through multiple sets of training data until the training termination condition is met, at which point the training is stopped and a well-trained deep learning model is obtained.

[0150] In some embodiments, the training module is used for:

[0151] Perform the following steps in each round of training:

[0152] Input the sample state space and sample intersection map of each training data set in this round of training into the initial deep learning model to obtain the right-of-way priority prediction value of each training data set output by the initial deep learning model. The right-of-way priority prediction value is used to indicate the right-of-way priority of multiple unmanned vehicles to be coordinated in the training data at the sample intersection.

[0153] Based on the label data and the corresponding right-of-way priority prediction value in each set of training data, the value of the loss function is calculated to obtain the loss value of the initial deep learning model. The loss function includes the cross-entropy loss function and the focus loss function.

[0154] If the training termination condition is not met, adjust the parameters of the initial deep learning model based on the loss value and proceed to the next round of training.

[0155] The unmanned vehicle coordination and control device provided in this application embodiment can achieve... Figures 1 to 7 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0156] Figure 9 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0157] Electronic device 900 may include processor 901 and memory 902 storing computer program instructions.

[0158] Specifically, the processor 901 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0159] Memory 902 may include a large-capacity memory for data or instructions. For example, and not limitingly, memory 902 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 902 may include removable or non-removable (or fixed) media. Where appropriate, memory 902 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 902 is non-volatile solid-state memory. Memory 902 may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, electrical, optical, or other physical / tangible memory storage devices. Thus, typically, memory 902 includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it can perform the operations described in any of the unmanned vehicle coordinated control methods in the above embodiments.

[0160] The processor 901 reads and executes computer program instructions stored in the memory 902 to implement any of the unmanned vehicle coordinated control methods in the above embodiments.

[0161] In one example, the electronic device 900 may also include a communication interface 903 and a bus 910. For example, Figure 9 As shown, the processor 901, memory 902, and communication interface 903 are connected through bus 910 and complete communication with each other.

[0162] The communication interface 903 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0163] Bus 910 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 910 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0164] Furthermore, in conjunction with the unmanned vehicle coordination and control method in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the unmanned vehicle coordination and control methods in the above embodiments.

[0165] This application also provides a computer program product, including a computer program that, when executed, implements any of the unmanned vehicle coordinated control methods described in the above embodiments.

[0166] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0167] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0168] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0169] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0170] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for coordinated control of unmanned vehicles, characterized in that, include: Identify the multiple unmanned vehicles to be coordinated at the target intersection; Based on the state information of the multiple unmanned vehicles to be coordinated, a state space corresponding to the target intersection is generated. Input the state space and intersection map corresponding to the target intersection into a pre-trained deep learning model to obtain the right-of-way priority of the multiple unmanned vehicles to be coordinated at the target intersection, as output by the deep learning model. Based on the priority of the right-of-way of the multiple unmanned vehicles to be coordinated at the intersection, the multiple unmanned vehicles to be coordinated are controlled to pass through the target intersection in sequence; The intersection map is generated based on the topology of the target intersection; The deep learning model is trained based on multiple sets of training data. Each set of training data includes a sample state space, a sample intersection map, and label data corresponding to the sample state space. The sample state space is generated based on the states of multiple sample unmanned vehicles corresponding to the sample intersection. The sample intersection map is generated based on the topology of the sample intersection. The label data is used to indicate the priority of the right-of-way of the multiple sample unmanned vehicles at the sample intersection.

2. The method according to claim 1, characterized in that, The multiple unmanned vehicles to be coordinated corresponding to the target intersection include: An unmanned vehicle located in the area to be coordinated corresponding to the target intersection and traveling towards the center point of the target intersection is considered as an unmanned vehicle to be coordinated corresponding to the target intersection. The area to be coordinated is the area where the unmanned vehicle needs to be coordinated and controlled.

3. The method according to claim 2, characterized in that, Before determining the multiple unmanned vehicles to be coordinated corresponding to the target intersection, the method further includes: Obtain the topology of the target intersection; Based on the aforementioned topology, an intersection map corresponding to the target intersection is generated; The area in the intersection map that is a first distance away from the center point of the target intersection is designated as the conflict zone; The area in the intersection map that is a second distance away from the center point of the target intersection is designated as a fixed right-of-way area, where the second distance is less than the first distance. The areas outside the fixed right-of-way areas within the conflict zones are designated as areas to be coordinated.

4. The method according to claim 1, characterized in that, The deep learning model includes an encoder and a decoder, wherein the encoder includes a deep convolutional neural network and the decoder includes a deep recurrent neural network; The step of inputting the state space and intersection map corresponding to the target intersection into a pre-trained deep learning model to obtain the right-of-way priority of the multiple unmanned vehicles to be coordinated at the target intersection, as output by the deep learning model, includes: The state space and the intersection map are stitched together to obtain the input data; The input data is input into the encoder, and the input data is convolutionally encoded by the deep convolutional neural network in the encoder to obtain the encoded vector. The encoded vector is input into the decoder, and the deep recurrent neural network in the decoder maps the encoded vector into a right-of-way priority sequence of the multiple unmanned vehicles to be coordinated. The right-of-way priority sequence includes the right-of-way priority of each of the multiple unmanned vehicles to be coordinated at the target intersection.

5. The method according to claim 4, characterized in that, The decoder includes a linear layer, a fully connected layer, and a normalized layer. The fully connected layer includes a mask matrix and a mask vector. The step of mapping the encoded vector to a priority sequence of right-of-way for the multiple unmanned vehicles to be coordinated via a deep recurrent neural network in the decoder includes: The encoding vector is mapped to the passage priority sequence through the linear layer to obtain the initial passage priority sequence; The initial right-of-way priority sequence is input into the fully connected layer, and the initial right-of-way priority sequence is subjected to a first masking process through the masking matrix in the fully connected layer to obtain a first masking sequence. The first masking process includes setting all elements in the initial right-of-way priority sequence except for the first N rows to 0, where N is the number of the multiple unmanned vehicles to be coordinated. The mask vector in the fully connected layer is used to perform a second masking process on each row of the first masking sequence to obtain a second masking sequence. The second masking process includes setting all elements in each row except the first N elements to 0, where N is the number of the multiple unmanned vehicles to be coordinated. The second mask sequence is input into the normalization layer, and the normalization layer performs normalization processing on the second mask sequence to obtain the right-of-way priority sequence of the multiple unmanned vehicles to be coordinated.

6. The method according to any one of claims 1-5, characterized in that, Before inputting the state space and intersection map into a pre-trained deep learning model to obtain the right-of-way priority of the multiple unmanned vehicles to be coordinated at the intersection, as output by the deep learning model, the method further includes: Multiple sets of sample data are acquired. Each set of sample data includes a sample intersection map and a sample state space corresponding to a sample intersection. The sample intersections include the target intersection and / or intersections that belong to the same scene or the same category as the target intersection. For each set of sample data, based on the sample state space and the preset multi-agent path planning algorithm, with the shortest sum of travel time and the least braking of the unmanned vehicles to be coordinated in the sample as the optimization objective, the priority of the right-of-way of the unmanned vehicles to be coordinated in the sample data at the sample intersection is solved. Based on the multiple sets of sample data and the solved passage priority, multiple sets of training data are constructed; The initial deep learning model is trained iteratively using the multiple sets of training data until the training termination condition is met, at which point training is stopped, and a well-trained deep learning model is obtained.

7. The method according to claim 6, characterized in that, The step of using the multiple sets of training data to perform multiple rounds of iterative training on the initial deep learning model includes: Perform the following steps in each round of training: The sample state space and sample intersection map in each group of training data corresponding to this round of training are respectively input into the initial deep learning model to obtain the right-of-way priority prediction value corresponding to each group of training data output by the initial deep learning model. The right-of-way priority prediction value is used to indicate the right-of-way priority of multiple sample unmanned vehicles to be coordinated in the training data at the sample intersection. Based on the label data and the corresponding right-of-way priority prediction value in each set of training data, the value of the loss function is calculated to obtain the loss value of the initial deep learning model. The loss function includes the cross-entropy loss function and the focus loss function. If the training termination condition is not met, the parameters of the initial deep learning model are adjusted based on the loss value, and the next round of training is performed.

8. A coordinated control device for unmanned vehicles, characterized in that, include: The determination module is used to identify multiple unmanned vehicles to be coordinated at the target intersection; The state space generation module is used to generate the state space corresponding to the target intersection based on the state information of the multiple unmanned vehicles to be coordinated. The right-of-way prediction module is used to input the state space and intersection map corresponding to the target intersection into a pre-trained deep learning model to obtain the right-of-way priority of the multiple unmanned vehicles to be coordinated at the target intersection, as output by the deep learning model. The control module is used to control the multiple unmanned vehicles to be coordinated to pass through the target intersection in sequence based on the priority of the right-of-way of the multiple unmanned vehicles to be coordinated at the intersection. The intersection map is generated based on the topology of the target intersection; The deep learning model is trained based on multiple sets of training data. Each set of training data includes a sample state space, a sample intersection map, and label data corresponding to the sample state space. The sample state space is generated based on the states of multiple sample unmanned vehicles corresponding to the sample intersection. The sample intersection map is generated based on the topology of the sample intersection. The label data is used to indicate the priority of the right-of-way of the multiple sample unmanned vehicles at the sample intersection.

9. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the unmanned vehicle coordinated control method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the unmanned vehicle coordinated control method as described in any one of claims 1-7.

11. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device causes the electronic device to perform the unmanned vehicle coordinated control method as described in any one of claims 1-7.

Images