An unmanned intelligent scheduling method and system based on multi-source data

Abstract

The application discloses a kind of unmanned intelligent scheduling method and system based on multi-source data, the method includes: obtaining the environment information around unmanned vehicle and traffic infrastructure information and carries out multi-source data fusion operation, obtains environment fusion data;According to environment fusion data, SLAM synchronous positioning and map construction operation are carried out, and map and positioning data are obtained;According to map and positioning data, fast expanding random tree connection operation is carried out, and planning path data are obtained;According to planning path data and environment fusion data, traffic condition prediction model construction operation is carried out, and traffic condition prediction data are obtained;According to traffic condition prediction data, decision model construction operation is carried out, and decision strategy data are obtained;According to decision strategy data, real-time decision operation is carried out, and adjusted driving instruction data are obtained, to make vehicle carry out path adjustment and travel optimization.This method can realize accurate prediction of traffic conditions and generate optimal path.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned driving, and particularly relates to an unmanned intelligent scheduling method and system based on multi-source data. BACKGROUND

[0002] With the rapid development of unmanned driving technology, autonomous driving systems have gradually become a research hotspot in the field of transportation. Existing autonomous driving systems usually rely on sensor data (such as cameras, lidar, millimeter wave radar, etc.) to perceive the surrounding environment, and control and plan the vehicle path through pre-set models. However, these sensor data-based models have certain limitations in complex and dynamic traffic environments. For example, sensor data may not accurately predict the behavior of other vehicles, increasing the risk of vehicle collisions.

[0003] In the prior art, intelligent scheduling for unmanned vehicles mainly relies on sensor data-based models or semi-automatic path planning algorithms. Sensor data-based models mainly perceive the surrounding environment through cameras, lidar, and other devices mounted on vehicles, and plan paths and make decisions according to pre-set rules. However, this scheduling method is affected by the limitations of sensor data, making it difficult to accurately predict the behavior of other vehicles, increasing the risk of vehicle collisions. With the development of unmanned driving technology, some companies have introduced scheduling systems based on road network information, integrating road network data and vehicle state information for dynamic scheduling. However, due to the complexity and variability of traffic environments, the adaptability and decision-making ability of existing scheduling systems are often insufficient, making it difficult to respond to unexpected situations and complex traffic rules.

[0004] In summary, existing scheduling technologies are susceptible to the limitations of sensor data and environmental complexity, making it difficult to accurately predict traffic conditions and generate optimal paths, resulting in insufficient adaptability and decision-making ability of unmanned vehicles in complex traffic environments. SUMMARY

[0005] The present application provides an unmanned intelligent scheduling method and system based on multi-source data to accurately predict traffic conditions and generate optimal paths.

[0006] In a first aspect, to solve the above technical problems, the present application provides an unmanned intelligent scheduling method based on multi-source data, comprising:

[0007] Obtaining surrounding environment information and traffic infrastructure information of an unmanned vehicle;

[0008] Performing multi-source data fusion operation according to the surrounding environment information and the traffic infrastructure information of the unmanned vehicle to obtain environment fusion data;

[0009] According to the environment fusion data, SLAM simultaneous localization and mapping operation is performed to obtain map and positioning data;

[0010] According to the map and the positioning data, a rapid extension random tree connection operation is performed to obtain planning path data;

[0011] According to the planning path data and the environment fusion data, a traffic condition prediction model construction operation is performed to obtain traffic condition prediction data;

[0012] According to the traffic condition prediction data, a decision model construction operation is performed to obtain decision strategy data;

[0013] According to the decision strategy data, a real-time decision operation is performed to obtain adjusted driving instruction data, so as to make the vehicle adjust the path and optimize the driving.

[0014] As an optional implementation, the matching detection on the to-be-detected image comprises:

[0015] According to the surrounding environment information of the unmanned vehicle and the traffic infrastructure information, a multi-source data fusion operation is performed to obtain environment fusion data, comprising:

[0016] According to the surrounding environment information of the unmanned vehicle and the traffic infrastructure information, data preprocessing is performed to obtain clean multi-source environment data;

[0017] According to the clean multi-source environment data, a time stamp synchronization operation is performed to obtain multi-source environment synchronization data;

[0018] According to the clean multi-source environment data, a spatial registration operation is performed to obtain multi-source environment registration data;

[0019] According to the multi-source environment synchronization data and the multi-source environment registration data, data fusion is performed to obtain environment fusion data.

[0020] As an optional implementation, according to the environment fusion data, SLAM simultaneous localization and mapping operation is performed to obtain map and positioning data, comprising:

[0021] According to the environment fusion data, a map construction operation is performed to obtain initial map data;

[0022] According to the initial map data and the environment fusion data, a positioning calculation operation is performed to obtain vehicle positioning data;

[0023] According to the vehicle positioning data and the initial map data, a map optimization operation is performed to obtain map optimization data;

[0024] According to the map optimization data and the vehicle positioning data, a semantic information integration operation is performed to obtain map and positioning data containing geometric and semantic information.

[0025] As an optional implementation, according to the map and the positioning data, a rapid expansion random tree connection operation is performed to obtain planning path data, including:

[0026] According to the map and the positioning data, a start point and target point setting operation is performed to obtain path planning start point and target point data;

[0027] According to the path planning start point and target point data, a random tree initialization operation is performed to obtain initial random tree data;

[0028] According to the initial random tree data, a random sampling operation is performed to obtain random sampling point coordinates;

[0029] According to the random sampling point coordinates, a nearest node searching operation is performed to obtain nearest node coordinates;

[0030] According to the nearest node coordinates and the random sampling point coordinates, a new node generation calculation is performed to obtain new node coordinates;

[0031] According to the new node coordinates, a collision detection operation is performed to obtain collision-free path segment data;

[0032] According to the collision-free path segment data, a random tree expansion operation is performed to obtain expanded random tree data;

[0033] According to the expanded random tree data, a target point connection operation is performed to obtain planning path data As an optional implementation, the calculation formula of the new node generation calculation is as follows:

[0034]

[0035] wherein, x new represents the generated new node coordinates, x near represents the nearest node coordinates, x rand represents the random sampling point coordinates, d step represents a preset step value, ||x rand -x near || represents the Euclidean distance between the random sampling point coordinates and the nearest node coordinates.

[0036] As an optional implementation, according to the planning path data and the environment fusion data, a traffic condition prediction model construction operation is performed to obtain traffic condition prediction data, including:

[0037] According to the planning path data and the environment fusion data, a historical traffic data extraction operation is performed to obtain historical traffic data;

[0038] According to the historical traffic data, a time series analysis operation is performed to obtain traffic trend data;

[0039] According to the traffic trend data, a neural network model training operation is performed to obtain a traffic prediction model;

[0040] According to the traffic prediction model and real-time environment fusion data, a traffic condition prediction operation is performed to obtain traffic condition prediction data.

[0041] As an optional implementation, the time series analysis operation according to the historical traffic data to obtain traffic trend data includes:

[0042] According to the historical traffic data, a graph attention network model construction operation is performed to obtain traffic graph structure data;

[0043] According to the traffic graph structure data, a node feature extraction operation is performed to obtain node feature data;

[0044] According to the node feature data, an attention mechanism calculation operation is performed to obtain node attention weight data;

[0045] According to the node attention weight data, a time series prediction operation is performed to obtain traffic trend data;

[0046] The calculation formula of the graph attention network model is as follows:

[0047]

[0048] Where h′ i represents the updated feature of node i, σ represents an activation function, represents the neighbor node set of node i, α ij represents the attention weight between node i and node j, W represents a learnable weight matrix, h j represents the feature of node j.

[0049] As an optional implementation, the decision model construction operation according to the traffic condition prediction data to obtain decision strategy data includes:

[0050] According to the traffic condition prediction data, a trajectory prediction model construction operation is performed to obtain trajectory prediction data of an unmanned vehicle;

[0051] According to the trajectory prediction data, an optimization function calculation operation is performed to obtain optimal trajectory data;

[0052] According to the optimal trajectory data, a decision strategy generation operation is performed to obtain decision strategy data.

[0053] As an optional implementation, the calculation formula of the optimization function calculation is as follows:

[0054]

[0055] wherein x represents an optimal trajectory of the unmanned vehicle, P represents a covariance matrix of the unmanned vehicle trajectory, q represents a parameter vector of the unmanned vehicle collision avoidance, represents minimizing the objective function J with respect to the variable x, and T represents the transpose of the vector.

[0056] In a second aspect, the present application provides an unmanned intelligent scheduling system based on multi-source data, comprising:

[0057] A data acquisition module is configured to acquire surrounding environment information and traffic infrastructure information of the unmanned vehicle.

[0058] A data fusion module is configured to perform multi-source data fusion operation according to the surrounding environment information and the traffic infrastructure information of the unmanned vehicle to obtain environment fusion data.

[0059] A SLAM module is configured to perform SLAM simultaneous localization and mapping operation according to the environment fusion data to obtain map and positioning data.

[0060] A path planning module is configured to perform rapid expansion random tree connection operation according to the map and the positioning data to obtain planning path data.

[0061] A traffic condition prediction module is configured to perform traffic condition prediction model construction operation according to the planning path data and the environment fusion data to obtain traffic condition prediction data.

[0062] A decision module is configured to perform decision model construction operation according to the traffic condition prediction data to obtain decision strategy data.

[0063] A path optimization module is configured to perform real-time decision operation according to the decision strategy data to obtain adjusted driving instruction data, so as to make the vehicle perform path adjustment and driving optimization.

[0064] Compared with the prior art, the present application has the following beneficial effects:

[0065] The application provides an unmanned intelligent scheduling method based on multi-source data, comprising: acquiring surrounding environment information and traffic infrastructure information of an unmanned vehicle; performing multi-source data fusion operation according to the surrounding environment information and the traffic infrastructure information of the unmanned vehicle to obtain environment fusion data; performing SLAM (Simultaneous Localization and Mapping) synchronous positioning and map construction operation according to the environment fusion data to obtain a map and positioning data; performing rapid extended random tree connection operation according to the map and the positioning data to obtain planning path data; performing traffic condition prediction model construction operation according to the planning path data and the environment fusion data to obtain traffic condition prediction data; performing decision model construction operation according to the traffic condition prediction data to obtain decision strategy data; and performing real-time decision operation according to the decision strategy data to obtain adjusted driving instruction data, so as to make the vehicle adjust the path and optimize the driving.

[0066] In the application, the surrounding environment information and the traffic infrastructure information of the unmanned vehicle are integrated through multi-source data fusion technology, then the unmanned system generates a map and positioning data by performing SLAM synchronous positioning and map construction operation according to the integrated environment data, and generates planning path data by performing rapid extended random tree connection algorithm according to the map and the positioning data. Then, the system constructs a traffic condition prediction model according to the planning path data and the environment fusion data, predicts the future traffic condition change, and generates decision strategy data according to the prediction result. Finally, the system adjusts the driving path and operation strategy of the vehicle in real time according to the decision strategy data, to ensure the safe driving of the vehicle in the complex traffic environment. The traffic condition prediction model is constructed based on historical traffic data and real-time environment data, can more accurately reflect the change trend of the traffic environment, and can significantly improve the adaptability and decision-making ability of the unmanned vehicle through the fusion and analysis of multi-source data. In the path planning and decision-making process, the system simultaneously considers the traffic infrastructure information and the change trend of the environment, can accurately predict the traffic condition and generate the optimal path. BRIEF DESCRIPTION OF DRAWINGS

[0067] Figure 1 It is a flowchart of an unmanned intelligent scheduling method based on multi-source data provided by an embodiment of the application.

[0068] Figure 2 It is a structural schematic diagram of an unmanned intelligent scheduling system based on multi-source data provided by an embodiment of the application. DETAILED DESCRIPTION

[0069] With the rapid development of unmanned technology, autonomous driving systems have gradually become a research hotspot in the field of transportation. Existing autonomous driving systems usually rely on sensor data (such as cameras, lidar, millimeter wave radar, etc.) to perceive the surrounding environment, and through pre-set models for vehicle control and path planning. However, these sensor data-based models have certain limitations in complex and dynamic traffic environments. For example, sensor data may not accurately predict the behavior of other vehicles, increasing the risk of vehicle collisions.

[0070] In the prior art, intelligent scheduling for unmanned vehicles mainly relies on sensor data-based models or semi-automatic path planning algorithms. Sensor data-based models mainly perceive the surrounding environment through cameras, lidar, and other devices mounted on vehicles, and make path planning and decision-making according to pre-set rules. However, this scheduling method is affected by the limitations of sensor data, making it difficult to accurately predict the behavior of other vehicles, increasing the risk of vehicle collisions. With the development of unmanned technology, some companies have introduced scheduling systems based on road network information, integrating road network data and vehicle state information for dynamic scheduling. However, due to the complexity and variability of the traffic environment, the adaptability and decision-making ability of existing scheduling systems are often insufficient, making it difficult to respond to unexpected situations and complex traffic rules.

[0071] In summary, existing scheduling technologies are susceptible to the limitations of sensor data and environmental complexity, making it difficult to accurately predict traffic conditions and generate optimal paths, resulting in insufficient adaptability and decision-making ability of unmanned vehicles in complex traffic environments.

[0072] To address the problems in the prior art, the present application provides a multi-source data-based intelligent scheduling method and system for unmanned vehicles to accurately predict traffic conditions and generate optimal paths.

[0073] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0074] Referring to Figure 1 , the first embodiment of the present application provides a multi-source data-based intelligent scheduling method for unmanned vehicles, including the following steps:

[0075] S11, obtaining surrounding environment information and traffic infrastructure information of the unmanned vehicle;

[0076] S12, performing multi-source data fusion operation according to the surrounding environment information of the unmanned vehicle and the traffic infrastructure information, to obtain environment fusion data;

[0077] S13, performing SLAM synchronous positioning and map construction operation according to the environment fusion data, to obtain map and positioning data;

[0078] S14, performing rapid extended random tree connection operation according to the map and the positioning data, to obtain planning path data;

[0079] S15, performing traffic condition prediction model construction operation according to the planning path data and the environment fusion data, to obtain traffic condition prediction data;

[0080] S16, performing decision model construction operation according to the traffic condition prediction data, to obtain decision strategy data;

[0081] S17, performing real-time decision operation according to the decision strategy data, to obtain adjusted driving instruction data, so as to make the vehicle adjust the path and optimize the driving.

[0082] In step S11, the surrounding environment information of the unmanned vehicle and the traffic infrastructure information are obtained.

[0083] It should be noted that the surrounding environment information of the unmanned vehicle includes but is not limited to the following information: the data collected by the sensors (such as cameras, laser radars, millimeter wave radars, ultrasonic sensors, etc.) installed on the vehicle, to obtain the position, speed, size and type information of other vehicles, pedestrians and obstacles around the vehicle. In addition, the traffic infrastructure information includes but is not limited to the following information: through the communication between the vehicle and the traffic infrastructure (such as traffic signal lights, roadside units, etc.), to obtain the state of the traffic signal lights (such as red light, green light, yellow light), road speed limit information, lane information, traffic event information (such as accident, construction, etc.) and the geometric information of the slope and curvature of the road. Through the acquisition of multi-source data, the system can comprehensively and accurately perceive the environmental state and traffic rules around the vehicle, and provide reliable data support for subsequent path planning, traffic condition prediction and decision making. The fusion and analysis of multi-source data can effectively improve the adaptability of unmanned vehicles in complex and dynamic traffic environment, and ensure the safe driving and efficient scheduling of vehicles.

[0084] In step S12, multi-source data fusion operation is performed according to the surrounding environment information of the unmanned vehicle and the traffic infrastructure information, to obtain environment fusion data, including:

[0085] According to the surrounding environment information of the unmanned vehicle and the traffic infrastructure information, data preprocessing is performed to obtain clean multi-source environment data;

[0086] According to the clean multi-source environment data, a timestamp synchronization operation is performed to obtain multi-source environment synchronization data;

[0087] According to the clean multi-source environment data, a spatial registration operation is performed to obtain multi-source environment registration data;

[0088] According to the multi-source environment synchronization data and the multi-source environment registration data, data fusion is performed to obtain environment fusion data.

[0089] It should be noted that the multi-source data fusion operation refers to integrating data from different sensors and infrastructures through a series of data processing steps to generate a unified environment identifier while ensuring data integrity and consistency. This operation minimizes information loss during data integration and ensures that important features of the original data are retained. In the embodiments of the present application, the multi-source data fusion operation adopts a multi-sensor fusion algorithm (such as Kalman filtering, particle filtering or deep learning model) to achieve data preprocessing, timestamp synchronization, spatial registration and data fusion. Data preprocessing refers to the preliminary processing of raw data from different sensors and infrastructures to eliminate noise, correct errors and handle outliers. In the embodiments of the present application, data preprocessing includes denoising and correcting camera images, filtering laser radar point clouds, calibrating radar data, etc. Timestamp synchronization refers to aligning data from different sensors and infrastructures according to a unified time reference. Since the data collection frequency and timestamp of different sensors may differ, timestamp synchronization ensures the consistency of data in time. In the embodiments of the present application, different sensors are synchronized to the same time point through interpolation or time alignment algorithm, ensuring the matching of multi-source data in the time dimension. Spatial registration refers to converting data from different sensors to a unified coordinate system (such as vehicle coordinate system or world coordinate system) to ensure the consistency of data in space. In the embodiments of the present application, the data of cameras, laser radars and radars are registered to the same spatial coordinate system through coordinate transformation and geometric correction, ensuring the matching of multi-source data in the spatial dimension. Data fusion refers to integrating multi-source data that has been preprocessed, timestamp synchronized and spatially registered to generate unified environment fusion data. In the embodiments of the present application, Kalman filtering fusion algorithm is used to integrate multi-source data into a comprehensive environment representation. Of course, according to different actual application scenarios and user requirements, the fusion algorithm can also use particle filtering or deep learning model, etc. The present application does not limit this.

[0090] In the unmanned intelligent scheduling, using multi-source data fusion operation can effectively integrate complex multi-source environment information (such as sensor data, traffic infrastructure data, etc.) into unified environment fusion data to improve the accuracy and comprehensiveness of environment perception, and can be used for path planning, traffic condition prediction and decision generation when needed, so as to make more accurate vehicle scheduling and path optimization. This operation ensures the efficiency and accuracy of data processing, significantly improves the adaptability and decision-making ability of unmanned vehicles in complex traffic environment.

[0091] In step S13, according to the environment fusion data, SLAM simultaneous localization and mapping operation is performed to obtain map and positioning data, including:

[0092] According to the environment fusion data, a map construction operation is performed to obtain initial map data;

[0093] According to the initial map data and the environment fusion data, a positioning calculation operation is performed to obtain vehicle positioning data;

[0094] According to the vehicle positioning data and the initial map data, a map optimization operation is performed to obtain map optimization data;

[0095] According to the map optimization data and the vehicle positioning data, a semantic information integration operation is performed to obtain a map and positioning data containing geometric and semantic information.

[0096] It should be noted that the SLAM simultaneous localization and mapping operation refers to generating the accurate position of the vehicle in the environment and the detailed map of the environment by simultaneously performing vehicle positioning and environment mapping. This operation plays an important role in the unmanned system, and can provide reliable environment representation and positioning information for path planning and decision making. In the embodiments of the present application, the SLAM operation is based on environment fusion data and is realized through the steps of map construction, positioning calculation, map optimization and semantic information integration. In the unmanned intelligent scheduling, using SLAM operation can effectively generate an environment map containing geometric information and semantic information, and determine the accurate position of the vehicle in the map in real time to support subsequent path planning and decision making.

[0097] The map construction operation refers to generating initial map data from environment fusion data. Through laser radar point cloud data and camera image data, an initial map containing geometric information such as roads, obstacles and traffic signs is constructed. This step provides a basic environment representation for subsequent positioning and optimization. The positioning calculation operation refers to calculating the accurate position of the vehicle in the map according to the initial map data and the environment fusion data. By matching the current sensor data with the initial map data, the real-time position of the vehicle is calculated using a particle filter algorithm. Of course, depending on the actual application scenario and user requirements, methods such as Kalman filtering or deep learning models can also be used, and the present application does not limit this. The map optimization operation refers to optimizing the map according to the vehicle positioning data and the initial map data to improve the accuracy and consistency of the map. Through a nonlinear least squares algorithm, the nodes and edges in the map are optimized to eliminate the cumulative errors in the positioning and map construction process. The semantic information integration operation refers to integrating semantic information (such as traffic signs, lane lines, pedestrians, etc.) into the optimized map to generate a detailed map containing geometric and semantic information. Through a deep learning model (such as a convolutional neural network), semantic segmentation and object detection are performed on the camera images, and the detected semantic information is fused with the map data.

[0098] In step S14, a rapid expansion random tree connection operation is performed according to the map and the positioning data to obtain planning path data, including:

[0099] According to the map and the positioning data, a start point and target point setting operation is performed to obtain path planning start point and target point data;

[0100] According to the path planning start point and target point data, a random tree initialization operation is performed to obtain initial random tree data;

[0101] According to the initial random tree data, a random sampling operation is performed to obtain random sampling point coordinates;

[0102] According to the random sampling point coordinates, a nearest node searching operation is performed to obtain nearest node coordinates;

[0103] According to the nearest node coordinates and the random sampling point coordinates, a new node generation calculation is performed to obtain new node coordinates;

[0104] According to the new node coordinates, a collision detection operation is performed to obtain collision-free path segment data;

[0105] According to the collision-free path segment data, a random tree expansion operation is performed to obtain expanded random tree data;

[0106] According to the expanded random tree data, a target point connection operation is performed to obtain planning path data.

[0107] It should be noted that the rapid expansion random tree connection operation refers to searching a collision-free path from the starting point to the target point in a complex environment through random sampling and tree structure expansion. This operation plays an important role in the unmanned system and can provide efficient and safe path planning support for vehicles. In the embodiment of the present application, the rapid expansion random tree connection operation is realized based on map and positioning data through the steps of starting point and target point setting, random tree initialization, random sampling, nearest node searching, new node generation, collision detection, random tree expansion and target point connection. In the unmanned intelligent scheduling, the use of the rapid expansion random tree connection operation can effectively generate a collision-free planning path to support the safe driving of vehicles in a complex traffic environment.

[0108] It should be noted that the starting point and target point setting operation refers to determining the starting point and target point of path planning according to the current position and target position of the vehicle. This step provides a clear search range for path planning. The random tree initialization operation refers to initializing the random tree structure with the starting point as the root node. This step provides a basic data structure for subsequent random sampling and tree expansion. The random sampling operation refers to randomly generating a sampling point coordinate in the search space. The nearest node searching operation refers to searching for the nearest node to the random sampling point in the random tree, which is used to determine the direction of tree expansion. The new node generation calculation refers to generating a new node coordinate according to the nearest node coordinate and the random sampling point coordinate, which is used to ensure the continuity and feasibility of the path. The collision detection operation refers to detecting whether the new node collides with the obstacles in the map, which is used to ensure that the generated path segment is collision-free. The random tree expansion operation refers to adding the collision-free path segment to the random tree to expand the tree structure. The target point connection operation refers to expanding the random tree to the target point to generate a complete path from the starting point to the target point.

[0109] It should be noted that the calculation formula of the new node generation calculation is as follows:

[0110]

[0111] where x new represents the generated new node coordinate, x near represents the nearest node coordinate, x rand represents the random sampling point coordinate, d step represents the preset step value, ||x rand -x near represents the Euclidean distance between the random sampling point coordinate and the nearest node coordinate.

[0112] It is worth noting that the new node generation calculation formula generates a new node coordinate by normalizing the vector between the randomly sampled point and the nearest node and multiplying it by a preset step value. This formula ensures that the new node expands in the direction of the randomly sampled point within the step range, while avoiding the problem of excessively long or short path segments. In this way, collision-free path segments can be efficiently generated in complex environments, providing reliable support for subsequent path planning.

[0113] In step S15, according to the planning path data and the environment fusion data, a traffic condition prediction model construction operation is performed to obtain traffic condition prediction data, including:

[0114] According to the planning path data and the environment fusion data, a historical traffic data extraction operation is performed to obtain historical traffic data;

[0115] According to the historical traffic data, a time series analysis operation is performed to obtain traffic trend data;

[0116] According to the traffic trend data, a neural network model training operation is performed to obtain a traffic prediction model;

[0117] According to the traffic prediction model and real-time environment fusion data, a traffic condition prediction operation is performed to obtain traffic condition prediction data.

[0118] It should be noted that the traffic condition prediction model construction operation refers to predicting the future traffic condition change trend by analyzing historical traffic data and real-time environment data. This operation plays an important role in the unmanned driving system and can provide scientific decision-making basis for vehicle scheduling and path planning. In the embodiments of the present application, the traffic condition prediction model construction operation is based on planning path data and environment fusion data, and is realized through historical traffic data extraction, time series analysis, neural network model training and traffic condition prediction steps. In the unmanned intelligent scheduling, using the traffic condition prediction model can effectively predict the future traffic condition change, support efficient scheduling and safe driving of vehicles in complex traffic environment.

[0119] The historical traffic data extraction operation refers to extracting historical traffic data from the planned path data and the environmental fusion data. These data include but are not limited to traffic flow, vehicle speed, congestion, traffic accidents, and other information. This step provides a data basis for subsequent time series analysis and model training. The time series analysis operation refers to time series analysis of historical traffic data to extract traffic trend data. By analyzing the time variation of traffic data, periodicity, trend and suddenness of traffic conditions can be identified to provide input features. The neural network model training operation refers to training a neural network model based on traffic trend data to predict future traffic conditions. In the embodiment of the application, a recurrent neural network (RNN) model is used for training to generate a traffic prediction model that can accurately predict traffic conditions. The traffic condition prediction operation refers to using the trained traffic prediction model and real-time environmental fusion data to predict future traffic condition changes. By inputting real-time environmental data (such as current traffic flow, vehicle speed, weather conditions, etc.), the traffic prediction model can output traffic condition prediction data for a future period of time.

[0120] It should be noted that according to the historical traffic data, the time series analysis operation is performed to obtain traffic trend data, including:

[0121] According to the historical traffic data, a graph attention network model construction operation is performed to obtain traffic graph structure data;

[0122] According to the traffic graph structure data, a node feature extraction operation is performed to obtain node feature data;

[0123] According to the node feature data, an attention mechanism calculation operation is performed to obtain node attention weight data;

[0124] According to the node attention weight data, a time series prediction operation is performed to obtain traffic trend data;

[0125] The calculation formula of the graph attention network model is as follows:

[0126]

[0127] where h′ i represents the updated feature of node i, σ represents an activation function, represents the neighbor node set of node i, α ij represents the attention weight between node i and node j, W represents a learnable weight matrix, h j represents the feature of node j.

[0128] It is worth noting that the time series analysis operation analyzes historical traffic data through a graph attention network model (GAT), which can effectively capture the spatio-temporal dependence relationship in the traffic data. Specifically, the traffic graph structure data represents the nodes (such as intersections, road segments) and edges (such as connection relationships) in the traffic network as a graph structure, and the node feature data contains the traffic characteristics (such as traffic volume, speed, congestion level, etc.) of each node. Through attention mechanism calculation operation, the model can dynamically allocate attention weights between nodes, thereby more accurately aggregating the information of neighboring nodes. Finally, through time series prediction operation, the model can generate future traffic trend data.

[0129] Among them, the graph attention network model construction operation refers to constructing traffic graph structure data according to historical traffic data. The nodes in the traffic graph represent the key points (such as intersections, road segments) in the traffic network, and the edges represent the connection relationship between the nodes. Through the graph structure, the topological relationship of the traffic network and the dependence relationship between the nodes can be effectively represented. The node feature extraction operation refers to extracting the feature data of each node according to the traffic graph structure data. The node feature data includes but is not limited to traffic volume, speed, congestion level, traffic accidents and other information. These feature data provide input for subsequent attention mechanism calculation and time series prediction. The attention mechanism calculation operation refers to calculating the attention weights between nodes according to the node feature data. Through the attention mechanism, the model can dynamically allocate weights between nodes, thereby more accurately aggregating the information of neighboring nodes. The calculation of attention weights is based on the similarity and importance of node features, ensuring that the information of key nodes is fully focused, which is used to capture the spatio-temporal dependence relationship in the traffic data and improve the prediction accuracy of the model. The time series prediction operation refers to performing time series prediction operation according to the node attention weight data, and generating traffic trend data. Through the time series prediction model LSTM, the model can capture the temporal dependence relationship in the traffic data and predict the future traffic condition change. The generated traffic trend data includes traffic flow, speed, congestion level and other prediction results.

[0130] In step S16, according to the traffic condition prediction data, a decision model construction operation is performed to obtain decision strategy data, including:

[0131] According to the traffic condition prediction data, a trajectory prediction model construction operation is performed to obtain trajectory prediction data of the unmanned vehicle;

[0132] According to the trajectory prediction data, an optimization function calculation operation is performed to obtain optimal trajectory data;

[0133] According to the optimal trajectory data, a decision strategy generation operation is performed to obtain decision strategy data.

[0134] It should be noted that the decision model construction operation refers to generating decision strategy data of the unmanned vehicle by analyzing traffic condition prediction data. This operation plays an important role in the unmanned system and can provide scientific driving strategies for the vehicle to ensure its safety and efficiency in complex traffic environments. In the embodiments of the present application, the decision model construction operation is based on traffic condition prediction data and is implemented through steps such as trajectory prediction model construction, optimization function calculation, and decision strategy generation. In the intelligent scheduling of unmanned vehicles, the use of the decision model construction operation can effectively generate optimal trajectories and decision strategies to support efficient scheduling and safe driving of vehicles in complex traffic environments.

[0135] Among them, the trajectory prediction model construction operation refers to constructing a trajectory prediction model of the unmanned vehicle according to the traffic condition prediction data. Through the trajectory prediction model, the driving trajectory of the vehicle in the future can be predicted. Specifically, the trajectory prediction model construction operation includes the following steps: data preprocessing, which preliminarily processes the traffic condition prediction data to eliminate noise, correct errors, and handle outliers. Feature extraction, which extracts key features from the preprocessed traffic condition prediction data. These features include but are not limited to traffic flow, vehicle speed, congestion level, traffic accidents, etc. In the embodiments of the present application, Fourier transform is used to extract the periodicity, trend, and suddenness features of the traffic data. Model training, which trains the trajectory prediction model based on the extracted feature data to predict the driving trajectory of the vehicle. In the embodiments of the present application, the Transformer model is used for training to capture the global dependency relationship in the traffic data. Trajectory prediction, which uses the trained trajectory prediction model and real-time traffic condition prediction data to predict the driving trajectory of the vehicle in the future. By inputting real-time traffic condition data (such as current traffic flow, vehicle speed, weather conditions, etc.), the trajectory prediction model can output predicted trajectory data of the vehicle. The optimization function calculation operation refers to calculating optimal trajectory data according to the trajectory prediction data. Through the optimization function, the safety, efficiency, and comfort of the path can be considered comprehensively to generate the optimal driving trajectory. The decision strategy generation operation refers to generating decision strategy data according to the optimal trajectory data. The decision strategy data includes driving path, speed, steering, and other operation instructions of the vehicle. This step generates a scientific driving strategy for the vehicle through decision strategy generation to ensure its safe driving in complex traffic environments.

[0136] It is worth noting that the calculation formula of the optimization function calculation is as follows:

[0137]

[0138] Among them, x represents the optimal trajectory of the unmanned vehicle, P represents the covariance matrix of the driving trajectory of the unmanned vehicle, q represents the parameter vector of the unmanned vehicle to avoid collision, represents minimizing the objective function J with respect to the variable x, and T represents the transpose of a vector.

[0139] It should be noted that the optimization function can comprehensively consider the safety, efficiency and comfort of the path by minimizing the objective function J, to generate the optimal driving trajectory. Specifically, x T The Px term represents the smoothness and stability of the trajectory, which is constrained by the covariance matrix P to the fluctuations of the trajectory; q T The x term represents the safety of the trajectory, which is constrained by the parameter vector q to the collision risk of the trajectory. Through the optimization function calculation, a smooth and safe driving trajectory can be generated to ensure efficient scheduling and safe driving of the vehicle in complex traffic environment.

[0140] In step S17, according to the decision strategy data, real-time decision operation is performed to obtain adjusted driving instruction data to make the vehicle adjust the path and optimize the driving.

[0141] It should be noted that the real-time decision operation refers to generating specific driving instruction data according to the decision strategy data to control the speed, steering and path adjustment of the vehicle. In the unmanned intelligent scheduling, using real-time decision operation can effectively respond to the changes of traffic environment and ensure the safe driving of the vehicle in complex traffic environment.

[0142] Referring to Figure 2 , the second embodiment of the present application provides an unmanned intelligent scheduling system based on multi-source data, comprising:

[0143] A data acquisition module is configured to acquire surrounding environment information and traffic infrastructure information of an unmanned vehicle.

[0144] A data fusion module is configured to perform multi-source data fusion operation according to the surrounding environment information of the unmanned vehicle and the traffic infrastructure information to obtain environment fusion data.

[0145] A SLAM module is configured to perform SLAM simultaneous localization and mapping operation according to the environment fusion data to obtain map and positioning data.

[0146] A path planning module is configured to perform rapid extension random tree connection operation according to the map and the positioning data to obtain planning path data.

[0147] A traffic condition prediction module is configured to perform traffic condition prediction model construction operation according to the planning path data and the environment fusion data to obtain traffic condition prediction data.

[0148] A decision module is configured to perform decision model construction operation according to the traffic condition prediction data to obtain decision strategy data.

[0149] a path optimization module, configured to perform real-time decision operation according to the decision strategy data, to obtain adjusted driving instruction data, so as to make the vehicle perform path adjustment and driving optimization.

[0150] It should be noted that the above-mentioned embodiment of the present application provides a kind of based on multi-source data's unmanned intelligent scheduling system for executing the all process steps of a kind of based on multi-source data's unmanned intelligent scheduling method of above-mentioned embodiment, the working principle and beneficial effects of the two are one to one correspondence, thus no longer repeat.

[0151] To sum up, in the present application, the surrounding environment information and traffic infrastructure information of the unmanned vehicle are integrated by multi-source data fusion technology, then the SLAM synchronous positioning and map construction operation are performed according to the fused environment data, the map and positioning data are generated, and the planning path data are generated by the rapidly expanding random tree connection algorithm according to the map and positioning data. Then, the system constructs the traffic condition prediction model according to the planning path data and environment fusion data, predicts the future traffic condition change, and generates decision strategy data according to the prediction result. Finally, the system adjusts the driving path and operation strategy of the vehicle in real time according to the decision strategy data, to ensure the safe driving of the vehicle in complex traffic environment. The traffic condition prediction model is constructed based on historical traffic data and real-time environment data, which can more accurately reflect the change trend of traffic environment, and through the fusion and analysis of multi-source data, the adaptability and decision-making ability of the unmanned vehicle can be significantly improved. In the process of path planning and decision-making, the system considers the traffic infrastructure information and environmental change trend at the same time, which can realize accurate prediction of traffic condition and generation of optimal path.

[0152] The embodiment of the present application also provides a terminal device. The terminal device comprises a processor, a memory and a computer program stored in the memory and executable on the processor, such as an unmanned intelligent scheduling program based on multi-source data. The processor implements the steps in the above-mentioned various embodiments of the unmanned intelligent scheduling method based on multi-source data when executing the computer program, such as Figure 1 The steps shown in S11. Alternatively, the processor implements the functions of each module / unit in the above-mentioned various system embodiments when executing the computer program.

[0153] For example, the computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the terminal device.

[0154] The terminal device can be a desktop computer, a notebook computer, a palm computer, a smart tablet, and the like. The terminal device can include, but is not limited to, a processor, a memory. Those skilled in the art can understand that the above components are only examples of the terminal device, and do not constitute a limitation on the terminal device, and can include more or fewer components than the above, or combine certain components, or different components, for example, the terminal device can also include an input / output device, a network access device, a bus, and the like.

[0155] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, and the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor and the like. The processor is a control center of the terminal device, and connects all parts of the terminal device through various interfaces and lines.

[0156] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the terminal device by running or executing the computer program and / or modules stored in the memory, and calling data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, and the like), and the like; and the data storage area can store data created according to use of the terminal device (such as audio data, a phone book, and the like), and the like. In addition, the memory can include a high-speed random access memory, and can also include a nonvolatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory device.

[0157] The modules / units integrated in the terminal device, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can implement the steps of each method embodiment when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer readable medium can include any entity or system, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. that can carry the computer program code. It should be noted that the computer readable medium can include or exclude contents according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0158] It should be noted that the above-described system embodiments are only illustrative, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. In addition, the connection relationship between the modules in the system embodiments provided by the present application indicates that there is a communication connection between them, which can be realized as one or more communication buses or signal lines. Those skilled in the art can understand and implement it without creative labor.

[0159] The above-described specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above-described specific embodiments are only examples of the present application and are not intended to limit the protection scope of the present application. It is particularly pointed out that any modification, equivalent replacement, improvement, etc. made by those skilled in the art within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A multi-source data-based unmanned intelligent scheduling method, characterized in that, The method comprises the following steps: acquiring surrounding environment information and traffic infrastructure information of an unmanned vehicle; performing multi-source data fusion operation according to the surrounding environment information and the traffic infrastructure information of the unmanned vehicle to obtain environment fusion data; performing SLAM (Simultaneous Localization and Mapping) operation according to the environment fusion data to obtain map and positioning data; performing rapid extended random tree connection operation according to the map and the positioning data to obtain planning path data; performing traffic condition prediction model construction operation according to the planning path data and the environment fusion data to obtain traffic condition prediction data; performing decision model construction operation according to the traffic condition prediction data to obtain decision strategy data; performing real-time decision operation according to the decision strategy data to obtain adjusted driving instruction data, so as to make the vehicle adjust the path and optimize the driving; wherein the rapid extended random tree connection operation according to the map and the positioning data to obtain the planning path data comprises: performing start point and target point setting operation according to the map and the positioning data to obtain path planning start point and target point data; performing random tree initialization operation according to the path planning start point and target point data to obtain initial random tree data; performing random sampling operation according to the initial random tree data to obtain random sampling point coordinates; performing nearest node searching operation according to the random sampling point coordinates to obtain nearest node coordinates; performing new node generation calculation according to the nearest node coordinates and the random sampling point coordinates to obtain new node coordinates; performing collision detection operation according to the new node coordinates to obtain non-collision path segment data; performing random tree expansion operation according to the non-collision path segment data to obtain expanded random tree data; performing target point connection operation according to the expanded random tree data to obtain planning path data; wherein the calculation formula of the new node generation calculation is as follows: ； wherein, represents the generated new node coordinate, represents the nearest node coordinate, represents the random sampling point coordinate, represents a preset step value, represents the Euclidean distance between the random sampling point coordinate and the nearest node coordinate. 2.The method of claim 1, wherein, the multi-source data fusion operation according to the surrounding environment information and the traffic infrastructure information of the unmanned vehicle to obtain the environment fusion data comprises: performing data preprocessing according to the surrounding environment information and the traffic infrastructure information of the unmanned vehicle to obtain clean multi-source environment data; performing time stamp synchronization operation according to the clean multi-source environment data to obtain multi-source environment synchronization data; performing space registration operation according to the clean multi-source environment data to obtain multi-source environment registration data; performing data fusion according to the multi-source environment synchronization data and the multi-source environment registration data to obtain environment fusion data. 3.The method of claim 1, wherein, the SLAM operation according to the environment fusion data to obtain the map and the positioning data comprises: performing map construction operation according to the environment fusion data to obtain initial map data; performing positioning calculation operation according to the initial map data and the environment fusion data to obtain vehicle positioning data; performing map optimization operation according to the vehicle positioning data and the initial map data to obtain map optimization data; According to the map optimization data and the vehicle positioning data, a semantic information integration operation is performed to obtain a map and positioning data containing geometric and semantic information. 4.The method of claim 1, wherein, According to the planning path data and the environment fusion data, a traffic condition prediction model construction operation is performed to obtain traffic condition prediction data, including: According to the planning path data and the environment fusion data, a historical traffic data extraction operation is performed to obtain historical traffic data; According to the historical traffic data, a time series analysis operation is performed to obtain traffic trend data; According to the traffic trend data, a neural network model training operation is performed to obtain a traffic prediction model; According to the traffic prediction model and real-time environment fusion data, a traffic condition prediction operation is performed to obtain traffic condition prediction data. 5.The method of claim 4, wherein, According to the historical traffic data, a time series analysis operation is performed to obtain traffic trend data, including: According to the historical traffic data, a graph attention network model construction operation is performed to obtain traffic graph structure data; According to the traffic graph structure data, a node feature extraction operation is performed to obtain node feature data; According to the node feature data, an attention mechanism calculation operation is performed to obtain node attention weight data; According to the node attention weight data, a time series prediction operation is performed to obtain traffic trend data; The calculation formula of the graph attention network model is as follows: ； wherein, represents an updated feature of node i, represents an activation function, represents a set of neighbor nodes of node i, represents an attention weight between node i and node j, represents a learnable weight matrix, represents a feature of node j. 6.The method of claim 1, wherein, According to the traffic condition prediction data, a decision model construction operation is performed to obtain decision strategy data, including: According to the traffic condition prediction data, a trajectory prediction model construction operation is performed to obtain trajectory prediction data of the unmanned vehicle; According to the trajectory prediction data, an optimization function calculation operation is performed to obtain optimal trajectory data; According to the optimal trajectory data, a decision strategy generation operation is performed to obtain decision strategy data. 7.The method of claim 6, wherein, The calculation formula of the optimization function calculation is as follows: ; wherein, represents an optimal trajectory for the autonomous vehicle to travel, represents a covariance matrix of the autonomous vehicle's trajectory, represents a parameter vector for the autonomous vehicle to avoid collision, represents a variable minimizing an objective function , represents the transpose of a vector.

8. An unmanned intelligent dispatching system based on multi-source data, characterized in that, The unmanned intelligent scheduling method based on multi-source data according to any one of claims 1-7, comprising: A data acquisition module for acquiring surrounding environment information and traffic infrastructure information of an unmanned vehicle; A data fusion module for performing multi-source data fusion operation according to the surrounding environment information and the traffic infrastructure information of the unmanned vehicle to obtain environment fusion data; A SLAM module for performing SLAM simultaneous localization and map construction operation according to the environment fusion data to obtain map and positioning data; A path planning module for performing rapid extended random tree connection operation according to the map and the positioning data to obtain planning path data; A traffic condition prediction module for performing traffic condition prediction model construction operation according to the planning path data and the environment fusion data to obtain traffic condition prediction data; A decision module for performing decision model construction operation according to the traffic condition prediction data to obtain decision strategy data; A path optimization module for performing real-time decision operation according to the decision strategy data to obtain adjusted driving instruction data to make the vehicle adjust the path and optimize the driving.

Images