A path planning method based on multi-source heterogeneous data

Abstract

The present invention discloses a path planning method based on multi-source heterogeneous data, which belongs to the field of path planning and includes the following steps: collecting multi-source heterogeneous data for path planning; generating core data related to the path; performing data enhancement on the multi-source heterogeneous data; performing deep learning of the multi-source heterogeneous data using a convolutional neural network (CNN); and performing parallel acceleration using a graphics processing unit (GPU). In path planning, the present invention can provide richer environmental information by fusing multi-source heterogeneous data, thereby improving planning accuracy and decision reliability.

Description

Technical Field

[0001] The present invention relates to the field of path planning, and in particular to a path planning method based on multi-source heterogeneous data. Background Art

[0002] With the rapid development of artificial intelligence technology, path planning, as the core technology in intelligent systems such as autonomous driving, robot navigation, and drone flight, has become one of the research hotspots. Traditional path planning algorithms, such as Dijkstra algorithm and Algorithms, which usually rely on graph search and heuristic search strategies, can effectively find the shortest path in a known environment. However, with the complexity of actual application scenarios, the single use of traditional search algorithms has certain limitations when processing multi-source heterogeneous data (such as images, text, sensor data, etc.), and it is difficult to cope with dynamic environments, complex constraints, and the fusion requirements of multimodal information.

[0003] In recent years, with the rapid development of deep learning, especially the widespread application of convolutional neural networks (CNN), path planning methods based on neural networks have gradually emerged. Convolutional neural networks can effectively extract features from image data, and achieve deep learning and feature extraction of complex data through a multi-layer structure. However, in the actual path planning process, in addition to image data, it is also necessary to process multi-source heterogeneous information such as text information and sensor data to provide more context and environmental information. The fusion and efficient processing of this multi-source data has become a major challenge facing the current path planning field. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a path planning method based on multi-source heterogeneous data with simple algorithm and high efficiency.

[0005] The technical solution of the present invention to solve the above technical problem is: a path planning method based on multi-source heterogeneous data, comprising the following steps:

[0006] Step S1: Collect multi-source heterogeneous data for path planning, extract useful features from the multi-source heterogeneous data and perform fusion processing;

[0007] Step S2: Use The algorithm generates core data related to the path, including the path link diagram, the distance between nodes, and cost information. The core data comes from many different types of inputs and is heterogeneous.

[0008] Step S3: Perform data enhancement on multi-source heterogeneous data;

[0009] Step S4: using a convolutional neural network (CNN) to perform deep learning on multi-source heterogeneous data, learning and extracting features from the multi-source heterogeneous data through the convolutional neural network, processing the image information, and combining auxiliary information from other modalities to improve the accuracy of path planning;

[0010] Step S5: Use a graphics processing unit (GPU) for parallel acceleration to accelerate the training and reasoning process of the convolutional neural network.

[0011] In the above-mentioned path planning method based on multi-source heterogeneous data, in step S1, multi-source heterogeneous data refers to a data set obtained from different data sources and having different data types, structures, formats or semantics;

[0012] The heterogeneity of multi-source heterogeneous data is manifested in the following aspects:

[0013] Diversity of data sources: Data generated by different data sources have different characteristics and uses;

[0014] Heterogeneity of data formats: Different types of data use different representations. Images are two-dimensional matrices, text is sequence data, and sensor data is time series or discrete point clouds.

[0015] Heterogeneity of data semantics: Each data type contains different semantic information. Images contain geometric information of paths, and text data describes the cost or risk of paths.

[0016] In the above-mentioned path planning method based on multi-source heterogeneous data, in step S1, in the path planning task, the multi-source heterogeneous data includes image data, text data, other sensor data and path link diagram:

[0017] Image data: Image data is represented as a two-dimensional matrix :

[0018] ;

[0019] in, is the resolution of the image, Indicates that the image The pixel value at the position; for a picture, it is essentially a two-dimensional matrix, assuming ,but , are the corresponding coordinates;

[0020] Text data: Text data is represented as a sequence of words, and the text data is converted into a vector representation using text vectorization technology:

[0021] ;

[0022] in, Indicates m word vector embedding; is a real number, is a word sequence, d represents the value of the norm;

[0023] Path link diagram: The path links generated by the algorithm are represented as a graph ,in is a node set, is an edge set, each edge Representation Node and The path links between nodes and The associated costs for:

[0024] ;

[0025] in, It is a path cost function based on node information and external data;

[0026] Multimodal data fusion: The following weighted sum form is used to integrate various heterogeneous data into a unified model:

[0027] ;

[0028] in, It represents the linear combination of the weighted image data features, text data features and other sensor data features, where Represents the characteristics of image data, Represents the characteristics of text data, Represents features of other sensor data; They respectively represent the weight parameters acting on the features of image data, the features of text data, and the features of other sensor data to balance the representation of related data.

[0029] In the above-mentioned path planning method based on multi-source heterogeneous data, in step S2, The algorithm finds the shortest path from the start node to the target node in the graph. The key to the algorithm is to use an evaluation function to estimate the minimum cost from the current node to the target node;

[0030] Each node in the algorithm has a scoring criterion, which consists of two parts:

[0031] : From the starting point to the node the actual cost of

[0032] : Indicates slave node the estimated cost to reach the destination;

[0033] The standard function for the algorithm to select a node is:

[0034] ;

[0035] in, From the starting point to the node The estimated total cost to reach the destination.

[0036] In the above-mentioned path planning method based on multi-source heterogeneous data, the specific process of step S2 is as follows:

[0037] Step S21: Initialization, adding the starting point to the open list;

[0038] Step S22: Determine whether there is only the starting point in the open list. If so, it means that the path to the target point has not been found. At this time, select the node that has not been explored. The node with the smallest value is selected as the current node; if not, select the node in the open list The node with the smallest value is used as the current node to continue The algorithm’s search process;

[0039] Step S23: determine whether the current node is the target node. If yes, end the search and backtrack the path; if no, proceed to the next step;

[0040] Step S24: For each neighbor node, calculate the temporary Value, that is, the cost of reaching the neighbor node from the starting point through the current node;

[0041] Step S25: Check whether the neighbor node is in the closed list or in the open list. The closed list contains all processed and expanded nodes, and the open list contains all unexplored nodes. If the neighbor node is in the closed list, it means that the neighbor node has been processed and expanded, so there is no need to consider it again and ignore it directly; if the neighbor node is in the open list, further determine the temporary Is the value less than the known If so, it means that a shorter path to the neighbor node has been found. The value is updated to the temporary value of the neighbor node value, and set the current node as the parent node of the neighbor node;

[0042] Step S26: According to the updated Value and heuristic functions Calculate the neighbor node value;

[0043] Step S27: Return to step S22 and continue searching until the end point is found or it is confirmed that there is no reachable path.

[0044] In the above-mentioned path planning method based on multi-source heterogeneous data, in step S3, data enhancement includes:

[0045] Image augmentation: augmenting datasets of maps or satellite images by rotating, scaling, cropping, and color transforming them to improve the robustness of models under different viewing angles and lighting conditions.

[0046] Text data supplement: introduce more route description information, including road conditions, weather, and emergencies, so as to consider more semantic factors when planning routes;

[0047] Sensor data processing: De-noise and interpolate the data collected by the sensor to reduce errors.

[0048] In the above-mentioned path planning method based on multi-source heterogeneous data, in step S4, a convolutional neural network is used to process multiple input data types, including images, texts, and sensor data, to achieve more accurate path planning, specifically including:

[0049] Image processing: CNN extracts features from the input enhanced image. Through multi-layer convolution, pooling, and fully connected layer operations, it gradually extracts key path information of roads and obstacles and generates high-dimensional feature vectors.

[0050] Text processing: CNN converts enhanced text data into vectors through the embedding layer, captures the contextual associations in the text through the convolution layer, and makes decisions based on image features;

[0051] Multimodal fusion: In the processing of multi-source heterogeneous data, CNN uses different convolution branches to process image and text data types, and then combines the features of different data sources through the feature fusion layer to generate a unified feature representation for path planning optimization.

[0052] In the above-mentioned path planning method based on multi-source heterogeneous data, step S5 specifically includes:

[0053] Parallel computing: Utilizing the multi-core architecture of GPU, the convolutional neural network's convolution, pooling, and matrix operations are performed simultaneously on multiple cores, increasing data processing speed;

[0054] Large-scale data processing: GPUs are good at processing large amounts of data and complex neural network structures, including large-scale matrix multiplication and accumulation operations in convolution operations;

[0055] Model training acceleration: When training deep learning models, the GPU coprocessor accelerates the gradient calculation in the back-propagation algorithm, improves the training speed of multi-layer neural networks, and reduces the training time of the model;

[0056] Reasoning acceleration: In the real-time reasoning stage of path planning tasks, GPU acceleration ensures that the neural network responds quickly, processes multi-source heterogeneous data in real time, and provides path decision suggestions.

[0057] The beneficial effects of the present invention are:

[0058] 1. In path planning, the present invention can provide richer environmental information by fusing multi-source heterogeneous data, thereby improving planning accuracy and decision reliability.

[0059] 2. The present invention performs data enhancement on multi-source heterogeneous data. Through data enhancement, the model performance in the path planning process can be further optimized to ensure that efficient and reliable path planning solutions can still be generated under changing environmental conditions.

[0060] 3. The present invention uses convolutional neural networks to perform deep learning on enhanced multi-source heterogeneous data, which can significantly improve the path planning accuracy and efficiency of the model in complex environments.

[0061] 4. The present invention uses the graphics processing unit GPU for parallel acceleration to accelerate the training and reasoning process of the convolutional neural network, greatly improving the computational efficiency of the convolutional neural network in deep learning tasks of multi-source heterogeneous data, especially in path planning tasks that require fast response, ensuring the efficient operation of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0063] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0064] like Figure 1 As shown, a path planning method based on multi-source heterogeneous data includes the following steps:

[0065] Step S1: Collect multi-source heterogeneous data for path planning, extract useful features from the multi-source heterogeneous data and perform fusion processing.

[0066] Multi-source heterogeneous data refers to a collection of data obtained from different data sources with different data types, structures, formats or semantics.

[0067] The heterogeneity of multi-source heterogeneous data is manifested in the following aspects:

[0068] Diversity of data sources: Data generated by different data sources have different characteristics and uses;

[0069] Heterogeneity of data formats: Different types of data use different representations. Images are two-dimensional matrices, text is sequence data, and sensor data is time series or discrete point clouds.

[0070] Heterogeneity of data semantics: Each data type contains different semantic information. Images contain geometric information of paths, and text data describes the cost or risk of paths.

[0071] In common path planning tasks, multi-source heterogeneous data include image data (such as high-precision maps, satellite images), text data (such as road conditions, link costs), other sensor data (such as lidar, GPS coordinates, etc.), and path link graphs.

[0072] In path planning, by fusing multi-source heterogeneous data, richer environmental information can be provided, thereby improving planning accuracy and decision reliability. In order to extract useful features from multi-source heterogeneous data and perform fusion processing, the following mathematical representation is used to describe it:

[0073] Image data: Image data is represented as a two-dimensional matrix :

[0074] ;

[0075] in, is the resolution of the image, Indicates that the image The pixel value at the position; for a picture, it is essentially a two-dimensional matrix, assuming ,but , are the corresponding coordinates.

[0076] Text data: Text data is represented as a sequence of words. To make it easier to process, the text data is converted into a vector representation using text vectorization technology:

[0077] ;

[0078] in, Indicates m word vector embeddings; is a real number, is a word sequence, and d represents the value of the norm.

[0079] Path link diagram: The path links generated by the algorithm are represented as a graph ,in is a node set, is an edge set, each edge Representation Node and The path links between nodes and The associated costs for:

[0080] ;

[0081] in, It is a path cost function based on node information and external data (such as text or sensor data).

[0082] Multimodal data fusion: The following weighted sum form is used to integrate various heterogeneous data into a unified model:

[0083] ;

[0084] in, It represents the linear combination of the weighted image data features, text data features and other sensor data features, where Represents the characteristics of image data, Represents the characteristics of text data, Represents features of other sensor data; They respectively represent the weight parameters acting on the features of image data, the features of text data, and the features of other sensor data to balance the representation of related data.

[0085] Step S2: Use The algorithm generates core data related to the path. The core data includes the path link diagram, the distance between nodes, and the cost information. The core data comes from many different types of inputs and is heterogeneous.

[0086] use Generate a hypothetical optimal path planning link, and use this link to derive the path cost overhead of path selection, as well as the corresponding link path decision graph.

[0087] The algorithm finds the shortest path from the start node to the target node in the graph. The key to the algorithm is to use an evaluation function to estimate the minimum cost from the current node to the target node.

[0088] Each node in the algorithm has a scoring criterion, which consists of two parts:

[0089] : From the starting point to the node the actual cost of

[0090] : Indicates slave node The estimated cost to reach the destination.

[0091] The standard function for the algorithm to select a node is:

[0092] ;

[0093] in, From the starting point to the node The estimated total cost to reach the destination.

[0094] The specific process of step S2 is:

[0095] Step S21: Initialization, adding the starting point to the open list.

[0096] Step S22: Determine whether there is only the starting point in the open list. If so, it means that the path to the target point has not been found. At this time, select the node that has not been explored. The node with the smallest value is selected as the current node; if not, select the node in the open list The node with the smallest value is used as the current node to continue The algorithm’s search process;

[0097] Step S23: determine whether the current node is the target node. If yes, end the search and backtrack the path; if no, proceed to the next step;

[0098] Step S24: For each neighbor node, calculate the temporary Value, that is, the cost of reaching the neighbor node from the starting point through the current node;

[0099] Step S25: Check whether the neighbor node is in the closed list or in the open list. The closed list contains all processed and expanded nodes, and the open list contains all unexplored nodes. If the neighbor node is in the closed list, it means that the neighbor node has been processed and expanded, so there is no need to consider it again and ignore it directly; if the neighbor node is in the open list, further determine the temporary Is the value less than the known If so, it means that a shorter path to the neighbor node has been found. The value is updated to the temporary value of the neighbor node value, and set the current node as the parent node of the neighbor node;

[0100] Step S26: According to the updated Value and heuristic functions Calculate the neighbor node value;

[0101] Step S27: Return to step S22 and continue searching until the target node is found or it is confirmed that there is no path to reach it.

[0102] Step S3: Perform data enhancement on multi-source heterogeneous data to improve the generalization ability of the model. When performing data enhancement on multi-source heterogeneous data, various technologies such as image enhancement, text data supplementation, and sensor data processing are used to improve the model's understanding and adaptability to the path planning environment. Specifically, it includes:

[0103] Image augmentation: augmenting datasets of maps or satellite images by rotating, scaling, cropping, and color transforming them to improve the robustness of models under different viewing angles and lighting conditions.

[0104] Text data supplement: introduce more route description information, including road conditions, weather, and emergencies, so as to consider more semantic factors when planning routes;

[0105] Sensor data processing: De-noise and interpolate the data collected by the sensor to reduce errors and improve data accuracy.

[0106] Through data enhancement technology, the model performance in the path planning process can be further optimized to ensure that efficient and reliable path planning solutions can be generated under changing environmental conditions.

[0107] Step S4: Use the convolutional neural network (CNN) to perform deep learning on multi-source heterogeneous data. The convolutional neural network learns and extracts features from the multi-source heterogeneous data, processes the image information, and combines auxiliary information from other modalities to improve the accuracy of path planning.

[0108] Convolutional neural networks are used to process a variety of input data types, including images, text, and sensor data, to achieve more accurate path planning, including:

[0109] Image processing: CNN extracts features from the input enhanced image. Through multi-layer convolution, pooling, and fully connected layer operations, it gradually extracts key path information of roads and obstacles and generates high-dimensional feature vectors.

[0110] Text processing: CNN converts enhanced text data into vectors through the embedding layer, captures the contextual associations in the text through the convolution layer, and makes decisions based on image features;

[0111] Multimodal fusion: In the processing of multi-source heterogeneous data, CNN uses different convolution branches to process image and text data types, and then combines the features of different data sources through the feature fusion layer to generate a unified feature representation for path planning optimization.

[0112] Deep learning of enhanced multi-source heterogeneous data through convolutional neural networks can significantly improve the model's path planning accuracy and efficiency in complex environments.

[0113] Step S5: Use the graphics processing unit (GPU) for parallel acceleration to accelerate the training and reasoning process of the convolutional neural network. Specifically, it includes:

[0114] Parallel computing: Utilizing the multi-core architecture of GPU, the convolutional neural network's convolution, pooling, and matrix operations are performed simultaneously on multiple cores, increasing data processing speed;

[0115] Large-scale data processing: GPUs are good at processing large amounts of data and complex neural network structures, including large-scale matrix multiplication and accumulation operations in convolution operations;

[0116] Model training acceleration: When training deep learning models, the GPU coprocessor accelerates the gradient calculation in the back-propagation algorithm, improves the training speed of multi-layer neural networks, and reduces the training time of the model;

[0117] Reasoning acceleration: In the real-time reasoning stage of path planning tasks, GPU acceleration ensures that the neural network responds quickly, processes multi-source heterogeneous data in real time, and provides path decision suggestions.

[0118] The acceleration technology based on GPU coprocessor has greatly improved the computing efficiency of convolutional neural networks in deep learning tasks of multi-source heterogeneous data, especially in path planning tasks that require fast response, ensuring the efficient operation of the model.

Claims

1. A path planning method based on multi-source heterogeneous data, characterized in that: The following steps are involved: Step S1: Collect multi-source heterogeneous data for path planning, extract useful features from the multi-source heterogeneous data and perform fusion processing; In the path planning task, multi-source heterogeneous data includes image data, text data, other sensor data, and path link graphs: Image data: Image data is represented as a two-dimensional matrix I: I={I x,y |x=1,2,…M,y=1,2,…N} Among them, M×N is the resolution of the image, I x,y Represents the pixel value at position (x,y) in the image; Text data: Text data is represented as a sequence of words, and the text data is converted into a vector representation using text vectorization technology: Among them, v m Represents the mth word vector embedding; is a real number, T is a word sequence, and d represents the value of the norm; Path link graph: The path link generated by the A* algorithm is represented as a graph G = (V, E), where V is the node set, E is the edge set, and each edge e ij ∈E represents node q i and q j The path link between nodes q i and q j The associated cost c ij for: c ij =f(q i ,q j ) Among them, f is the path cost function based on node information and external data; Multimodal data fusion: F fusion The following weighted sum form is used to integrate various heterogeneous data into a unified model: F fusion =α·F I +β·F T +γ·F S Among them, F fusion It represents the linear combination of the weighted image data features, text data features and other sensor data features, where F I Represents the characteristics of image data, F T Represents the characteristics of text data, F S represents the features of other sensor data; α, β, and γ represent the weight parameters acting on the features of image data, text data, and other sensor data, respectively, to balance the representation of related data; Step S2: Use the A* algorithm to generate path-related core data, which includes path link graphs, distances between nodes, and cost information. The core data comes from a variety of different types of inputs and is heterogeneous. Step S3: Perform data enhancement on multi-source heterogeneous data; Step S4: using a convolutional neural network (CNN) to perform deep learning on multi-source heterogeneous data, learning and extracting features from the multi-source heterogeneous data through the convolutional neural network, processing the image information, and combining auxiliary information from other modalities to improve the accuracy of path planning; Step S5: Use a graphics processing unit (GPU) for parallel acceleration to accelerate the training and reasoning process of the convolutional neural network.

2. The path planning method based on multi-source heterogeneous data according to claim 1, characterized in that: In step S1, multi-source heterogeneous data refers to a data set obtained from different data sources and having different data types, structures, formats or semantics; The heterogeneity of multi-source heterogeneous data is manifested in the following aspects: Diversity of data sources: Data generated by different data sources have different characteristics and uses; Heterogeneity of data formats: Different types of data use different representations. Images are two-dimensional matrices, text is sequence data, and sensor data is time series or discrete point clouds. Heterogeneity of data semantics: Each data type contains different semantic information. Images contain geometric information of paths, and text data describes the cost or risk of paths.

3. The path planning method based on multi-source heterogeneous data according to claim 1, characterized in that: In step S2, the A* algorithm searches for the shortest path from the starting node to the target node in the graph. The key to the A* algorithm is to use an evaluation function to estimate the minimum cost from the current node to the target node. Each node in the A* algorithm has a scoring criterion, which consists of two parts: g(n): represents the actual cost from the starting point to node n; h(n): represents the estimated cost from node n to the end point; The standard function for selecting nodes by the A* algorithm is: f(n)=g(n)+h(n) Among them, f(n) represents the estimated total cost from the starting point to node n and then to the end point.

4. The path planning method based on multi-source heterogeneous data according to claim 3 is characterized in that: The specific process of step S2 is: Step S21: Initialization, adding the starting point to the open list; Step S22: Determine whether there is only the starting point in the open list. If so, it means that the path to the target point has not been found. At this time, the node with the smallest f(n) value from the unexplored nodes is selected as the current node; if not, the node with the smallest f(n) value in the open list is selected as the current node to continue the search process of the A* algorithm; Step S23: determine whether the current node is the target node. If so, end the search and backtrack the path. If not, continue to the next step; Step S24: For each neighbor node, calculate the temporary g value, that is, the cost of reaching the neighbor node from the starting point through the current node; Step S25: Check whether the neighbor node is in the closed list or in the open list. The closed list contains all processed and expanded nodes, and the open list contains all unexplored nodes. If the neighbor node is in the closed list, it means that the neighbor node has been processed and expanded, so there is no need to consider it again, and the neighbor node is directly ignored; if the neighbor node is in the open list, further determine whether the temporary g value of the neighbor node is less than the known g value. If so, it means that a shorter path to the neighbor node is found. At this time, the known g value is updated to the temporary g value of the neighbor node, and the current node is set as the parent node of the neighbor node; Step S26: Calculate the f value of the neighbor node according to the updated g value and the heuristic function h; Step S27: Return to step S22 and continue searching until the end point is found or it is confirmed that there is no reachable path.

5. The path planning method based on multi-source heterogeneous data according to claim 2, characterized in that: In step S3, data enhancement includes: Image augmentation: augmenting datasets of maps or satellite images by rotating, scaling, cropping, and color transforming them to improve the robustness of models under different viewing angles and lighting conditions. Text data supplement: introduce more route description information, including road conditions, weather, and emergencies, so as to consider more semantic factors when planning routes; Sensor data processing: De-noise and interpolate the data collected by the sensor to reduce errors.

6. The path planning method based on multi-source heterogeneous data according to claim 2, characterized in that: In step S4, the convolutional neural network is used to process multiple input data types, including images, text, and sensor data, to achieve more accurate path planning, specifically including: Image processing: CNN extracts features from the input enhanced image. Through multi-layer convolution, pooling, and fully connected layer operations, it gradually extracts key path information of roads and obstacles and generates high-dimensional feature vectors. Text processing: CNN converts enhanced text data into vectors through the embedding layer, captures the contextual associations in the text through the convolution layer, and makes decisions based on image features; Multimodal fusion: In the processing of multi-source heterogeneous data, CNN uses different convolution branches to process image and text data types, and then combines the features of different data sources through the feature fusion layer to generate a unified feature representation for path planning optimization.

7. The path planning method based on multi-source heterogeneous data according to claim 2, characterized in that: The step S5 specifically includes: Parallel computing: Utilizing the multi-core architecture of GPU, the convolutional neural network's convolution, pooling, and matrix operations are performed simultaneously on multiple cores, increasing data processing speed; Large-scale data processing: GPUs are good at processing large amounts of data and complex neural network structures, including large-scale matrix multiplication and accumulation operations in convolution operations; Model training acceleration: When training deep learning models, the GPU coprocessor accelerates the gradient calculation in the back-propagation algorithm, improves the training speed of multi-layer neural networks, and reduces the training time of the model; Reasoning acceleration: In the real-time reasoning stage of path planning tasks, GPU acceleration ensures that the neural network responds quickly, processes multi-source heterogeneous data in real time, and provides path decision suggestions.

Images