Path planning method and device, equipment, storage medium and product

Abstract

The invention discloses a path planning method and device, equipment, a storage medium and a product, and the method comprises the steps: inputting multi-source data into a preset hybrid expert architecture model, and setting a plurality of expert modules and an expert fusion module in the preset hybrid expert architecture model; feature extraction is carried out on the various kinds of source data through expert modules corresponding to the various kinds of source data, a plurality of feature extraction results are obtained, and each expert module learns feature information of the corresponding source data through pre-training; and performing feature fusion on the plurality of feature extraction results through an expert fusion module to obtain a path planning result. According to the method, feature extraction is carried out on various types of source data through the expert modules corresponding to the various types of source data, and then feature fusion is carried out on multiple feature extraction results to obtain the path planning result, so that compared with an existing way of carrying out path planning by using single-source data, features of multi-source data can be fused, and the path planning efficiency is improved. Path planning is performed by exerting the advantages of each source data, and the path planning efficiency is improved.

Description

Technical Field

[0001] The present application relates to the technical field of path planning, and in particular, to a path planning method, apparatus, device, storage medium and product. Background Art

[0002] With the wide application of robots and intelligent devices, intelligent path planning is a key component in robots and automation systems, which involves finding the best path from a starting point to an end point in a given environment. This technology is crucial for autonomous vehicles, unmanned aerial vehicles, service robots, and various autonomous navigation devices. Intelligent path planning not only needs to consider static obstacles but also be able to handle dynamically changing environments, such as moving objects or newly emerging obstacles. Existing global path planning is usually based on a pre-constructed map, aiming to determine a general route, which is a rough direction from the starting point to the target point. This method uses graph search algorithms such as the A* algorithm and Dijkstra algorithm to calculate the optimal path, relying on a single type of information source, such as only a map or only real-time sensor data, which may lead to incomplete or outdated information, thus affecting the accuracy and robustness of path planning. Therefore, how to improve path planning efficiency has become an urgent technical problem to be solved. Summary of the Invention

[0003] The main purpose of the present application is to provide a path planning method, apparatus, device, storage medium and product, aiming to solve the technical problem of low efficiency of existing path planning.

[0004] To achieve the above object, the present application proposes a path planning method, which includes:

[0005] Input multi-source data into a preset mixture-of-experts architecture model, wherein a plurality of expert modules and an expert fusion module are set in the preset mixture-of-experts model;

[0006] Extract features from various types of source data through the expert modules corresponding to various types of source data, and obtain a plurality of feature extraction results, wherein each expert module learns the feature information of the corresponding source data through pre-training;

[0007] Perform feature fusion on the plurality of feature extraction results through the expert fusion module to obtain a path planning result.

[0008] Optionally, before the step of inputting multi-source data into the preset mixture-of-experts architecture model, it further includes:

[0009] Obtain image information and lidar data;

[0010] Input the image information into a preset image understanding model to obtain text information corresponding to the image information;

[0011] Perform data preprocessing on the image information, the lidar data, and the text information to obtain multi-source data.

[0012] Optionally, the step of performing data preprocessing on the image information, the lidar data, and the text information to obtain multi-source data includes:

[0013] Perform plane fitting on the lidar data to obtain three-dimensional shape features;

[0014] Input the image information into a preset convolutional neural network to obtain image encoding features corresponding to the image information;

[0015] Extract features from the text information to obtain a one-dimensional semantic feature vector;

[0016] Determine multi-source data based on the image encoding features, the one-dimensional semantic feature vector, and the three-dimensional shape features.

[0017] Optionally, the expert module includes an adaptive gating network;

[0018] Before the step of inputting the multi-source data into the preset mixture-of-experts architecture model, it further includes:

[0019] Input multi-source sample data into the initial adaptive gating network for feature selection to obtain a feature selection result output by the initial adaptive gating network;

[0020] Adjust the weight matrix of the initial adaptive gating network based on the feature selection result and the multi-source sample data to obtain the adaptive gating network;

[0021] Correspondingly, the step of extracting features from various types of source data through the expert modules corresponding to the various types of source data to obtain multiple feature extraction results includes:

[0022] Perform feature selection on the various types of source data through the adaptive gating network of the expert modules corresponding to the various types of source data to obtain multiple feature extraction results.

[0023] Optionally, the expert module further includes a multi-head attention mechanism layer and a feed-forward layer module;

[0024] The step of performing feature selection on the various types of source data through the adaptive gating network of the expert modules corresponding to the various types of source data to obtain multiple feature extraction results includes:

[0025] Perform feature selection on the various types of source data through the adaptive gating network of the expert modules corresponding to the various types of source data to obtain multiple feature selection results;

[0026] Performing preliminary feature extraction on the feature selection results through the multi-head attention mechanism layer to obtain multiple preliminary feature extraction results;

[0027] Based on the forward layer module, the preliminary feature extraction result is nonlinearly transformed to obtain multiple feature extraction results.

[0028] Optionally, after the step of performing feature fusion on the multiple feature extraction results by the expert fusion module to obtain a path planning result, the method further includes:

[0029] Get the current position and speed trajectory of the moving target;

[0030] predicting motion trajectory information based on the current position and the speed trajectory;

[0031] determining a collision probability based on the motion trajectory information and the path planning result;

[0032] When the collision probability is greater than a preset collision threshold, the multi-source data is collected again, and the process returns to the step of inputting the multi-source data into the preset hybrid expert framework model.

[0033] Optionally, the step of performing feature fusion on the multiple feature extraction results by the expert fusion module to obtain a path planning result includes:

[0034] The expert fusion module performs feature fusion on the multiple feature extraction results using the following algorithm:

[0035]

[0036] in, is used to represent the path planning results, G is used to represent the softmax activation function, h is used to represent the number of hidden layers, and W i and b i are used to represent the weight and bias of the i-th hidden layer, It is used to characterize the feature extraction result, and s is used to characterize the activation function.

[0037] In addition, to achieve the above objectives, the present application also proposes a path planning device, which includes:

[0038] An input module, configured to input multi-source data into a preset hybrid expert architecture model, wherein the preset hybrid expert model is provided with a plurality of expert modules and an expert fusion module;

[0039] A feature extraction module for extracting features from various types of source data through expert modules corresponding to the various types of source data to obtain multiple feature extraction results, where each expert module learns the feature information of the corresponding source data through pre-training.

[0040] A feature fusion module for fusing the multiple feature extraction results through the expert fusion module to obtain a path planning result.

[0041] In addition, to achieve the above object, the present application also provides a path planning device, the device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program is configured to implement the steps of the path planning method as described above.

[0042] In addition, to achieve the above object, the present application also provides a storage medium, the storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium, and when the computer program is executed by a processor, it implements the steps of the path planning method as described above.

[0043] In addition, to achieve the above object, the present application also provides a computer program product, the computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps of the path planning method as described above.

[0044] The present application inputs multi-source data into a preset mixture-of-experts architecture model, where a plurality of expert modules and an expert fusion module are provided in the preset mixture-of-experts model; features of various types of source data are extracted through expert modules corresponding to the various types of source data to obtain multiple feature extraction results, where each expert module learns the feature information of the corresponding source data through pre-training; the multiple feature extraction results are fused through the expert fusion module to obtain a path planning result. Since the preset mixture-of-experts architecture model of the present application extracts features of various types of source data in multi-source data through expert modules corresponding to the various types of source data, and then fuses the multiple feature extraction results to obtain a path planning result, compared with the existing method of using a single source data for path planning, the above method of the present application can fuse the features of multi-source data, give play to the advantages of each source data for path planning, and improve the path planning efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0046] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0047] Figure 1 It is a schematic flowchart provided for the first embodiment of the path planning method of the present application;

[0048] Figure 2 It is a schematic flowchart provided for the second embodiment of the path planning method of the present application;

[0049] Figure 3 It is a schematic flowchart provided for the third embodiment of the path planning method of the present application;

[0050] Figure 4 It is a schematic diagram of the module structure of the path planning device in the embodiment of the present application;

[0051] Figure 5 It is a schematic diagram of the device structure of the hardware operating environment involved in the path planning method in the embodiment of the present application.

[0052] The realization of the purpose, functional features and advantages of the present application will be further described with reference to the accompanying drawings in combination with the embodiments. Specific Embodiments

[0053] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0054] To better understand the technical solutions of the present application, the following will be described in detail in combination with the accompanying drawings of the specification and the specific embodiments.

[0055] The main solution of the embodiment of the present application is to input multi-source data into a preset mixture-of-experts architecture model. Among them, multiple expert modules and an expert fusion module are set in the preset mixture-of-experts model; feature extraction is performed on various types of source data through the expert modules corresponding to various types of source data to obtain multiple feature extraction results. Among them, each expert module learns the feature information of the corresponding source data through pre-training; the expert fusion module performs feature fusion on the multiple feature extraction results to obtain a path planning result. Since the preset mixture-of-experts architecture model of the present application performs feature extraction on various types of source data through the expert modules corresponding to various types of source data in the multi-source data, and then performs feature fusion on the multiple feature extraction results to obtain a path planning result, compared with the existing method of using a single source data for path planning, the above method of the present application can fuse the features of multi-source data, give play to the advantages of each source data for path planning, and improve the path planning efficiency.

[0056] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, an intelligent robot, etc. that can implement the above functions. Hereinafter, an intelligent robot will be taken as an example to illustrate this embodiment and the following embodiments.

[0057] Based on this, an embodiment of the present application provides a path planning method. Referring to Figure 1 , Figure 1 is a schematic flowchart provided for the first embodiment of the path planning method of the present application.

[0058] In this embodiment, the path planning method includes the following steps:

[0059] Step S10, input multi-source data into a preset mixture-of-experts architecture model, where multiple expert modules and an expert fusion module are set in the preset mixture-of-experts model;

[0060] It should be noted that the multi-source data may be real-time scene data collected by various preset sensors installed on the intelligent robot. It may include image information of the surrounding environment collected by a vision sensor, lidar point cloud data, etc. It may also include text information corresponding to the image information output by the image recognition model obtained by recognizing the image information through the image recognition model. The image recognition model may include: a convolutional neural network (CNN), a support vector machine (SVM), a deep belief network (DBN), and a feature pyramid network (FPN), etc. that can recognize images and then describe in detail the scene layout, object categories, relationships between objects, and relative position information of the surrounding environment. The preset mixture-of-experts architecture model is pre-trained and is used to extract features from data of multiple sources, fuse the extracted features, and then predict an optimal planned path. The preset mixture-of-experts architecture model is a mixture-of-experts architecture, which includes multiple expert modules and a feature fusion module, and each expert module can learn high-semantic features of the corresponding data source. The expert fusion module is used to fuse the features learned by each expert module to obtain a final path planning result.

[0061] Step S20, extract features from various types of source data through the expert modules corresponding to various types of source data to obtain multiple feature extraction results, where each expert module learns the feature information of the corresponding source data through pre-training;

[0062] It should be noted that the feature extraction of various source data through the expert modules corresponding to various source data to obtain multiple feature extraction results can be to input the multi-source data into the multiple expert modules. The expert modules will learn the high-semantic features of the corresponding source data and output the feature extraction results. For example, the preset hybrid expert architecture model includes three expert modules. Expert module 1 can learn the high-semantic features of the image data source and then output the learning result, that is, the feature extraction result; expert module 2 can learn the high-semantic features of the lidar data source, and expert module 3 can learn the high-semantic features of the text data source.

[0063] Step S30, perform feature fusion on the multiple feature extraction results through the expert fusion module to obtain a path planning result.

[0064] It should be noted that the expert fusion module in the preset hybrid expert architecture model includes a multi-layer perceptron. The multi-layer perceptron includes an input layer, a hidden layer, and an output layer. The input layer has O neural units, the hidden layer includes P neural units, and the output layer includes N neural units. Among them, O is the dimension of the feature vector corresponding to the feature extraction result. Generally, P is greater than O, and N is the number of path planning coordinate values (that is, the path nodes to be output) to be output. The path planning result output by the output layer includes N path planning coordinate values. The network structure of the multi-layer perceptron is shown in the following formula:

[0065]

[0066] Among them, is used to represent the path planning result, G is used to represent the softmax activation function, h is used to represent the number of hidden layers, W i and b i are respectively used to represent the weight and bias of the i-th hidden layer. is used to represent the feature extraction result, and s is used to represent the activation function. Since G needs to output path planning coordinate values, it is preferably the softmax activation function, and s corresponds to the prediction step from the input layer to the hidden layer, and various activation functions can be used, which are not limited in this embodiment.

[0067] Furthermore, even if the path planning result is obtained, there may be moving objects on the planned path in the path planning result at any time. Therefore, in order to adapt to the rapidly changing environment, after step S30, it further includes: obtaining the current position and speed trajectory of the moving target;

[0068] Predict the motion trajectory information based on the current position and the speed trajectory;

[0069] Determine the collision probability according to the motion trajectory information and the path planning result;

[0070] When the collision probability is greater than a preset collision threshold, multi-source data is re-acquired, and the step of inputting the multi-source data into a preset mixture-of-experts architecture model is returned.

[0071] It should be noted that the current position may be the position where the moving target is currently located, the velocity trajectory may be the motion trajectory and motion velocity information of the moving target obtained through observation, and the moving target may be an object moving in the same environment as the intelligent robot. The motion trajectory information of the moving target may be to identify the moving target existing during the motion of the intelligent robot based on a deep learning-based object detection algorithm (YOLO series), and then obtain the running trajectory of the moving target according to the current position and velocity trajectory of the moving target. The determination of the collision probability according to the motion trajectory information and the path planning result may be to estimate the future moving trajectory of the moving target according to the motion trajectory information of the moving target, and determine whether there is an intersection between the moving trajectory and the planned path in the path planning result. If there is an intersection, then determine whether there is a collision risk according to the velocity information of the moving target and its own motion velocity, obtain the collision probability. When the collision probability is greater than the preset collision threshold, the possible collision position and the safety distance are evaluated. The safety distance may be the minimum distance threshold that can avoid a collision with the moving target. According to the collision position, the safety distance, and the multi-source data collected in real time, the path is re-planned. That is, multi-source data is re-acquired, and the step of inputting the multi-source data into a preset mixture-of-experts architecture model is returned. At this time, the re-acquired multi-source data may also include the motion trajectory information of the moving target, the collision position, the safety distance, and other information.

[0072] In this embodiment, multi-source data is input into a preset mixture-of-experts architecture model. Among them, multiple expert modules and an expert fusion module are set in the preset mixture-of-experts model; feature extraction is performed on various types of source data through the expert modules corresponding to the various types of source data, and multiple feature extraction results are obtained. Among them, each expert module learns the feature information of the corresponding source data through pre-training; the expert fusion module performs feature fusion on the multiple feature extraction results to obtain a path planning result. Since the preset mixture-of-experts architecture model in this embodiment performs feature extraction on various types of source data through the expert modules corresponding to the various types of source data in the multi-source data, and then performs feature fusion on the multiple feature extraction results to obtain a path planning result. Compared with the existing method of using a single source data for path planning, the above method in this embodiment can fuse the features of multi-source data, give full play to the advantages of each source data for path planning, and improve the path planning efficiency.

[0073] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar content as that in the above-mentioned first embodiment can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 2 ,Figure 2 This is a flowchart provided for the second embodiment of the path planning method of this application. The expert module includes an adaptive gating network;

[0074] Before the step S10, the following steps are further included:

[0075] Step S001: Input multi-source sample data into the initial adaptive gating network for feature selection to obtain the feature selection result output by the initial adaptive gating network;

[0076] It should be noted that each expert module is provided with an adaptive gating network (Gating Network), that is, the adaptive gating network. The adaptive gating network is used to perform feature selection on the source data corresponding to the expert module, so that the expert module can better learn the feature information of the corresponding data source. For example, the adaptive gating network in the expert module for learning (i.e., feature extraction) text data will perform feature selection on the text data in the multi-source data, so that the expert module for learning text data can perform better feature extraction on the text data. The multi-source sample data can be various data sources or data sources in various forms. For example, text data, image data, and lidar data. The input of the multi-source sample data into the initial adaptive gating network for feature selection can be: if the initial adaptive gating network is the initial adaptive gating network corresponding to the selection of text data, the feature selection result can be the text data selected by the initial adaptive gating network from the multi-source sample data.

[0077] Step S002: Adjust the weight matrix of the initial adaptive gating network based on the feature selection result and the multi-source sample data to obtain the adaptive gating network;

[0078] It should be noted that the adjustment of the weight matrix of the initial adaptive gating network based on the feature selection result and the multi-source sample data can be to perform backpropagation training on the initial adaptive gating network according to the feature selection result and the labeled information in the multi-source sample data, so as to adjust the weight matrix in the initial adaptive gating network, so that the initial adaptive gating network can better select the data of the corresponding data source, and then perform iterative training by calculating the loss value to obtain an adaptive gating network that meets the preset training conditions. The preset training conditions can be that the loss value is less than the pre-set loss value threshold. The adaptive gating network uses a fully connected layer to perform feature selection on the input multi-source data, and its formula is as follows:

[0079] G σ (x) = Softmax(W g *X + σ * ε)

[0080] where, W grepresents the weight matrix of the adaptive gate network, X represents the multi-source data, ε is a noise vector sampled from a standard Gaussian distribution, softmax is the activation function, and σ is a hyperparameter that controls the noise intensity.

[0081] Correspondingly, the feature extraction of various types of source data is performed by the expert modules corresponding to various types of source data, and multiple feature extraction results are obtained, including:

[0082] Feature selection of various types of source data is performed by the adaptive gate network of the expert modules corresponding to various types of source data, and multiple feature extraction results are obtained.

[0083] Further, the expert module further includes a multi-head attention mechanism layer and a forward layer module;

[0084] The step of performing feature selection on various types of source data by the adaptive gate network of the expert modules corresponding to various types of source data to obtain multiple feature extraction results includes:

[0085] Feature selection of various types of source data is performed by the adaptive gate network of the expert modules corresponding to various types of source data, and multiple feature selection results are obtained;

[0086] Preliminary feature extraction is performed on the feature selection results by the multi-head attention mechanism layer to obtain multiple preliminary feature extraction results;

[0087] Nonlinear transformation is performed on the preliminary feature extraction results based on the forward layer module to obtain multiple feature extraction results.

[0088] It should be noted that the preliminary feature extraction of the feature selection results by the multi-head attention mechanism layer to obtain multiple preliminary feature extraction results can refer to the following formula:

[0089] Q = G σ (x) * W Q

[0090] [[ID=3ed]]K = G σ (x) * W K

[0091] V = G σ (x) * W V

[0092]

[0093] Among them, W Q , W k , W v respectively represent the weight matrices of the query, key, and value of the multi-head attention mechanism layer, d kis the vector dimension. Q, K, and V respectively represent the results of high-semantic feature learning of the data through the multi-head attention mechanism layer. T is used to represent transpose, and F is used to characterize the preliminary feature extraction result.

[0094] It should be noted that the forward layer module consists of two fully connected layers and a residual connection. The residual connection directly adds the input to the output of the forward layer module. The fully connected layer includes a linear transformation and an activation function. In the linear transformation, the input (i.e., the preliminary feature extraction result) is multiplied by a weight matrix W1 and added with a bias b1, and then the result is input into the activation function for non-linear transformation. The activation function can be the rectified linear unit (ReLU) and max in the following formula. The calculation formula of the forward layer module is as follows:

[0095] FFN i (X) = τ i (max(0, F * W1 + b1)W2 + b2 + F)

[0096]

[0097] Among them, FFN i (X) is used to characterize the feature extraction result of the i-th expert module, is used to characterize all the feature extraction results corresponding to the multi-source data. i is used to characterize different data sources. W1 and W2 represent the weights of the forward module, b1 and b2 are the bias values of the forward module, and τ i represents the adaptive weight value of the i-th expert module.

[0098] In this embodiment, the multi-source sample data is input into the initial adaptive gate network for feature selection to obtain the feature selection result output by the initial adaptive gate network; based on the feature selection result and the multi-source sample data, the weight matrix of the initial adaptive gate network is adjusted to obtain the adaptive gate network; then, through the adaptive gate network of the expert module corresponding to each type of source data, feature selection is performed on each type of source data to obtain multiple feature extraction results. This enables different expert modules to learn the feature information of their respective sources, thereby adapting to different usage scenarios and better leveraging the advantages of different signal sources in different environments.

[0099] Based on the above embodiments of the present application, in the third embodiment of the present application, for the same or similar content as the above embodiments, reference can be made to the above introduction and will not be elaborated hereinafter. On this basis, please refer to Figure 3 , Figure 3 is the flow diagram provided for the third embodiment of the path planning method of the present application. Before the step S10, the following steps are further included:

[0100] Step S003: Obtain image information and lidar data;

[0101] It should be noted that the obtaining of the image information and lidar data may be to use a vision sensor to obtain the image information of the surrounding environment, and a lidar is used to emit laser pulses to the surrounding environment to obtain lidar data.

[0102] Step S004: Input the image information into a preset image understanding model to obtain the text information corresponding to the image information;

[0103] It should be noted that the preset image understanding model may be a neural network model that can recognize the object information in the image and obtain the relevant text description corresponding to the image. The text information corresponding to the image information may be information such as the scene layout, object categories, relationships between objects, and relative positions of the surrounding environment in the image.

[0104] Step S005: Perform data preprocessing on the image information, the lidar data, and the text information to obtain multi-source data.

[0105] It should be noted that the performing of data preprocessing on the image information, the lidar data, and the text information to obtain multi-source data may be to perform denoising and color correction on the image information, perform filtering and denoising on the lidar data, and perform feature extraction on the text information to obtain the key text information.

[0106] Further, in order to improve the path planning efficiency, step S005 may include: performing plane fitting on the lidar data to obtain three-dimensional shape features;

[0107] Input the image information into a preset convolutional neural network to obtain the image coding features corresponding to the image information;

[0108] Perform feature extraction on the text information to obtain a one-dimensional semantic feature vector;

[0109] Determine multi-source data according to the image coding features, the one-dimensional semantic feature vector, and the three-dimensional shape features.

[0110] It should be noted that the plane fitting of the lidar data to obtain the three-dimensional shape features may be to construct a three-dimensional point cloud map based on the lidar data, use techniques such as the least squares method to perform plane fitting on the points in the local area, so as to obtain the position information and shape information of the object, and use a 3D convolutional neural network to extract features from the obtained three-dimensional position information and shape information to obtain the three-dimensional shape feature encoding, that is, the three-dimensional shape features. The inputting the image information into a preset convolutional neural network to obtain the image encoding feature corresponding to the image information may be to use a preset convolutional neural network to extract features from the image information obtained by the visual sensor to obtain the image encoding feature. The preset convolutional neural network may be a deep learning model for processing structured grid data, especially images. The extracting features from the text information to obtain a one-dimensional semantic feature vector may be to use a text editor of the CLIP (Contrastive Language-Image Pre-training) model to extract features from the text information and the coordinate information of the object in the text information to obtain a one-dimensional semantic feature vector. The determining the multi-source data according to the image encoding feature, the one-dimensional semantic feature vector, and the three-dimensional shape feature may be to use the image encoding feature, the one-dimensional semantic feature vector, and the three-dimensional shape feature as the multi-source data.

[0111] This embodiment acquires image information and lidar data; inputs the image information into a preset image understanding model to obtain the text information corresponding to the image information; performs data preprocessing on the image information, the lidar data, and the text information to obtain multi-source data. By collecting image information and lidar data, and performing image understanding on the collected image information to obtain text information, and then obtaining multi-source data composed of image information, lidar data, and text information, this embodiment can improve the efficiency of path planning.

[0112] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the path planning method of this application. Based on this technical concept, more forms of simple transformations are within the protection scope of this application.

[0113] This application also provides a path planning device. Please refer to Figure 4 , the path planning device includes:

[0114] An input module 10, configured to input multi-source data into a preset mixture of experts architecture model, where multiple expert modules and an expert fusion module are set in the preset mixture of experts model;

[0115] The feature extraction module 20 is configured to extract features from various types of source data through the expert modules corresponding to the various types of source data, obtaining multiple feature extraction results, wherein each expert module learns the feature information of the corresponding source data through pre-training.

[0116] The expert fusion module 30 is configured to perform feature fusion on the multiple feature extraction results through the expert fusion module to obtain a path planning result.

[0117] In this embodiment, multiple source data are input into a preset mixture-of-experts architecture model, wherein multiple expert modules and an expert fusion module are provided in the preset mixture-of-experts model; features are extracted from various types of source data through the expert modules corresponding to the various types of source data, obtaining multiple feature extraction results, wherein each expert module learns the feature information of the corresponding source data through pre-training; and feature fusion is performed on the multiple feature extraction results through the expert fusion module to obtain a path planning result. Since the preset mixture-of-experts architecture model in this embodiment extracts features from various types of source data in the multiple source data through the expert modules corresponding to the various types of source data, and then performs feature fusion on the multiple feature extraction results to obtain a path planning result, compared with the existing method of using a single source data for path planning, the above method in this embodiment can fuse the features of multiple source data, give play to the advantages of each source data for path planning, and improve the path planning efficiency.

[0118] The path planning device provided in this application adopts the path planning method in the above embodiment, and can solve the technical problem of low existing path planning efficiency. Compared with the prior art, the beneficial effects of the path planning device provided in this application are the same as those of the path planning method provided in the above embodiment, and the other technical features in the path planning device are the same as those disclosed in the method of the above embodiment, and will not be elaborated here.

[0119] This application provides a path planning device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the path planning method in the first embodiment above.

[0120] Next, refer to Figure 5, which shows a schematic structural diagram of a path planning device suitable for implementing the embodiments of the present application. The path planning device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions: tablet computers), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The shown path planning device is merely an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0121] As Figure 5 shown, the path planning device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 into the random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the path planning device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the path planning device to communicate with other devices wirelessly or wiredly to exchange data. Although the path planning device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the shown systems. More or fewer systems may be alternatively implemented or had.

[0122] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product that includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by a processing device 1001, the above functions defined in the methods of the embodiments disclosed in the present application are executed.

[0123] The path planning device provided by the present application adopts the path planning method in the above embodiments, and can solve the technical problem of low efficiency of existing path planning. Compared with the prior art, the beneficial effects of the path planning device provided by the present application are the same as those of the path planning method provided by the above embodiments, and other technical features in the path planning device are the same as the features disclosed in the method of the previous embodiment, which will not be elaborated here.

[0124] It should be understood that the various parts disclosed in the present application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0125] As described above, the above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0126] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the path planning method in the above embodiments.

[0127] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0128] The above computer-readable storage medium can be included in the path planning device; it can also exist separately without being assembled into the path planning device.

[0129] The computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0130] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0131] The modules described in the embodiments of the present application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation on the unit itself in some cases.

[0132] The readable storage medium provided by the present application is a computer-readable storage medium, and the computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above path planning method, which can solve the technical problem of low efficiency of existing path planning. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as those of the path planning method provided by the above embodiments, and will not be elaborated here.

[0133] The present application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the steps of the path planning method as described above.

[0134] The computer program product provided by the present application can solve the technical problem of low efficiency of existing path planning. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as those of the path planning method provided by the above embodiments, and will not be elaborated here.

[0135] The above are only some embodiments of the present application, and do not limit the patent scope of the present application. All equivalent structural transformations made under the technical concept of the present application by using the content of the specification and drawings of the present application, or directly / indirectly applied to other related technical fields, are included in the patent protection scope of the present application.

Claims

1. A path planning method, characterized in that, The path planning method includes the following steps: Input multi-source data into a preset mixture-of-experts architecture model, where multiple expert modules and an expert fusion module are set in the preset mixture-of-experts model; Extract features from various types of source data through the expert modules corresponding to the various types of source data, obtaining multiple feature extraction results, where each expert module learns the feature information of the corresponding source data through pre-training; Perform feature fusion on the multiple feature extraction results through the expert fusion module to obtain a path planning result.

2. The path planning method according to claim 1, wherein Before the step of inputting multi-source data into the preset mixture-of-experts architecture model, it further includes: Obtain image information and lidar data; Input the image information into a preset image understanding model to obtain text information corresponding to the image information; Perform data preprocessing on the image information, the lidar data, and the text information to obtain multi-source data.

3. The path planning method according to claim 2, characterized in that, The step of performing data preprocessing on the image information, the lidar data, and the text information to obtain multi-source data includes: Perform plane fitting on the lidar data to obtain three-dimensional shape features; Input the image information into a preset convolutional neural network to obtain image encoding features corresponding to the image information; Extract features from the text information to obtain a one-dimensional semantic feature vector; Determine multi-source data according to the image encoding features, the one-dimensional semantic feature vector, and the three-dimensional shape features.

4. The path planning method according to claim 1, wherein The expert module includes an adaptive gating network; Before the step of inputting multi-source data into the preset mixture-of-experts architecture model, it further includes: Input multi-source sample data into an initial adaptive gating network for feature selection to obtain a feature selection result output by the initial adaptive gating network; Adjust the weight matrix of the initial adaptive gating network based on the feature selection result and the multi-source sample data to obtain the adaptive gating network; Correspondingly, the step of extracting features from various types of source data through the expert modules corresponding to the various types of source data to obtain multiple feature extraction results includes: Perform feature selection on the various types of source data through the adaptive gating networks of the expert modules corresponding to the various types of source data to obtain multiple feature extraction results.

5. The path planning method according to claim 4, characterized in that The expert module further includes a multi-head attention mechanism layer and a feed-forward layer module; The step of performing feature selection on the various types of source data through the adaptive gating networks of the expert modules corresponding to the various types of source data to obtain multiple feature extraction results includes: Perform feature selection on the various types of source data through the adaptive gating networks of the expert modules corresponding to the various types of source data to obtain multiple feature selection results; Perform preliminary feature extraction on the feature selection results through the multi-head attention mechanism layer to obtain multiple preliminary feature extraction results; Perform non-linear transformation on the preliminary feature extraction results based on the feed-forward layer module to obtain multiple feature extraction results.

6. The path planning method according to any one of claims 1-5, characterized in that, After the step of performing feature fusion on the multiple feature extraction results through the expert fusion module to obtain a path planning result, it further includes: Obtain the current position and speed trajectory of the moving target; Predict the motion trajectory information based on the current position and the velocity trajectory; Determine the collision probability according to the motion trajectory information and the path planning result; When the collision probability is greater than a preset collision threshold, re-collect multi-source data and return to the step of inputting the multi-source data into the preset mixture-of-experts architecture model.

7. The path planning method according to any one of claims 1-5, characterized in that, The step of obtaining the path planning result by performing feature fusion on the multiple feature extraction results through the expert fusion module includes: The expert fusion module performs feature fusion on the multiple feature extraction results through the following algorithm: Among them, is used to represent the path planning result, G is used to represent the softmax activation function, h is used to represent the number of hidden layers, W i and b i are respectively used to represent the weight and bias of the i-th hidden layer, is used to represent the feature extraction result, and s is used to represent the activation function.

8. A path planning device, characterized in that, The path planning device includes: An input module, configured to input multi-source data into a preset mixture-of-experts architecture model, wherein a plurality of expert modules and an expert fusion module are provided in the preset mixture-of-experts model; A feature extraction module, configured to perform feature extraction on various types of source data through the expert modules corresponding to the various types of source data to obtain a plurality of feature extraction results, wherein each expert module learns the feature information of the corresponding source data through pre-training; A feature fusion module, configured to perform feature fusion on the multiple feature extraction results through the expert fusion module to obtain a path planning result.

9. A path planning device, characterized in that, The device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the path planning method according to any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium, and when the computer program is executed by a processor, it implements the steps of the path planning method according to any one of claims 1 to 7.

11. A computer program product, characterized in that, The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps of the path planning method according to any one of claims 1 to 7.

Images