Unmanned navigation method and system based on three-dimensional convolution attention and vehicle

Abstract

The invention relates to an unmanned navigation method and system based on three-dimensional convolution attention and a vehicle, and the method comprises the steps: obtaining a three-dimensional raster image of a road, constructing a three-dimensional convolution attention network based on three-dimensional convolution and a self-attention mechanism, and obtaining a three-dimensional raster image of the road; performing three-dimensional convolution feature extraction on the road three-dimensional grid image through a multi-layer three-dimensional convolution attention network to obtain road convolution features; performing up-sampling processing on the road convolution features, and mapping to obtain up-sampled road convolution features; performing global adaptive three-dimensional pooling on the road convolution features after up-sampling to obtain target road convolution features after down-sampling; and performing classification according to the target road convolution features, and outputting to obtain a path planning map. The method can improve the feature extraction precision, reduces the calculation complexity, and guarantees the accuracy of the path planning diagram.

Description

Technical Field

[0001] The present disclosure relates to the field of unmanned driving technology, and in particular to an unmanned driving navigation method, system, vehicle, electronic device and computer-readable storage medium based on three-dimensional convolutional attention. Background Art

[0002] In related technologies, three-dimensional convolutional neural networks are widely used in three-dimensional raster data processing. Traditional convolutional neural networks are mainly used to process two-dimensional images, while three-dimensional convolutional neural networks can process volumetric data by extending the convolution operation to three-dimensional space.

[0003] When dealing with complex three-dimensional grid environments, such as unmanned driving and robot navigation, three-dimensional convolutional neural networks have the following problems:

[0004] First, the spatial information is not captured enough. The traditional 3D convolutional model lacks the ability to dynamically focus on important areas and is easily disturbed by irrelevant background information, resulting in limitations in feature extraction.

[0005] Second, the computational complexity is high. As the dimension of the input data increases, the computational and memory overhead of the 3D convolutional network also grows exponentially, resulting in low training efficiency.

[0006] Third, information fusion is insufficient. The traditional convolutional layer fails to effectively integrate features at different levels, making it difficult to learn data features from different perspectives, resulting in insufficient feature expression.

[0007] Therefore, there is an urgent need for an unmanned driving navigation method that can improve feature extraction accuracy, reduce computational complexity, and ensure path planning accuracy. Summary of the invention

[0008] In order to overcome the problems existing in the related art, the present disclosure provides an unmanned driving navigation method, system, vehicle, electronic device and computer-readable storage medium based on three-dimensional convolutional attention.

[0009] According to a first aspect of an embodiment of the present disclosure, there is provided an unmanned driving navigation method based on three-dimensional convolutional attention, comprising: obtaining a three-dimensional grid image of a road, constructing a three-dimensional convolutional attention network based on three-dimensional convolution and self-attention mechanism, performing three-dimensional convolution feature extraction on the three-dimensional grid image of the road through a multi-layer three-dimensional convolutional attention network to obtain road convolution features; upsampling the road convolution features to map the upsampled road convolution features; performing global adaptive three-dimensional pooling on the upsampled road convolution features to obtain downsampled target road convolution features; classifying according to the target road convolution features to output a path planning map.

[0010] In some embodiments, the three-dimensional convolution attention network is constructed based on the three-dimensional convolution and self-attention mechanism, and three-dimensional convolution features are extracted from the three-dimensional grid image of the road through the multi-layer three-dimensional convolution attention network to obtain the road convolution features, including:

[0011] A three-dimensional convolutional attention network is constructed based on three-dimensional convolution and self-attention mechanisms, wherein the three-dimensional convolutional attention network includes three three-dimensional convolution modules, an average three-dimensional pooling module, a self-attention module, and a parameter weighting module;

[0012] Inputting the road three-dimensional grid image into a three-dimensional convolutional attention network, and performing feature extraction on the road three-dimensional grid image through the three-dimensional convolution module;

[0013] Assume that the shape of the input image is Wherein, B is the batch size, C is the number of input channels, D, H and W are the depth, height and width of the three-dimensional grid respectively, and the input image is subjected to sliding weighted summation by the three-dimensional convolution kernel of the three-dimensional convolution module, and the shape of the three-dimensional convolution kernel is Among them, C out is the number of output channels, C in is the number of input channels, D k ,H k ,W k The depth, height and width of the three-dimensional convolution kernel are respectively used to generate the output feature map. The formula is:

[0014]

[0015] Where Y b,c,d,h,w is the value of the output feature map at position (d, h, w), Z c,c',d',h',w' is the weight of the three-dimensional convolution kernel, X b,c',d,h,w is the value of the input data;

[0016] The average three-dimensional pooling module is used to average pool the output feature maps of the three three-dimensional convolution modules to obtain the pooled convolution features. The pooled convolution features include query, key and value. The formula is:

[0017] Q = Conv3D Q (X),K=Conv3D K (X),V=Conv3D V (X);

[0018] Where Q represents query, K represents key, V represents value, and Conv3D Q 、Conv3D K and Conv3D V Represent the three-dimensional convolution of Q, K and V respectively;

[0019] The pooled convolutional features are input into the self-attention module, and the dot product between the query and the key is calculated to obtain the attention matrix, which is:

[0020] attn = softmax(Q·K T );

[0021] In the formula, softmax represents the softmax normalization function, attn represents the attention matrix;

[0022] The attention matrix and the value are weighted and summed by the parameter weighting module to obtain the output feature, and the formula is:

[0023] output = attn·V;

[0024] In the formula, output represents the output feature;

[0025] The output feature is used as the input of the next three-dimensional convolutional attention network, and three-dimensional convolution processing is performed on it until the last three-dimensional convolutional attention network is processed, and the road convolution feature is output.

[0026] In some embodiments, before inputting the road three-dimensional grid image into the three-dimensional convolutional attention network and extracting features from the road three-dimensional grid image through the three-dimensional convolution module, the method further includes: initializing the weight of the three-dimensional convolution kernel using an orthogonal initialization method, and the formula is:

[0027]

[0028] In the formula, Z init represents the weight of the three-dimensional convolution kernel after initialization, and Z represents the weight of the three-dimensional convolution kernel before initialization.

[0029] In some embodiments, the weighted summing of the attention matrix and the value by the parameter weighting module to obtain the output feature includes: introducing a weighted parameter by the parameter weighting module, weighted summing of the output feature and the final output to obtain the output feature, and the formula is:

[0030] final output =γ·output+(1-γ)·V;

[0031] Where γ represents a learnable weighting parameter, which is used to adjust the weight ratio between the final output of the attention mechanism and the original convolution output feature.

[0032] In some embodiments, upsampling the road convolution feature to map the upsampled road convolution feature includes: upsampling the road convolution feature by an interpolation method, and the formula is:

[0033] output=F.interpolate(output,size=(D,H,W),mode='trilinear');

[0034] Where, F.interpolate represents the upsampling function;

[0035] The road convolution feature is mapped to a three-dimensional grid whose depth, height and width are D, H and W respectively through upsampling, so as to obtain the upsampled road convolution feature.

[0036] In some embodiments, the classification is performed according to the target road convolution features, and a path planning map is outputted, including: processing the target road convolution features through an activation function; inputting the activated target road convolution features into a fully connected layer for classification, identifying and marking obstacles and roads in the road, and outputting a path planning map.

[0037] According to a second aspect of an embodiment of the present disclosure, an unmanned driving navigation system based on three-dimensional convolutional attention is provided, comprising: an input layer, a three-dimensional convolutional attention layer, an upsampling layer, a global adaptive three-dimensional pooling layer, a fully connected layer and an output layer connected in sequence; the input layer is used to obtain a three-dimensional grid image of a road and input it into the multiple three-dimensional convolutional attention layers; the three-dimensional convolutional attention layer comprises a multi-layer three-dimensional convolutional attention network, which is used to perform three-dimensional convolutional feature extraction on the three-dimensional grid image of the road to obtain road convolutional features, and the three-dimensional convolutional attention network is constructed based on three-dimensional convolution and self-attention mechanism; the upsampling layer is used to perform upsampling processing on the road convolutional features and map to obtain upsampled road convolutional features; the globally adaptive three-dimensional pooling layer is used to perform global adaptive three-dimensional pooling on the upsampled road convolutional features to obtain downsampled target road convolutional features; the fully connected layer is used to classify according to the target road convolutional features to obtain a classification result; the output layer is used to output a path planning map according to the classification result.

[0038] According to a third aspect of an embodiment of the present disclosure, a vehicle is provided, storing a set of instruction sets, which are executed by the vehicle to implement the unmanned driving navigation method based on three-dimensional convolutional attention provided by the first aspect of the present disclosure.

[0039] According to the fourth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the unmanned driving navigation method based on three-dimensional convolutional attention provided in the first aspect of the present disclosure.

[0040] According to a fifth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the unmanned driving navigation method based on three-dimensional convolutional attention provided in the first aspect of the present disclosure are implemented.

[0041] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: by acquiring a three-dimensional grid image of a road, a three-dimensional convolution attention network is constructed based on three-dimensional convolution and self-attention mechanism, three-dimensional convolution feature extraction is performed on the three-dimensional grid image of the road through a multi-layer three-dimensional convolution attention network to obtain road convolution features, and by introducing an attention mechanism, attention is paid to key information areas, thereby improving the accuracy of feature extraction of three-dimensional grid data; upsampling the road convolution features is performed to map the upsampled road convolution features; global adaptive three-dimensional pooling is performed on the upsampled road convolution features to obtain downsampled target road convolution features, and through the combination of pooling and self-attention mechanism, redundant calculations are effectively reduced and training efficiency is improved; classification is performed according to the target road convolution features, and a path planning map is output, thereby improving the stability and generalization ability of the network, and improving the path planning and obstacle detection capabilities, thereby ensuring the accuracy of the path planning map.

[0042] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0044] Figure 1 It is a flowchart of an unmanned driving navigation method based on three-dimensional convolutional attention according to an exemplary embodiment.

[0045] Figure 2 It is a framework diagram of an unmanned driving navigation method based on three-dimensional convolutional attention according to an exemplary embodiment.

[0046] Figure 3 The figure is a flow chart of a road convolution feature extraction method according to an exemplary embodiment.

[0047] Figure 4The figure is a flow chart of a method for outputting a path planning graph according to an exemplary embodiment.

[0048] Figure 5 A path planning diagram is shown according to an exemplary embodiment.

[0049] Figure 6 It is a block diagram of a drone driving navigation system based on a three-dimensional convolutional neural network according to an exemplary embodiment.

[0050] Figure 7 is a block diagram of a vehicle according to an exemplary embodiment.

[0051] Figure 8 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0052] Exemplary embodiments will be described in detail below with reference to the accompanying drawings.

[0053] It should be pointed out that the relevant embodiments and drawings are only for describing exemplary embodiments provided by the present disclosure, rather than all embodiments of the present disclosure, and it should not be understood that the present disclosure is limited to the relevant exemplary embodiments.

[0054] It should be noted that the terms "first", "second", etc. used in the present disclosure are only used to distinguish different steps, devices or modules, etc. The related terms neither represent any specific technical meanings nor indicate the order or interdependence between them.

[0055] It should be noted that the modifications of the terms "one", "multiple", and "at least one" used in the present disclosure are illustrative rather than restrictive. Unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0056] It should be noted that the term "and / or" used in this disclosure is used to describe the association relationship between associated objects, and generally indicates that there are at least three association relationships. For example, A and / or B can at least indicate the existence of three association relationships: A exists alone, A and B exist at the same time, and B exists alone.

[0057] It should be noted that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. Unless otherwise specified, the scope of the present disclosure is not limited by the order of description of the steps in the relevant embodiments.

[0058] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0059] Exemplary Methods

[0060] Figure 1 is a flowchart of an unmanned driving navigation method based on three-dimensional convolutional attention according to an exemplary embodiment. Figure 1 As shown, the unmanned driving navigation method based on three-dimensional convolutional attention is used in unmanned driving and robot navigation, including the following steps:

[0061] In step S110, a three-dimensional grid image of the road is obtained, and a three-dimensional convolutional attention network is constructed based on three-dimensional convolution and self-attention mechanism. Three-dimensional convolutional features are extracted from the three-dimensional grid image of the road through the multi-layer three-dimensional convolutional attention network to obtain road convolution features.

[0062] In some embodiments, Figure 2 As shown in the figure, it is an architecture diagram of the unmanned driving navigation method based on three-dimensional convolutional attention. Road condition images are collected and processed to obtain three-dimensional grid images of the road. A three-dimensional convolutional attention network is constructed based on three-dimensional convolution and self-attention mechanisms. The three-dimensional grid images of the road are input into the three-dimensional convolutional attention network. The three-dimensional convolutional features of the three-dimensional grid images of the road are extracted through the multi-layer three-dimensional convolutional attention network to obtain road convolution features. The three-dimensional convolution and self-attention mechanisms in the three-dimensional convolutional attention network can effectively process three-dimensional grid data, extract spatial and depth information, and enhance the expressiveness of the model and its learning ability for key areas.

[0063] In step S120, the road convolution features are upsampled and mapped to obtain upsampled road convolution features.

[0064] In some embodiments, since the output features tend to become smaller after extracting features through the three-dimensional convolutional attention network, the road convolution features can be upsampled, mapped to obtain the upsampled road convolution features, and the image can be restored to its original size to facilitate subsequent operations.

[0065] In step S130, globally adaptive three-dimensional pooling is performed on the up-sampled road convolution features to obtain down-sampled target road convolution features.

[0066] In some embodiments, in order to reduce the amount of calculation, the upsampled road convolution features are globally adaptively pooled to obtain the downsampled target road convolution features. Global pooling can greatly reduce the amount of calculation, making the model more lightweight and preventing overfitting. It can also focus on the overall features of the entire channel, which is conducive to extracting all features.

[0067] In step S140, classification is performed according to the convolutional features of the target road, and a path planning map is output.

[0068] In some embodiments, the pooled convolutional features of the target road are integrated to form higher-level semantic information, which is classified through the integrated features to output a high-precision path planning map, so that unmanned driving and robot navigation can be performed based on the path planning map, thereby ensuring the accuracy of path planning.

[0069] In the above embodiment, by acquiring a three-dimensional grid image of the road, a three-dimensional convolution attention network is constructed based on three-dimensional convolution and self-attention mechanism, and three-dimensional convolution feature extraction is performed on the three-dimensional grid image of the road through a multi-layer three-dimensional convolution attention network to obtain road convolution features. By introducing the attention mechanism, attention is paid to key information areas, and the accuracy of three-dimensional grid data feature extraction is improved; the road convolution features are upsampled and mapped to obtain upsampled road convolution features; the upsampled road convolution features are globally adaptively pooled to obtain downsampled target road convolution features, and through the combination of pooling and self-attention mechanism, redundant calculations are effectively reduced and training efficiency is improved; classification is performed according to the target road convolution features, and a path planning map is output, which improves the stability and generalization ability of the network, and improves the path planning and obstacle detection capabilities, thereby ensuring the accuracy of the path planning map.

[0070] Figure 3 is a flow chart of a road convolution feature extraction method according to an exemplary embodiment. Figure 3 As shown, the road convolution feature extraction method is used in step S110, including the following steps:

[0071] In step S210, a three-dimensional convolutional attention network is constructed based on three-dimensional convolution and self-attention mechanisms. The three-dimensional convolutional attention network includes three three-dimensional convolution modules, an average three-dimensional pooling module, a self-attention module and a parameter weighting module.

[0072] Specifically, a three-dimensional convolutional attention network is constructed based on three-dimensional convolution and self-attention mechanism. The structure of the three-dimensional convolutional attention network is as follows: Figure 2 As shown in the figure, the 3D convolutional attention network includes three 3D convolution modules, an average 3D pooling module, a self-attention module and a parameter weighting module. Feature extraction is performed through the 3D convolution operation of the 3D convolutional attention network, and the extracted features are weighted summed based on the self-attention mechanism, thereby enhancing the network's learning ability for key areas.

[0073] In step S220, the road 3D grid image is input into the 3D convolutional attention network, and features of the road 3D grid image are extracted through the 3D convolution module. Assuming that the shape of the input image is Among them, B is the batch size, C is the number of input channels, D, H and W are the depth, height and width of the three-dimensional grid respectively. The input image is subjected to sliding weighted summation by the three-dimensional convolution kernel of the three-dimensional convolution module. The shape of the three-dimensional convolution kernel is Among them, C out is the number of output channels, C in is the number of input channels, D k ,H k ,W k The depth, height and width of the three-dimensional convolution kernel are respectively used to generate the output feature map. The formula is:

[0074]

[0075] Where Y b,c,d,h,w is the value of the output feature map at position (d, h, w), Z c,c',d',h',w' is the weight of the three-dimensional convolution kernel, X b,c',d,h,w is the value of the input data.

[0076] Specifically, the three-dimensional convolution operation can effectively extract spatial features in depth, height, and width, and can also extract rich multi-channel information from the input data. Through the design of the convolution kernel, different spatial information can be extracted and passed to subsequent network layers.

[0077] Before step S220, the method further includes: initializing the three-dimensional convolution kernel weights using an orthogonal initialization method, the formula being:

[0078]

[0079] In the formula, Z init represents the weight of the three-dimensional convolution kernel after initialization, and Z represents the weight of the three-dimensional convolution kernel before initialization.

[0080] Specifically, the orthogonal initialization method is used to initialize the weights of the three-dimensional convolution kernel to avoid the explosion or disappearance of the output loss gradient of the activation function during the forward propagation of the convolutional network, thereby accelerating the convergence of network training.

[0081] In step S230, an average three-dimensional pooling module is used to average pool the output feature maps of the three three-dimensional convolution modules to obtain pooled convolution features. The pooled convolution features include query, key and value, and the formula is:

[0082] Q = Conv3D Q (X),K=Conv3D K (X),V=Conv3D V (X);

[0083] Where Q represents query, K represents key, V represents value, and Conv3DQ 、Conv3D K and Conv3D V Represent the three-dimensional convolution of Q, K and V respectively.

[0084] Specifically, the three 3D convolution modules perform 3D convolution operations on the input features respectively, and output three features Q, K, and V. The average 3D pooling module is used to average pool the output features to obtain the pooled convolution features, reduce the size of the feature map and retain the input information, thereby eliminating pixel-level noise and improving the robustness of the model.

[0085] In step S240, the pooled convolutional features are input into the self-attention module, and the dot product between the query and the key is calculated to obtain the attention matrix, which is:

[0086] attn = softmax(Q·K T );

[0087] Where softmax represents the softmax normalization function and attn represents the attention matrix.

[0088] Specifically, the pooled convolutional features are input into the self-attention module, and the input convolutional features are processed through the self-attention mechanism to calculate the dot product between the query and the key to obtain the attention matrix, thereby effectively reducing redundant calculations and improving training efficiency.

[0089] In step S250, the attention matrix and the value are weighted and summed by the parameter weighting module to obtain the output feature, and the formula is:

[0090] output = attn·V;

[0091] In the formula, output represents the output feature.

[0092] Specifically, the attention matrix and value are weighted and summed through the parameter weighting module to obtain the output features, realize dynamic adjustment of the weight of each position, and enhance the network's learning ability for key areas, so that the network can focus more on key areas and ignore irrelevant background information.

[0093] Wherein, step S250 includes: introducing weighted parameters through a parameter weighting module, performing weighted summation on the output features and the final output, and obtaining the output features, the formula is:

[0094] final output =γ·output+(1-γ)·V;

[0095] In the formula, γ represents a learnable weighting parameter, which adjusts the weight ratio between the final output of the attention mechanism and the original convolution output features.

[0096] Specifically, between the final output features of the three-dimensional convolutional attention network and the convolution output features, weighted parameters are introduced through the parameter weighting module for weighted summation to obtain the final output features, so that the network can learn a dynamically adjusted weight coefficient and adjust the weight ratio between the attention mechanism output and the original convolution output through training optimization.

[0097] In step S260, the output feature is used as the input of the next three-dimensional convolutional attention network, and three-dimensional convolution processing is performed on it until the last three-dimensional convolutional attention network is processed and the road convolution feature is output.

[0098] Specifically, since multiple layers of three-dimensional convolutional attention networks are set up, after the previous layer of three-dimensional convolutional attention network completes convolution of the input features to obtain the output features, the output features are used as the input of the next three-dimensional convolutional attention network, and three-dimensional convolution processing is performed again until the last three-dimensional convolutional attention network is processed, and the road convolution features are output, so that the three-dimensional raster data can automatically focus on the key information area and improve the feature extraction effect.

[0099] In this embodiment, a three-dimensional convolution and self-attention mechanism is combined to construct a three-dimensional convolution attention network, including three three-dimensional convolution modules, an average three-dimensional pooling module, a self-attention module and a parameter weighting module. The three-dimensional grid image of the road is input into the three-dimensional convolution attention network, and features are extracted. The input features are sliding weighted summed through the three-dimensional convolution module to generate an output feature map; the output feature map is averaged by the average three-dimensional pooling module to obtain the pooled convolution features, the attention matrix is ​​calculated by the self-attention module, and the output features are obtained by weighted summation through the parameter weighting module. The output features are used as the input of the next three-dimensional convolution attention network, and three-dimensional convolution processing is performed until the last three-dimensional convolution attention network is processed and the road convolution features are output, thereby realizing the combination of three-dimensional convolution operation and self-attention, so that the model can focus on key information areas in three-dimensional grid data, improve feature extraction accuracy, and dynamically adjust feature weighting, so as to more accurately capture important feature information in space.

[0100] Figure 4 is a flow chart of a method for outputting a path planning graph according to an exemplary embodiment. Figure 4 As shown, the path planning graph output method is used in step S140, including the following steps:

[0101] In step S310, the target road convolution feature is processed by an activation function.

[0102] Specifically, the convolutional features of the target road are processed through an activation function, such as the ReLU activation function, so as to introduce nonlinear factors, so that the network can learn and represent complex nonlinear relationships. The neural network can learn different features, thereby improving the generalization ability of the model and avoiding the problem of gradient disappearance.

[0103] In step S320, the activated target road convolution features are input into the fully connected layer for classification, obstacles and roads in the road are identified and marked, and a path planning map is output.

[0104] Specifically, the convolutional features of the target road after activation processing are input into the fully connected layer for classification, and the obstacles and roads on the driving road are identified and marked. After marking, the path planning map is output, which can be used for unmanned driving and robot navigation to improve path planning and obstacle detection capabilities.

[0105] like Figure 5 As shown in the figure, the gray squares are obstacles, the red dots are starting points, the purple lines are paths, and the green dots are destinations. Figure (a) is a relatively global map in the path planning scenario, and Figure (b) is a local map of the local environment where the vehicle is currently located. The orange shadow represents the heat map, which is the area of ​​the current local environment that needs attention. Obviously, in the three-dimensional grid path planning task, the model pays more attention to the key decision-making parts, and the parts that the model pays attention to are marked with heat maps.

[0106] In this embodiment, the target road convolution features are processed by an activation function, and the activated target road convolution features are input into the fully connected layer for classification, and obstacles and roads in the road are identified and marked, and a path planning map is output, so that the network can learn and represent complex nonlinear relationships. The obtained path planning map can be used for unmanned driving and robot navigation, and the path planning and obstacle detection capabilities are improved, thereby ensuring the accuracy of the path planning map.

[0107] Exemplary Systems

[0108] Figure 6 is a block diagram of an unmanned driving navigation system based on three-dimensional convolutional attention according to an exemplary embodiment. Figure 6The system 600 includes an input layer 610, multiple three-dimensional convolutional attention layers 620, an upsampling layer 630, a global adaptive three-dimensional pooling layer 640, a fully connected layer 650 and an output layer 660 connected in sequence; the input layer 610 is used to obtain a three-dimensional grid image of the road and input it into the multiple three-dimensional convolutional attention layers 620; the three-dimensional convolutional attention layer 620 is used to include a multi-layer three-dimensional convolutional attention network, which is used to extract three-dimensional convolutional features of the three-dimensional grid image of the road to obtain road convolutional features, and the three-dimensional convolutional attention network is constructed based on three-dimensional convolution and self-attention mechanism; the upsampling layer 630 is used to upsample the road convolution features and map the upsampled road convolution features; the global adaptive three-dimensional pooling layer 640 is used to perform global adaptive three-dimensional pooling on the upsampled road convolution features to obtain the downsampled target road convolution features; the fully connected layer 650 is used to classify according to the target road convolution features to obtain the classification results; the output layer 660 is used to output a path planning map according to the classification results.

[0109] In one embodiment, the three-dimensional convolutional attention layer 620 is specifically used to: construct a three-dimensional convolutional attention network based on three-dimensional convolution and self-attention mechanisms, and the three-dimensional convolutional attention network includes three three-dimensional convolution modules, an average three-dimensional pooling module, a self-attention module and a parameter weighting module; input the three-dimensional grid image of the road into the three-dimensional convolutional attention network, and extract features of the three-dimensional grid image of the road through the three-dimensional convolution module; perform sliding weighted summation on the input image through the three-dimensional convolution kernel of the three-dimensional convolution module to generate an output feature map; use the average three-dimensional pooling module to average pool the output feature maps of the three three-dimensional convolution modules to obtain pooled convolution features, and the pooled convolution features include query, key and value; input the pooled convolution features into the self-attention module, calculate the dot product between the query and the key, and obtain the attention matrix; perform weighted summation on the attention matrix and the value through the parameter weighting module to obtain the output feature; use the output feature as the input of the next three-dimensional convolutional attention network, and perform three-dimensional convolution processing on it until the last three-dimensional convolutional attention network is processed, and the road convolution feature is output.

[0110] In one embodiment, the fully connected layer 650 is specifically used to: process the target road convolution features through an activation function; input the activated target road convolution features into the fully connected layer for classification, identify and mark obstacles and roads in the road, and output a path planning map.

[0111] In the above embodiment, through the operation of the system, an unmanned driving navigation method based on three-dimensional convolutional attention is realized, ensuring that the system can realize all the contents of an unmanned driving navigation method based on three-dimensional convolutional attention.

[0112] Example Vehicles

[0113] Figure 7 is a block diagram of a vehicle 800 according to an exemplary embodiment. The vehicle 800 may be a fuel vehicle, a hybrid vehicle, an electric vehicle, a fuel cell vehicle or other types of vehicles.

[0114] Reference Figure 7 The vehicle 800 may include multiple subsystems, for example, a driving system 810, a control system 820, a perception system 830, a communication system 840, an information display system 850, and a computing system 860. The vehicle 800 may also include more or fewer subsystems, and each subsystem may also include multiple components, which are not described one by one here.

[0115] The driving system 810 includes components that provide power movement for the vehicle 800, such as an engine, an energy source, a transmission device, etc.

[0116] The control system 820 includes components that provide control for the vehicle 800, such as vehicle control, cockpit equipment control, and driving assistance control.

[0117] The perception system 830 includes components that provide the vehicle 800 with surrounding environment perception, such as a vehicle positioning system, a laser sensor, a voice sensor, an ultrasonic sensor, a camera device, and the like.

[0118] The communication system 840 includes components that provide communication connections for the vehicle 800, such as mobile communication networks (such as 3G, 4G, 5G networks, etc.), WiFi, Bluetooth, and Internet of Vehicles.

[0119] The information display system 850 includes components that provide various information displays for the vehicle 800, such as vehicle information display, navigation information display, entertainment information display, etc.

[0120] The computing and processing system 860 includes components that provide data computing and processing capabilities for the vehicle 800. The computing and processing system 860 may include at least one processor 861 and a memory 862. The processor 861 may execute instructions stored in the memory 862.

[0121] The processor 861 may be any conventional processor, such as a commercially available CPU. The processor may also include a graphics processor (Graphic Process Unit, GPU), a field programmable gate array (Field Programmable Gate Array, FPGA), a system on chip (System on Chip, SOC), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC) or a combination thereof.

[0122] Memory 862 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0123] In an embodiment of the present disclosure, a set of instruction sets is stored in the memory 862, and the processor 861 can execute the instruction set to implement all or part of the steps of the unmanned driving navigation method based on three-dimensional convolutional attention described in any of the above exemplary embodiments.

[0124] Exemplary Electronic Devices

[0125] Figure 8 1 is a block diagram of an electronic device 900 according to an exemplary embodiment. The electronic device 900 may be a vehicle controller, a vehicle terminal, a vehicle computer or other types of electronic devices.

[0126] Reference Figure 8 , the electronic device 900 may include at least one processor 910 and a memory 920. The processor 910 may execute instructions stored in the memory 920. The processor 910 is communicatively connected to the memory 920 via a data bus. In addition to the memory 920, the processor 910 may also be communicatively connected to an input device 930, an output device 940, and a communication device 950 via a data bus.

[0127] The processor 910 may be any conventional processor, such as a commercially available CPU. The processor may also include a graphics processor (Graphic Process Unit, GPU), a field programmable gate array (Field Programmable Gate Array, FPGA), a system on chip (System on Chip, SOC), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC) or a combination thereof.

[0128] The memory 920 may be implemented by any type of volatile or nonvolatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0129] In an embodiment of the present disclosure, executable instructions are stored in the memory 920, and the processor 910 can read the executable instructions from the memory 920 and execute the instructions to implement all or part of the steps of the unmanned driving navigation method based on three-dimensional convolutional attention described in any of the above exemplary embodiments.

[0130] Exemplary computer-readable storage media

[0131] In addition to the above methods and systems, the exemplary embodiments of the present disclosure may also be a computer program product or a computer-readable storage medium storing the computer program product. The computer product includes computer program instructions that can be executed by a processor to implement all or part of the steps described in any method in the above exemplary embodiments.

[0132] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and also conventional procedural programming languages, such as "C" language or similar programming languages ​​and scripting languages ​​(e.g., Python). The program code may be executed entirely on the user computing device, partially on the user computing device, as an independent software package, partially on the user computing device and partially on the remote computing device, or entirely on the remote computing device or server.

[0133] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of readable storage media include: a static random access memory (SRAM) with one or more wires electrically connected, an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk or an optical disk, or any suitable combination of the above.

[0134] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0135] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An unmanned driving navigation method based on three-dimensional convolutional attention, characterized in that: The following steps are involved: Acquire a three-dimensional grid image of a road, construct a three-dimensional convolution attention network based on three-dimensional convolution and self-attention mechanism, and extract three-dimensional convolution features of the three-dimensional grid image of the road through the multi-layer three-dimensional convolution attention network to obtain road convolution features; Performing upsampling processing on the road convolution feature, and mapping to obtain the upsampled road convolution feature; Perform global adaptive 3D pooling on the upsampled road convolution features to obtain the downsampled target road convolution features; Classification is performed according to the convolutional features of the target road, and a path planning map is output.

2. The unmanned driving navigation method based on three-dimensional convolutional attention according to claim 1 is characterized in that: The three-dimensional convolution attention network is constructed based on the three-dimensional convolution and self-attention mechanism, and the three-dimensional convolution feature extraction is performed on the three-dimensional grid image of the road through the multi-layer three-dimensional convolution attention network to obtain the road convolution feature, including: A three-dimensional convolutional attention network is constructed based on three-dimensional convolution and self-attention mechanisms, wherein the three-dimensional convolutional attention network includes three three-dimensional convolution modules, an average three-dimensional pooling module, a self-attention module, and a parameter weighting module; Inputting the road three-dimensional grid image into a three-dimensional convolutional attention network, and performing feature extraction on the road three-dimensional grid image through the three-dimensional convolution module; Assume that the shape of the input image is Wherein, B is the batch size, C is the number of input channels, D, H and W are the depth, height and width of the three-dimensional grid respectively, and the input image is subjected to sliding weighted summation by the three-dimensional convolution kernel of the three-dimensional convolution module, and the shape of the three-dimensional convolution kernel is Among them, C out is the number of output channels, C in is the number of input channels, D k ,H k ,W k The depth, height and width of the three-dimensional convolution kernel are respectively used to generate the output feature map. The formula is: Where Y b,c,d,h,w is the value of the output feature map at position (d, h, w), Z c,c',d',h',w' is the weight of the three-dimensional convolution kernel, X b,c',d,h,w is the value of the input data; The average three-dimensional pooling module is used to average pool the output feature maps of the three three-dimensional convolution modules to obtain the pooled convolution features. The pooled convolution features include query, key and value. The formula is: Q=Conv3D Q (X),K=Conv3D K (X),V=Conv3D V (X); Where Q represents query, K represents key, V represents value, and Conv3D Q 、Conv3D K and Conv3D V Represent the three-dimensional convolution of Q, K and V respectively; The pooled convolutional features are input into the self-attention module, and the dot product between the query and the key is calculated to obtain the attention matrix, which is: attn=softmax(Q·K T ); In the formula, softmax represents the softmax normalization function, attn represents the attention matrix; The attention matrix and the value are weighted and summed by the parameter weighting module to obtain the output feature, and the formula is: output = attn·V; In the formula, output represents the output feature; The output feature is used as the input of the next three-dimensional convolutional attention network, and three-dimensional convolution processing is performed on it until the last three-dimensional convolutional attention network is processed, and the road convolution feature is output.

3. The unmanned driving navigation method based on three-dimensional convolutional attention according to claim 2 is characterized in that: Before inputting the road three-dimensional grid image into the three-dimensional convolutional attention network and performing feature extraction on the road three-dimensional grid image through the three-dimensional convolution module, the method further includes: The weight of the three-dimensional convolution kernel is initialized using an orthogonal initialization method, and the formula is: In the formula, Z init represents the weight of the three-dimensional convolution kernel after initialization, and Z represents the weight of the three-dimensional convolution kernel before initialization.

4. The unmanned driving navigation method based on three-dimensional convolutional attention according to claim 2 is characterized in that: The weighted summing of the attention matrix and the value by the parameter weighting module to obtain the output feature includes: The weighted parameters are introduced through the parameter weighting module, and the output features and the final output are weighted and summed to obtain the output features. The formula is: final output =γ·output+(1-γ)·V; Where γ represents a learnable weighting parameter, which is used to adjust the weight ratio between the final output of the attention mechanism and the original convolution output feature.

5. The unmanned driving navigation method based on three-dimensional convolutional attention according to claim 2 is characterized in that: The upsampling of the road convolution features to map the upsampled road convolution features includes: The road convolution feature is upsampled by an interpolation method, and the formula is: output=F.interpolate(output,size=(D,H,W),mode='trilinear'); Where, F.interpolate represents the upsampling function; The road convolution feature is mapped to a three-dimensional grid whose depth, height and width are D, H and W respectively through upsampling, so as to obtain the upsampled road convolution feature.

6. The unmanned driving navigation method based on three-dimensional convolutional attention according to claim 1 is characterized in that: The method of classifying the target road convolution features and outputting a path planning graph includes: Processing the target road convolution feature through an activation function; The activated convolutional features of the target road are input into the fully connected layer for classification, obstacles and roads in the road are identified and marked, and the path planning map is output.

7. An unmanned driving navigation system based on three-dimensional convolutional attention, characterized in that: The method for implementing an unmanned driving navigation method based on three-dimensional convolutional attention as claimed in any one of claims 1 to 6 comprises: The sequentially connected input layer, 3D convolutional attention layer, upsampling layer, global adaptive 3D pooling layer, fully connected layer and output layer; The input layer is used to obtain a three-dimensional grid image of the road and input it into the multiple three-dimensional convolutional attention layers; The three-dimensional convolutional attention layer includes a multi-layer three-dimensional convolutional attention network, which is used to perform three-dimensional convolutional feature extraction on the three-dimensional grid image of the road to obtain road convolution features, and the three-dimensional convolutional attention network is constructed based on three-dimensional convolution and self-attention mechanism; The upsampling layer is used to perform upsampling processing on the road convolution feature, and map to obtain the upsampled road convolution feature; The global adaptive three-dimensional pooling layer is used to perform global adaptive three-dimensional pooling on the upsampled road convolution features to obtain downsampled target road convolution features; The fully connected layer is used to classify the target road according to the convolutional features to obtain a classification result; The output layer is used to output a path planning graph according to the classification result.

8. A vehicle, characterized in that: A set of instruction sets is stored, and the instruction sets are executed by the vehicle to implement the unmanned driving navigation method based on three-dimensional convolutional attention as described in any one of claims 1-6.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the unmanned driving navigation method based on three-dimensional convolutional attention as described in any one of claims 1-6.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by the processor, the steps of the unmanned driving navigation method based on three-dimensional convolutional attention described in any one of claims 1-6 are implemented.

Images