Road surface point set height prediction method, device, equipment and storage medium

Abstract

The present invention relates to the field of autonomous driving technology, and discloses a road point set height prediction method, device, equipment and storage medium, which are used to obtain the height value corresponding to each road point through a trained road height prediction model, so as to improve the accuracy of road point set height prediction and road morphology fitting. The road point set height prediction method comprises: obtaining a multi-frame three-dimensional road point set and a multi-frame two-dimensional point set to be calculated; performing feature processing on the multi-frame three-dimensional road point set and the multi-frame two-dimensional point set to be calculated respectively through the trained road height prediction model, and obtaining a target road point set global feature tensor and a target point set feature tensor to be calculated; performing point set height calculation according to the target road point set global feature tensor and the target point set feature tensor to be calculated, and obtaining the height value of each road point in each frame of the two-dimensional point set to be calculated.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method, device, equipment and storage medium for predicting the height of a road surface point set. Background Art

[0002] Unmanned vehicles perceive the surrounding environment information during driving, and road surface information is one of the important information in the surrounding environment information. From a qualitative perspective, road surface information includes plane, slope, and inclination angle. From a quantitative perspective, the vehicle body coordinate system is generally constructed with the center of the rear axle of the unmanned vehicle as the origin, the direction of the front of the vehicle as the x-axis, and the direction perpendicular to the road surface as the z-axis. For any perceived surrounding road surface point (x, y), the road surface height value z_ground corresponding to the road surface point is obtained in this way. The functional expression is z_ground=f(x, y), where f represents the function to be calculated.

[0003] In the prior art, in order to solve the equation, a set of laser point cloud information is usually obtained and an equation is fitted to express the road surface morphology. For example, it can be assumed that the road surface is a second-order polynomial surface. However, the road surface is not actually a polynomial surface. The road surface fitting effect is not bad, and for road surface areas farther away, the value of the polynomial is easy to diverge, resulting in low accuracy in predicting the height of road points and low accuracy in fitting the road surface morphology. Summary of the invention

[0004] The present invention provides a road surface point set height prediction method, device, equipment and storage medium, which are used to obtain the height value corresponding to each road surface point through a trained road surface height prediction model, thereby improving the accuracy of road surface point set height prediction and the accuracy of road surface morphology fitting.

[0005] To achieve the above-mentioned purpose, the first aspect of the present invention provides a method for predicting the height of a road surface point set, comprising: obtaining a multi-frame three-dimensional road surface point set and a multi-frame two-dimensional point set to be calculated; performing feature processing on the multi-frame three-dimensional road surface point set and the multi-frame two-dimensional point set to be calculated respectively through a trained road surface height prediction model to obtain a global feature tensor of a target road surface point set and a feature tensor of a target point set to be calculated; performing point set height calculation according to the global feature tensor of the target road surface point set and the feature tensor of the target point set to be calculated to obtain the height value of each road surface point in each frame of the two-dimensional point set to be calculated.

[0006] In a feasible implementation manner, the road surface height prediction model that has been trained performs feature processing on the multi-frame three-dimensional road surface point set and the multi-frame two-dimensional point set to be calculated, respectively, to obtain a global feature tensor of a target road surface point set and a feature tensor of a target point set to be calculated, including: performing dimension expansion on the multi-frame three-dimensional road surface point set through the first fully connected network layer in the trained road surface height prediction model to obtain an expanded road surface point set feature tensor; performing a global maximum pooling operation on the expanded road surface point set feature tensor to obtain a global feature tensor of a target road surface point set; and performing feature dimension transformation on the multi-frame two-dimensional point set to be calculated according to the second fully connected network layer in the trained road surface height prediction model to obtain a feature tensor of a target point set to be calculated.

[0007] In a feasible implementation manner, the first fully connected network layer in the road height prediction model completed through training expands the dimension of the multi-frame three-dimensional road point set to obtain an expanded road point point set feature tensor, including: the first fully connected network layer in the road height prediction model completed through training increases the feature dimension of the multi-frame three-dimensional road point set according to a first preset dimension to obtain a first road point set feature tensor; expands the dimension of the first road point set feature tensor according to a second preset dimension to obtain a second road point set feature tensor; maps the feature dimension corresponding to the second road point set feature tensor to a target preset dimension to obtain an expanded road point point set feature tensor.

[0008] In a feasible implementation manner, the second fully connected network layer in the trained road height prediction model transforms the feature dimension of the multi-frame two-dimensional point set to be calculated to obtain a feature tensor of the target point set to be calculated, including: according to the second fully connected network layer in the trained road height prediction model, expanding the feature dimension corresponding to the multi-frame two-dimensional point set to be calculated to the first preset high-dimensional space to obtain a feature tensor of the high-dimensional point set to be calculated; converting the feature dimension corresponding to the feature tensor of the high-dimensional point set to be calculated to the second preset high-dimensional space to obtain a feature tensor of the target point set to be calculated.

[0009] In a feasible implementation manner, the point set height calculation is performed according to the target road surface point set global feature tensor and the target point set feature tensor to be calculated to obtain the height value of each road surface point in each frame of the two-dimensional point set to be calculated, including: dimensional superposition of the target road surface point set global feature tensor and the target point set feature tensor to be calculated to obtain the superimposed point set global feature tensor; and dimensionality reduction processing is performed on the superimposed point set global feature tensor through the third fully connected network layer in the trained road surface height prediction model to obtain the height value of each road surface point in each frame of the two-dimensional point set to be calculated.

[0010] In a feasible implementation manner, the third fully connected network layer in the road height prediction model completed through the training performs dimensionality reduction processing on the global feature tensor of the superimposed point set to obtain the height value of each road surface point in the two-dimensional point set to be calculated in each frame, including: reducing the feature dimension corresponding to the global feature tensor of the superimposed point set to a low-dimensional space through the third fully connected network layer in the road height prediction model completed through the training to obtain the global feature tensor of the low-dimensional point set; reducing the feature dimension corresponding to the global feature tensor of the low-dimensional point set to one dimension to obtain the height value of each road surface point in the two-dimensional point set to be calculated in each frame.

[0011] In a feasible implementation manner, before obtaining a multi-frame three-dimensional road surface point set and a multi-frame two-dimensional point set to be calculated, the road surface point set height prediction method also includes: obtaining a multi-frame three-dimensional road surface point cloud sample data, a multi-frame two-dimensional point cloud sample data to be calculated, and height annotation true value data corresponding to the multi-frame two-dimensional point cloud sample data to be calculated; performing model training on an initial deep neural network model based on the multi-frame three-dimensional road surface point cloud sample data and the multi-frame two-dimensional point cloud sample data to be calculated to obtain a trained deep neural network model and a road surface prediction height value corresponding to each frame of the point cloud sample data to be calculated; performing model iterative training on the trained deep neural network model using a preset loss function, the height annotation true value data, and the road surface prediction height value corresponding to each frame of the point cloud sample data to be calculated to obtain a trained road surface height prediction model.

[0012] The second aspect of the present invention provides a road surface point set height prediction device, including: a first acquisition module, used to acquire a multi-frame three-dimensional road surface point set and a multi-frame two-dimensional point set to be calculated; a processing module, used to perform feature processing on the multi-frame three-dimensional road surface point set and the multi-frame two-dimensional point set to be calculated respectively through a trained road surface height prediction model, so as to obtain a target road surface point set global feature tensor and a target point set feature tensor to be calculated; a calculation module, used to perform point set height calculation based on the target road surface point set global feature tensor and the target point set feature tensor to be calculated, so as to obtain the height value of each road surface point in each frame of the two-dimensional point set to be calculated.

[0013] In a feasible implementation, the processing module also includes: an expansion unit, which is used to expand the dimension of the multi-frame three-dimensional road point set through the first fully connected network layer in the trained road height prediction model to obtain the expanded road point set feature tensor; a pooling unit, which is used to perform a global maximum pooling operation on the expanded road point set feature tensor to obtain the target road point set global feature tensor; a transformation unit, which is used to transform the feature dimension of the multi-frame two-dimensional point set to be calculated according to the second fully connected network layer in the trained road height prediction model to obtain the target point set feature tensor to be calculated.

[0014] In a feasible implementation manner, the expansion unit is also specifically used to: perform feature dimension upgrading on the multi-frame three-dimensional road surface point set according to a first preset dimension through the first fully connected network layer in the trained road height prediction model to obtain a first road surface point set feature tensor; perform dimension expansion on the first road surface point set feature tensor according to a second preset dimension to obtain a second road surface point set feature tensor; map the feature dimension corresponding to the second road surface point set feature tensor to a target preset dimension to obtain an expanded road surface point set feature tensor.

[0015] In a feasible implementation manner, the transformation unit is specifically used to: expand the feature dimension corresponding to the multi-frame two-dimensional point set to be calculated to the first preset high-dimensional space according to the second fully connected network layer in the trained road height prediction model, and obtain the high-dimensional point set feature tensor to be calculated; convert the feature dimension corresponding to the high-dimensional point set feature tensor to be calculated to the second preset high-dimensional space, and obtain the target point set feature tensor to be calculated.

[0016] In a feasible implementation, the calculation module also includes: a superposition unit, which is used to perform dimensional superposition on the target road surface point set global feature tensor and the target to-be-calculated point set feature tensor to obtain the superimposed point set global feature tensor; a dimensionality reduction unit, which is used to perform dimensionality reduction processing on the superimposed point set global feature tensor through the third fully connected network layer in the trained road surface height prediction model to obtain the height value of each road surface point in each frame of the two-dimensional to-be-calculated point set.

[0017] In a feasible implementation, the dimension reduction unit is specifically used to: reduce the feature dimension corresponding to the superimposed point set global feature tensor to a low-dimensional space through the third fully connected network layer in the trained road height prediction model, and obtain the low-dimensional point set global feature tensor; reduce the feature dimension corresponding to the low-dimensional point set global feature tensor to one dimension, and obtain the height value of each road surface point in the two-dimensional point set to be calculated in each frame.

[0018] In a feasible implementation manner, the road surface point set height prediction device also includes: a second acquisition module, used to acquire multiple frames of three-dimensional road surface point cloud sample data, multiple frames of two-dimensional point cloud sample data to be calculated, and height annotation true value data corresponding to the multiple frames of two-dimensional point cloud sample data to be calculated; a training module, used to perform model training on the initial deep neural network model based on the multiple frames of three-dimensional road surface point cloud sample data and the multiple frames of two-dimensional point cloud sample data to be calculated, and obtain the trained deep neural network model and the road surface prediction height value corresponding to each frame of point cloud sample data to be calculated; an iterative training module, used to perform model iterative training on the trained deep neural network model through a preset loss function, the height annotation true value data and the road surface prediction height value corresponding to each frame of point cloud sample data to be calculated, and obtain a trained road surface height prediction model.

[0019] The third aspect of the present invention provides a road surface point set height prediction device, comprising: a memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected via lines; the at least one processor calls the instructions in the memory so that the road surface point set height prediction device executes the above-mentioned road surface point set height prediction method.

[0020] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned road surface point set height prediction method.

[0021] In the technical solution provided by the present invention, a multi-frame three-dimensional road surface point set and a multi-frame two-dimensional point set to be calculated are obtained; the multi-frame three-dimensional road surface point set and the multi-frame two-dimensional point set to be calculated are respectively subjected to feature processing by a trained road surface height prediction model to obtain a target road surface point set global feature tensor and a target point set feature tensor to be calculated; point set height calculation is performed according to the target road surface point set global feature tensor and the target point set feature tensor to be calculated to obtain the height value of each road surface point in each frame of the two-dimensional point set to be calculated. In the embodiment of the present invention, feature processing and point set height calculation are respectively performed on a multi-frame three-dimensional road surface point set and a multi-frame two-dimensional point set to be calculated by a trained road surface height prediction model to obtain the height value corresponding to each road surface point, thereby improving the accuracy of road surface point set height prediction and the accuracy of road surface morphology fitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of an embodiment of a method for predicting the height of a road surface point set in an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of another embodiment of a method for predicting the height of a road surface point set in an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of an embodiment of a device for predicting the height of a road surface point set in an embodiment of the present invention;

[0025] Figure 4 It is a schematic diagram of another embodiment of the device for predicting the height of a road surface point set in an embodiment of the present invention;

[0026] Figure 5 It is a schematic diagram of an embodiment of a road surface point set height prediction device in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The embodiments of the present invention provide a road surface point set height prediction method, device, equipment and storage medium, which are used to obtain the height value corresponding to each road surface point through a trained road surface height prediction model, thereby improving the accuracy of road surface point set height prediction and the accuracy of road surface morphology fitting.

[0028] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 An embodiment of the road surface point set height prediction method in the embodiment of the present invention includes:

[0030] 101. Obtain a multi-frame three-dimensional road surface point set and a multi-frame two-dimensional point set to be calculated.

[0031] It should be noted that unmanned vehicles need to perceive the topographic features of the road surface in real time during driving. That is, whether the road surface on which the unmanned vehicle is driving is flat, has a slope, or has some inclination angles and turns. In general, unmanned vehicles can collect multiple frames of two-dimensional point sets to be calculated based on the constructed vehicle body coordinate system and preset sensors. Each position point in each frame of the two-dimensional point set to be calculated is represented by two-dimensional coordinates (x, y). The multi-frame three-dimensional road point set is a pre-set multiple groups of known ground point cloud data.

[0032] In some embodiments, the server receives initial road surface laser point cloud data collected by the unmanned vehicle in real time; the server performs data preprocessing on the initial road surface laser point cloud data, that is, the server performs data denoising, data deduplication, and data missing interpolation on the initial road surface laser point cloud data to obtain a multi-frame two-dimensional point set to be calculated, and each frame of the two-dimensional point set to be calculated is a set of sparse road surface points; the server obtains a preset multi-frame three-dimensional road surface point set; the server stores the multi-frame three-dimensional road surface point set and the multi-frame two-dimensional point set to be calculated in a preset data table.

[0033] It is understandable that the execution subject of the present invention may be a road surface point set height prediction device, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0034] 102. The trained road height prediction model performs feature processing on a multi-frame three-dimensional road point set and a multi-frame two-dimensional point set to be calculated, respectively, to obtain a global feature tensor of a target road point set and a feature tensor of a target point set to be calculated.

[0035] The trained road height prediction model includes two input branches and multiple multilayer perceptrons (MLP) of various scales. The two input branches include a first input branch and a second input branch. The first input branch includes a pooling layer and a fully connected layer (e.g., a first fully connected network layer and a second fully connected network layer) for processing a multi-frame three-dimensional road point set, and the second input branch includes a fully connected layer (e.g., a third fully connected network layer) for processing a multi-frame two-dimensional point set to be calculated. And other network structures.

[0036] In some embodiments, first, the server inputs a multi-frame three-dimensional road point set into the first input branch of the trained road height prediction model, and performs convolution processing on the feature length corresponding to each frame of the three-dimensional road point set through the first multi-layer perceptron in the first fully connected layer corresponding to the first input branch, so as to obtain an expanded road point set feature tensor. For example, the feature size corresponding to the multi-frame three-dimensional road point set is B×N×3, and the first multi-layer perceptron is mlp(64,64,128). The server convolves the feature length corresponding to each frame of the three-dimensional road point set from 3 to 64, then to 64, and then to 128 through mlp(64,64,128), so as to obtain an expanded road point set feature tensor with a feature size of B×N×128, wherein B is used to indicate the number of multiple frames corresponding to the three-dimensional road point set (that is, the number of batches), N is used to indicate the number of three-dimensional road point sets in each frame, and 3 is used to indicate that each road point in each frame of the three-dimensional road point set contains three dimensions (x, y, z), and the first multi-layer perceptron is a three-layer two-dimensional convolution. Next, the server performs an average value or maximum value pooling operation on the global feature tensor of the target road surface point set to obtain the global feature tensor of the target road surface point set. The feature size corresponding to the global feature tensor of the target road surface point set is B×c'.

[0037] Then, the server inputs the multi-frame two-dimensional point set to be calculated into the second input branch of the trained road height prediction model, and convolves the feature lengths corresponding to each frame of the two-dimensional point set to be calculated through the second multi-layer perceptron in the second fully connected layer corresponding to the second input branch, and obtains the feature tensor of the target point set to be calculated. For example, the feature size corresponding to the multi-frame two-dimensional point set to be calculated is B'×M×2, and the second multi-layer perceptron is mlp(64,64). The server convolves the feature lengths corresponding to each frame of the two-dimensional point set to be calculated from 2 to 64, and then to 64 through mlp(64,64), and the feature tensor of the target point set to be calculated with a feature size of B'×M×64, where B' is used to indicate the number of multiple frames corresponding to the two-dimensional point set to be calculated (that is, the number of batches), M is used to indicate the number of position points of each frame of the two-dimensional point set to be calculated, and 2 is used to indicate that each point to be calculated in each frame of the two-dimensional point set to be calculated contains two dimensions (x, y), B' is the same as B, and the second multi-layer perceptron is a two-layer two-dimensional convolution.

[0038] 103. Calculate the point set height according to the global feature tensor of the target road surface point set and the feature tensor of the target to-be-calculated point set, and obtain the height value of each road surface point in the two-dimensional to-be-calculated point set of each frame.

[0039] It can be understood that the two input branches and multiple multi-layer perceptrons of various scales in the trained road height prediction model are finally merged to generate an output layer to obtain the height value of each road point in the two-dimensional point set to be calculated in each frame.

[0040] In some embodiments, the server merges the target road surface point set global feature tensor output by the first input branch and the target to-be-calculated point set feature tensor output by the second input branch in the channel dimension to obtain a superimposed point set global feature tensor. For example, the feature size corresponding to the target road surface point set global feature tensor is B×256, and the feature size corresponding to the target to-be-calculated point set feature tensor is B'×M×64, then the feature size corresponding to the superimposed point set global feature tensor is B'×M×320. Then, the server performs convolution processing on the feature length corresponding to the superimposed point set global feature tensor through the third multi-layer perceptron to obtain the target to-be-calculated point set feature tensor. For example, the third multi-layer perceptron is mlp(64,1), and the server reduces the feature length corresponding to the superimposed point set global feature tensor from 320 to 64, and then to 1 through mlp(64,1) to obtain the height value of each road point in each frame of the two-dimensional to-be-calculated point set with a feature size of B'×M×1. Furthermore, the server performs associative mapping storage processing on the multi-frame two-dimensional point sets to be calculated and the height value of each road surface point in each frame of the two-dimensional point sets to be calculated.

[0041] It should be noted that the third multi-layer perceptron is a two-layer two-dimensional convolution, and the third multi-layer perceptron can be pre-set in the global average pooling layer, or pre-set in the third fully connected layer, and can also be other network hierarchical structures, which are not limited here. In addition, the specific network hierarchical structures corresponding to the two input branches in the trained road height prediction model and the specific values ​​corresponding to the multiple multi-layer perceptrons of various scales can be adjusted according to actual business needs, which are not limited here.

[0042] In an embodiment of the present invention, the trained road surface height prediction model performs feature processing and point set height calculation on a multi-frame three-dimensional road surface point set and a multi-frame two-dimensional point set to be calculated, respectively, to obtain the height value corresponding to each road surface point, thereby improving the accuracy of road surface point set height prediction and the accuracy of road surface morphology fitting.

[0043] See also Figure 2 Another embodiment of the method for predicting the height of a road surface point set in the embodiment of the present invention includes:

[0044] 201. Obtain a multi-frame three-dimensional road surface point set and a multi-frame two-dimensional point set to be calculated.

[0045] The specific execution process of step 201 is similar to the specific execution process of step 101, and will not be repeated here.

[0046] Furthermore, before step 201, the server obtains multiple frames of three-dimensional road surface point cloud sample data, multiple frames of two-dimensional point cloud sample data to be calculated, and height-labeled true value data corresponding to the multiple frames of two-dimensional point cloud sample data to be calculated; the server performs model training on the initial deep neural network model based on the multiple frames of three-dimensional road surface point cloud sample data and the multiple frames of two-dimensional point cloud sample data to be calculated, and obtains the trained deep neural network model and the road surface predicted height value corresponding to each frame of point cloud sample data to be calculated; the server performs model iterative training on the trained deep neural network model through a preset loss function, the height-labeled true value data, and the road surface predicted height value corresponding to each frame of point cloud sample data to be calculated, and obtains a trained road surface height prediction model.

[0047] It should be noted that the preset loss function is used to estimate the degree of inconsistency between the predicted road height value z corresponding to each frame of the point cloud sample data to be calculated in the trained deep neural network model and the height annotation true value data z^ corresponding to the multi-frame two-dimensional point cloud sample data to be calculated, and to supervise the iterative training of the trained deep neural network model. The preset loss function is a non-negative real-valued function. The smaller the loss function value, the better the robustness of the trained deep neural network model. The preset loss function can be a square loss function, a regression loss function, or other loss functions, which are not limited here. For example, when the preset loss function is a regression loss function, the corresponding preset loss function L is L = |∑zz^|.

[0048] 202. The first fully connected network layer in the trained road height prediction model is used to expand the dimension of the multi-frame three-dimensional road point set to obtain the expanded road point set feature tensor.

[0049] Among them, the feature size corresponding to the multi-frame 3D road point set is B×N×3, B is used to indicate the number of multi-frames corresponding to the 3D road point set (that is, the batch number), N is used to indicate the number of 3D road point sets in each frame, 3 is used to indicate that each road point in each frame 3D road point set contains three dimensions (x, y, z), and N and B are both positive integers. For example, if N is 128 and B is 8, the feature size corresponding to the multi-frame 3D road point set is 8×128×3. In some embodiments, the server performs feature dimension enhancement on the multi-frame three-dimensional road point set according to the first preset dimension through the first fully connected network layer in the trained road height prediction model, and obtains the first road point set feature tensor, the first preset dimension is a, and the first road point set feature tensor corresponds to a feature size of B×N×a; the server performs dimension expansion on the first road point set feature tensor according to the second preset dimension, and obtains the second road point set feature tensor, the second preset dimension is b, and the second road point set feature tensor corresponds to a feature size of B×N×b; the server maps the feature dimension corresponding to the second road point set feature tensor to the target preset dimension, and obtains the expanded road point set feature tensor, the target preset dimension is c, and the expanded road point set feature tensor corresponds to a feature size of B×N×c. It should be noted that the first fully connected network layer is a fully connected network with shared weights, and each convolution filter in the fully connected network with shared weights shares the same weight matrix and bias term, and does not need to consider the position of its local features when expanding the dimension of the multi-frame three-dimensional road point set, thereby improving the efficiency of the model in obtaining the expanded road point set feature tensor.

[0050] For example, the feature size corresponding to the multi-frame three-dimensional road point set is B×N×3, the first preset dimension a is 32, the first preset dimension b is 64, and the target preset dimension is 128. Then the feature size corresponding to the first road point set feature tensor is B×N×32, and the feature size corresponding to the second road point set feature tensor is B×N×64. The expanded road point set feature tensor is B×N×128.

[0051] 203. Perform a global maximum pooling operation on the expanded road surface point set feature tensor to obtain a global feature tensor of the target road surface point set.

[0052] Specifically, the server performs a global maximum pooling operation on the expanded road surface point set feature tensor through the pooling layer in the trained road height prediction model (that is, the server takes the maximum value of the N expanded road surface point set feature tensors), and obtains the target road surface point set global feature tensor with a feature size of B×c'. That is, the server performs a global feature tensor on the channel dimension c' and the target preset dimension c, where c' can be consistent with 128 or 256, which is not limited here. The channel dimension c' is used to represent the global feature dimension corresponding to each frame of the three-dimensional road surface point set.

[0053] 204. Perform feature dimension transformation on the multi-frame two-dimensional point set to be calculated according to the second fully connected network layer in the trained road height prediction model to obtain a feature tensor of the target point set to be calculated.

[0054] Among them, the feature size corresponding to the multi-frame two-dimensional point set to be calculated is B'×M×2, B' is used to indicate the number of multiple frames corresponding to the two-dimensional point set to be calculated (that is, the number of batches), M is used to indicate the number of position points of each frame of the two-dimensional point set to be calculated, 2 is used to indicate that each point to be calculated in each frame of the two-dimensional point set to be calculated contains two dimensions (x, y), M and B' are both positive integers, and the values ​​corresponding to B' and B are the same. For example, M is 64 and B' is 8, then the feature size corresponding to the multi-frame two-dimensional point set to be calculated is 8×64×2. In some embodiments, the server expands the feature dimensions corresponding to the multi-frame two-dimensional point set to be calculated to the first preset high-dimensional space according to the second fully connected network layer in the trained road height prediction model, and obtains the feature tensor of the high-dimensional point set to be calculated; the server converts the feature dimensions corresponding to the feature tensor of the high-dimensional point set to be calculated to the second preset high-dimensional space to obtain the feature tensor of the target point set to be calculated, and the feature size corresponding to the feature tensor of the target point set to be calculated is B'×M×d. For example, based on the second fully connected network layer in the trained road height prediction model, the server expands the feature dimension 2 corresponding to the multi-frame two-dimensional point set to be calculated to the first preset high-dimensional space 32, and obtains a high-dimensional point set feature tensor to be calculated with a feature size of B'×M×32; the server converts the feature dimension 32 corresponding to the feature tensor of the high-dimensional point set to be calculated to the second preset high-dimensional space 64, and obtains a target point set feature tensor to be calculated with a feature size of B'×M×64.

[0055] It should be noted that the second fully connected network layer is a fully connected network with shared weights. When performing feature dimension transformation on a multi-frame two-dimensional point set to be calculated, the second fully connected network layer does not need to consider the position of its local features, thereby improving the efficiency of the model in obtaining the feature tensor of the target point set to be calculated.

[0056] 205. Calculate the height of the point set according to the global feature tensor of the target road surface point set and the feature tensor of the target point set to be calculated, and obtain the height value of each road surface point in the two-dimensional point set to be calculated in each frame.

[0057] It can be understood that the server takes multi-frame three-dimensional road surface point sets (belonging to known ground points) as input and obtains the global feature tensor of the target road surface point set through the global maximum pooling layer. The server calculates the point set height of the target road surface point set global feature tensor and the two-dimensional point set to be calculated in each frame to obtain the road surface height fitting result, that is, the height value of each road surface point in the two-dimensional point set to be calculated in each frame.

[0058] In some embodiments, the server performs dimension superposition on the target road surface point set global feature tensor and the target to-be-calculated point set feature tensor to obtain a superimposed point set global feature tensor, that is, the server performs channel dimension addition processing on the target road surface point set global feature tensor with a feature size of B×c' and the target to-be-calculated point set feature tensor with a feature size of B'×M×d, that is, the target to-be-calculated point set feature tensors are all increased by the corresponding target road surface point set global feature tensor to obtain a superimposed point set global feature tensor with a feature size of B'×M×(c'+d), for example, the target road The feature size corresponding to the global feature tensor of the surface point set is B×128, and the feature size corresponding to the feature tensor of the target point set to be calculated is B'×M×64. The feature size corresponding to the global feature tensor of the superimposed point set is B'×M×192. The 192-dimensional feature contains both the coordinate information of the two-dimensional point set to be calculated in each frame and the global features of the three-dimensional road surface point set in each frame. The server performs dimensionality reduction processing on the superimposed global feature tensor of the point set through the third fully connected network layer in the trained road height prediction model to obtain the height value of each road surface point in the two-dimensional point set to be calculated in each frame.

[0059] Furthermore, when the server executes the third fully connected network layer in the trained road height prediction model, performs dimensionality reduction processing on the global feature tensor of the superimposed point set, and obtains the height value of each road point in the two-dimensional point set to be calculated in each frame, in some embodiments, the server reduces the feature dimension corresponding to the global feature tensor of the superimposed point set to a low-dimensional space through the third fully connected network layer in the trained road height prediction model, and obtains the global feature tensor of the low-dimensional point set; the server reduces the feature dimension corresponding to the global feature tensor of the low-dimensional point set to one dimension, and obtains the height value of each road point in the two-dimensional point set to be calculated in each frame. It should be noted that the third fully connected network layer is a fully connected network with shared weights, and the third fully connected network layer does not need to consider the position of its local features when performing dimensionality reduction on the global feature tensor of the superimposed point set, which improves the efficiency of the model in obtaining the height value of each road point in the two-dimensional point set to be calculated in each frame. For example, the feature dimension corresponding to the global feature tensor of the superimposed point set is 64, and the low-dimensional space is 32, then the feature size corresponding to the global feature tensor of the low-dimensional point set is B'×M×32; the server reduces the feature dimension corresponding to the global feature tensor of the low-dimensional point set to one dimension, and obtains output data with a feature size of B'×M×1, where 1 represents the height value of each road surface point in the two-dimensional point set to be calculated in each frame.

[0060] In some embodiments, the server performs road morphology fitting processing based on multiple frames of two-dimensional point sets to be calculated and the height values ​​of each road surface point in each frame of the two-dimensional point sets to be calculated to obtain a road morphology fitting image, and draws and displays the road morphology fitting image through a preset simulation tool to facilitate the target personnel to intuitively view the multiple frames of two-dimensional point sets to be calculated and the height values ​​of each road surface point in each frame of the two-dimensional point sets to be calculated.

[0061] In an embodiment of the present invention, the trained road height prediction model performs feature processing and point set height calculation on a multi-frame three-dimensional road point set and a multi-frame two-dimensional point set to be calculated, respectively, to obtain the height value corresponding to each road point, thereby improving the accuracy of road point set height prediction and the accuracy of road morphology fitting.

[0062] The above describes the road surface point set height prediction method in the embodiment of the present invention. The following describes the road surface point set height prediction device in the embodiment of the present invention. Figure 3 In one embodiment of the present invention, a device for predicting the height of a road surface point set includes:

[0063] The first acquisition module 301 is used to acquire a multi-frame three-dimensional road surface point set and a multi-frame two-dimensional point set to be calculated;

[0064] The processing module 302 is used to perform feature processing on the multi-frame three-dimensional road surface point set and the multi-frame two-dimensional point set to be calculated respectively through the trained road surface height prediction model to obtain a global feature tensor of the target road surface point set and a feature tensor of the target point set to be calculated;

[0065] The calculation module 303 is used to calculate the height of the point set according to the global feature tensor of the target road surface point set and the feature tensor of the target to-be-calculated point set, so as to obtain the height value of each road surface point in each frame of the two-dimensional to-be-calculated point set.

[0066] In an embodiment of the present invention, the trained road surface height prediction model performs feature processing and point set height calculation on a multi-frame three-dimensional road surface point set and a multi-frame two-dimensional point set to be calculated, respectively, to obtain the height value corresponding to each road surface point, thereby improving the accuracy of road surface point set height prediction and the accuracy of road surface morphology fitting.

[0067] See also Figure 4 Another embodiment of the road surface point set height prediction device in the embodiment of the present invention includes:

[0068] The first acquisition module 301 is used to acquire a multi-frame three-dimensional road surface point set and a multi-frame two-dimensional point set to be calculated;

[0069] The processing module 302 is used to perform feature processing on the multi-frame three-dimensional road surface point set and the multi-frame two-dimensional point set to be calculated respectively through the trained road surface height prediction model to obtain a global feature tensor of the target road surface point set and a feature tensor of the target point set to be calculated;

[0070] The calculation module 303 is used to calculate the height of the point set according to the global feature tensor of the target road surface point set and the feature tensor of the target to-be-calculated point set, so as to obtain the height value of each road surface point in each frame of the two-dimensional to-be-calculated point set.

[0071] In a feasible implementation manner, the processing module 302 may further include:

[0072] An expansion unit 3021 is used to expand the dimension of the multi-frame three-dimensional road surface point set through the first fully connected network layer in the trained road height prediction model to obtain an expanded road surface point set feature tensor;

[0073] A pooling unit 3022 is used to perform a global maximum pooling operation on the expanded road surface point set feature tensor to obtain a target road surface point set global feature tensor;

[0074] The transformation unit 3023 is used to transform the feature dimension of the multi-frame two-dimensional point set to be calculated according to the second fully connected network layer in the trained road height prediction model to obtain a feature tensor of the target point set to be calculated.

[0075] In a feasible implementation manner, the expansion unit 3021 may also be specifically used for:

[0076] By using a first fully connected network layer in the trained road height prediction model, the multi-frame three-dimensional road point set is subjected to feature dimension upgrading according to a first preset dimension to obtain a first road point set feature tensor;

[0077] Expanding the dimension of the first road surface point set feature tensor according to a second preset dimension to obtain a second road surface point set feature tensor;

[0078] The feature dimension corresponding to the second road surface point set feature tensor is mapped to a target preset dimension to obtain an expanded road surface point set feature tensor.

[0079] In a feasible implementation manner, the transformation unit 3023 may also be specifically used for:

[0080] According to the second fully connected network layer in the trained road height prediction model, the feature dimension corresponding to the multi-frame two-dimensional point set to be calculated is expanded to a first preset high-dimensional space to obtain a feature tensor of the high-dimensional point set to be calculated;

[0081] The feature dimension corresponding to the high-dimensional feature tensor of the point set to be calculated is converted to a second preset high-dimensional space to obtain a target feature tensor of the point set to be calculated.

[0082] In a feasible implementation manner, the calculation module 303 may further include:

[0083] The superposition unit 3031 is used to perform dimension superposition on the target road surface point set global feature tensor and the target to-be-calculated point set feature tensor to obtain a superimposed point set global feature tensor;

[0084] The dimension reduction unit 3032 is used to perform dimensionality reduction processing on the superimposed point set global feature tensor through the third fully connected network layer in the trained road height prediction model to obtain the height value of each road point in the two-dimensional point set to be calculated in each frame.

[0085] In a feasible implementation manner, the dimension reduction unit 3032 may also be specifically used for:

[0086] By using the third fully connected network layer in the trained road height prediction model, the feature dimension corresponding to the superimposed point set global feature tensor is reduced to a low-dimensional space to obtain a low-dimensional point set global feature tensor;

[0087] The feature dimension corresponding to the global feature tensor of the low-dimensional point set is reduced to one dimension, and the height value of each road surface point in the two-dimensional point set to be calculated in each frame is obtained.

[0088] In a feasible implementation manner, the road surface point set height prediction device further includes:

[0089] The second acquisition module 304 is used to acquire multiple frames of three-dimensional road surface point cloud sample data, multiple frames of two-dimensional point cloud sample data to be calculated, and height annotation true value data corresponding to the multiple frames of two-dimensional point cloud sample data to be calculated;

[0090] The training module 305 is used to perform model training on the initial deep neural network model based on the multiple frames of three-dimensional road surface point cloud sample data and the multiple frames of two-dimensional point cloud sample data to be calculated, so as to obtain the trained deep neural network model and the road surface predicted height value corresponding to each frame of point cloud sample data to be calculated;

[0091] The iterative training module 306 is used to perform iterative model training on the trained deep neural network model through a preset loss function, the true value data of the height annotation and the road surface predicted height value corresponding to each frame of the point cloud sample data to be calculated, so as to obtain a trained road surface height prediction model.

[0092] In an embodiment of the present invention, the trained road surface height prediction model performs feature processing and point set height calculation on a multi-frame three-dimensional road surface point set and a multi-frame two-dimensional point set to be calculated, respectively, to obtain the height value corresponding to each road surface point, thereby improving the accuracy of road surface point set height prediction and the accuracy of road surface morphology fitting.

[0093] above Figure 3 and Figure 4 The road surface point set height prediction device in the embodiment of the present invention is described in detail from the perspective of modularization, and the road surface point set height prediction device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0094] Figure 5 1 is a schematic diagram of the structure of a road surface point set height prediction device provided by an embodiment of the present invention. The road surface point set height prediction device 500 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 510 (for example, one or more processors) and a memory 520, and one or more storage media 530 (for example, one or more mass storage devices) storing application programs 533 or data 532. Among them, the memory 520 and the storage medium 530 can be short-term storage or permanent storage. The program stored in the storage medium 530 may include one or more modules (not shown in the figure), and each module may include a series of computer program operations in the road surface point set height prediction device 500. Furthermore, the processor 510 may be configured to communicate with the storage medium 530 to execute a series of computer program operations in the storage medium 530 on the road surface point set height prediction device 500.

[0095] The road surface point set height prediction device 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input and output interfaces 560, and / or one or more operating systems 531, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 5 The structure of the road surface point set height prediction device shown does not constitute a limitation on the road surface point set height prediction device, and may include more or less components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0096] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the road surface point set height prediction method.

[0097] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0098] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0099] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A road surface point set height prediction method, characterized in that: The road surface point set height prediction method comprises: Acquire a multi-frame three-dimensional road surface point set and a multi-frame two-dimensional point set to be calculated; The trained road height prediction model is used to perform feature processing on the multi-frame three-dimensional road point set and the multi-frame two-dimensional point set to be calculated, respectively, to obtain a global feature tensor of the target road point set and a feature tensor of the target point set to be calculated; Perform point set height calculation based on the global feature tensor of the target road surface point set and the feature tensor of the target to-be-calculated point set to obtain the height value of each road surface point in each frame of the two-dimensional to-be-calculated point set; The road height prediction model completed through training respectively performs feature processing on the multi-frame three-dimensional road point set and the multi-frame two-dimensional point set to be calculated to obtain a global feature tensor of the target road point set and a feature tensor of the target point set to be calculated, including: expanding the dimension of the multi-frame three-dimensional road point set through the first fully connected network layer in the road height prediction model completed through training to obtain an expanded road point set feature tensor; performing a global maximum pooling operation on the expanded road point set feature tensor to obtain a global feature tensor of the target road point set; performing feature dimension transformation on the multi-frame two-dimensional point set to be calculated according to the second fully connected network layer in the road height prediction model completed through training to obtain a feature tensor of the target point set to be calculated; The point set height calculation is performed according to the target road surface point set global feature tensor and the target to-be-calculated point set feature tensor to obtain the height value of each road surface point in each frame of the two-dimensional point set to be calculated, including: dimensional superposition of the target road surface point set global feature tensor and the target to-be-calculated point set feature tensor to obtain the superimposed point set global feature tensor; and dimensionality reduction processing is performed on the superimposed point set global feature tensor through the third fully connected network layer in the trained road surface height prediction model to obtain the height value of each road surface point in each frame of the two-dimensional point set to be calculated.

2. The method for predicting the height of a road surface point set according to claim 1, characterized in that: The first fully connected network layer in the trained road height prediction model performs dimension expansion on the multi-frame three-dimensional road point set to obtain an expanded road point set feature tensor, including: By using a first fully connected network layer in the trained road height prediction model, the multi-frame three-dimensional road point set is subjected to feature dimension upgrading according to a first preset dimension to obtain a first road point set feature tensor; Expanding the dimension of the first road surface point set feature tensor according to a second preset dimension to obtain a second road surface point set feature tensor; The feature dimension corresponding to the second road surface point set feature tensor is mapped to a target preset dimension to obtain an expanded road surface point set feature tensor.

3. The road surface point set height prediction method according to claim 1, characterized in that: The second fully connected network layer in the trained road height prediction model performs feature dimension transformation on the multi-frame two-dimensional point set to be calculated to obtain a feature tensor of the target point set to be calculated, including: According to the second fully connected network layer in the trained road height prediction model, the feature dimension corresponding to the multi-frame two-dimensional point set to be calculated is expanded to a first preset high-dimensional space to obtain a feature tensor of the high-dimensional point set to be calculated; The feature dimension corresponding to the high-dimensional feature tensor of the point set to be calculated is converted to a second preset high-dimensional space to obtain a target feature tensor of the point set to be calculated.

4. The method for predicting the height of a road surface point set according to claim 3, characterized in that: The third fully connected network layer in the road height prediction model completed through the training performs dimension reduction processing on the global feature tensor of the superimposed point set to obtain the height value of each road point in the two-dimensional point set to be calculated in each frame, including: By using the third fully connected network layer in the trained road height prediction model, the feature dimension corresponding to the superimposed point set global feature tensor is reduced to a low-dimensional space to obtain a low-dimensional point set global feature tensor; The feature dimension corresponding to the global feature tensor of the low-dimensional point set is reduced to one dimension, and the height value of each road surface point in the two-dimensional point set to be calculated in each frame is obtained.

5. The method for predicting the height of a road surface point set according to any one of claims 1 to 4, characterized in that: Before acquiring the multi-frame three-dimensional road surface point set and the multi-frame two-dimensional point set to be calculated, the road surface point set height prediction method further includes: Acquire multiple frames of three-dimensional road surface point cloud sample data, multiple frames of two-dimensional point cloud sample data to be calculated, and highly annotated true value data corresponding to the multiple frames of two-dimensional point cloud sample data to be calculated; Based on the multiple frames of three-dimensional road surface point cloud sample data and the multiple frames of two-dimensional point cloud sample data to be calculated, the initial deep neural network model is trained to obtain the road surface predicted height value corresponding to the trained deep neural network model and each frame of point cloud sample data to be calculated; The trained deep neural network model is iteratively trained using a preset loss function, the height-labeled true value data, and the road surface predicted height value corresponding to each frame of the point cloud sample data to be calculated, to obtain a trained road surface height prediction model.

6. A road surface point set height prediction device, characterized in that: The road surface point set height prediction device comprises: The first acquisition module is used to acquire a multi-frame three-dimensional road surface point set and a multi-frame two-dimensional point set to be calculated; A processing module, used to perform feature processing on the multi-frame three-dimensional road surface point set and the multi-frame two-dimensional point set to be calculated respectively through the trained road surface height prediction model to obtain a global feature tensor of the target road surface point set and a feature tensor of the target point set to be calculated; A calculation module, used to calculate the height of the point set according to the global feature tensor of the target road surface point set and the feature tensor of the target to-be-calculated point set, and obtain the height value of each road surface point in each frame of the two-dimensional to-be-calculated point set; The processing module further includes: an expansion unit, which is used to perform dimension expansion on the multi-frame three-dimensional road point set through the first fully connected network layer in the trained road height prediction model to obtain a feature tensor of the expanded road point set; a pooling unit, which is used to perform a global maximum pooling operation on the expanded road point set feature tensor to obtain a global feature tensor of a target road point set; a transformation unit, which is used to perform feature dimension transformation on the multi-frame two-dimensional point set to be calculated according to the second fully connected network layer in the trained road height prediction model to obtain a feature tensor of a target point set to be calculated; The calculation module also includes: a superposition unit, which is used to perform dimensional superposition on the target road surface point set global feature tensor and the target to-be-calculated point set feature tensor to obtain the superimposed point set global feature tensor; a dimensionality reduction unit, which is used to perform dimensionality reduction processing on the superimposed point set global feature tensor through the third fully connected network layer in the trained road surface height prediction model to obtain the height value of each road surface point in each frame of the two-dimensional to-be-calculated point set.

7. A road surface point set height prediction device, characterized in that: The road surface point set height prediction device comprises: a memory and at least one processor, wherein the memory stores a computer program; The at least one processor calls the computer program in the memory to enable the road surface point set height prediction device to execute the road surface point set height prediction method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for predicting the height of a road surface point set as described in any one of claims 1 to 5 is implemented.

Images