A method, apparatus, device and storage medium for enhancing a point cloud

Abstract

The application discloses a method, device and equipment for enhancing a point cloud and a storage medium. The method comprises the following steps: determining an initial distance image of an initial three-dimensional point cloud, and determining a depth feature map according to the initial distance image and a preset feature encoder; inputting the depth feature map and alignment data into a first preset model to obtain a predicted weight vector, and determining a predicted distance image according to the predicted weight vector; determining a target distance image according to the initial distance image and the predicted distance image, and determining a target three-dimensional point cloud according to a preset polynomial difference value algorithm and the target distance image. The technical scheme of the embodiment of the application improves the enhancement effect of the point cloud with uneven distribution, and solves the problem that the conventional point cloud enhancement method has poor enhancement effect on the point cloud with uneven distribution.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular, to a method and device for enhancing point cloud, equipment and storage medium. BACKGROUND

[0002] With the continuous development of information technology, laser radar has become an important part of the automatic driving car technology. LiDAR point cloud data is a set of spatial point data scanned by a three-dimensional laser radar device, and each point contains three-dimensional coordinate information.

[0003] High-precision and high-density point cloud data plays an important role in the field of automatic driving. In the process of collecting point cloud data and modeling by using laser radar, the point cloud data usually has a lot of irregularities, such as great differences in the density of local point cloud data, overlapping or missing of point cloud data, and point cloud noise, etc. At present, the common way to enhance point cloud data usually needs to use deep learning and up-sampling methods, such as sparse point cloud up-sampling and time interpolation of point cloud sequence on the original point cloud data.

[0004] However, the traditional point cloud enhancement method has good enhancement effect for uniformly distributed point cloud data, but poor enhancement effect for non-uniformly distributed point cloud data. SUMMARY

[0005] The present application provides a method, device, equipment and storage medium for enhancing point cloud to solve the problem of poor enhancement effect of traditional point cloud enhancement method for non-uniformly distributed point cloud data.

[0006] In a first aspect, the present application provides a method for enhancing point cloud, comprising:

[0007] determining an initial distance image of an initial three-dimensional point cloud, and determining a depth feature map according to the initial distance image and a preset feature encoder, wherein the initial three-dimensional point cloud is detected by a radar configured on a preset vehicle;

[0008] inputting the depth feature map and alignment data into a first preset model to obtain a predicted weight vector, and determining a predicted distance image according to the predicted weight vector, wherein the dimension of the predicted distance image is greater than that of the initial distance image, the first preset model is determined based on a time self-attention mechanism, and the alignment data is determined according to the pose information and the historical predicted weight vector of the preset vehicle;

[0009] According to the initial distance image and the predicted distance image, a target distance image is determined, and according to a preset polynomial difference value algorithm and the target distance image, a target three-dimensional point cloud is determined, wherein the number of points in the target three-dimensional point cloud is more than the number of points in the initial three-dimensional point cloud, all initial points in the initial distance image are contained in the target distance image, and the dimension of the target distance image is the same as the dimension of the predicted distance image.

[0010] In a second aspect, the present application provides an apparatus for enhancing a point cloud, comprising:

[0011] a depth feature map determination module configured to determine an initial distance image of an initial three-dimensional point cloud, and determine a depth feature map according to the initial distance image and a preset feature encoder, wherein the initial three-dimensional point cloud is detected by a radar configured on a preset vehicle;

[0012] a predicted distance image determination module configured to input the depth feature map and alignment data into a first preset model to obtain a predicted weight vector, and determine a predicted distance image according to the predicted weight vector, wherein the dimension of the predicted distance image is greater than the dimension of the initial distance image, the first preset model is determined based on a time self-attention mechanism, and the alignment data is determined according to pose information and historical predicted weight vectors of the preset vehicle;

[0013] a three-dimensional point cloud determination module configured to determine a target distance image according to the initial distance image and the predicted distance image, and determine a target three-dimensional point cloud according to a preset polynomial difference value algorithm and the target distance image, wherein the number of points in the target three-dimensional point cloud is more than the number of points in the initial three-dimensional point cloud, all initial points in the initial distance image are contained in the target distance image, and the dimension of the target distance image is the same as the dimension of the predicted distance image.

[0014] In a third aspect, the present application provides an electronic device, comprising:

[0015] at least one processor;

[0016] and a memory in communication connection with the at least one processor;

[0017] wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the method for enhancing a point cloud of the first aspect.

[0018] In a fourth aspect, the present application provides a computer readable storage medium storing computer instructions, and the computer instructions are used to enable a processor to implement the method for enhancing a point cloud of the first aspect when executed.

[0019] The method for enhancing the point cloud provided by the application first determines a depth feature map according to an initial distance image of an initial three-dimensional point cloud and a preset feature encoder, then inputs the depth feature map and alignment data into a first preset model to obtain a predicted weight vector, determines a predicted distance image according to the predicted weight vector, determines a target distance image according to the initial distance image and the predicted distance image, and finally obtains a target three-dimensional point cloud by using a preset polynomial difference algorithm and the target distance image. Compared with the traditional method, the method can improve the accuracy of the predicted weight vector and the predicted distance image by using the preset model to process the depth feature map and the alignment data of the vehicle, the target distance image determined according to the predicted distance image contains all initial points in the initial distance image, which reduces the training time of the preset model in the early stage, improves the learning efficiency of the preset model, further improves the enhancement effect of the point cloud with uneven distribution, and solves the problem that the traditional point cloud enhancement method has poor enhancement effect on the point cloud with uneven distribution.

[0020] It should be understood that the content described in this part is not intended to identify key or important features of the application, nor is it used to limit the scope of the application. Other features of the application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0022] Figure 1 is a flowchart of a method for enhancing a point cloud according to an embodiment of the application;

[0023] Figure 2 is a flowchart of a method for enhancing a point cloud according to an embodiment of the application;

[0024] Figure 3 is a schematic diagram of a preset feature encoder processing data according to an embodiment of the application;

[0025] Figure 4 is a flowchart of a method for enhancing a point cloud according to an embodiment of the application;

[0026] Figure 5 is a training sample diagram according to an embodiment of the application;

[0027] Figure 6 is a flowchart of a method for enhancing a point cloud according to an embodiment of the application;

[0028] Figure 7 is a structural schematic diagram of an apparatus for enhancing point cloud according to the fourth embodiment of the present application;

[0029] Figure 8 is a structural schematic diagram of an electronic device according to the fifth embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the personnel in the technical field better understand the present application scheme, the technical scheme in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0031] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In the description of the present application, "a plurality of" means two or more, unless otherwise specified. The "and / or" describes the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. The character " / " generally represents an "or" relationship between the associated objects. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0032] Embodiment one

[0033] Figure 1 A flowchart of a method for enhancing point cloud is provided for the first embodiment of the present application. The present embodiment can be applicable to the case of enhancing point cloud data. The method can be performed by an apparatus for enhancing point cloud, which can be realized in the form of hardware and / or software. The apparatus for enhancing point cloud can be configured in an electronic device, which can be composed of two or more physical entities or one physical entity.

[0034] As Figure 1As shown, the method for enhancing a point cloud provided by the first embodiment of the application specifically comprises the following steps:

[0035] In the embodiment, the initial distance image of the initial three-dimensional point cloud can be obtained by using a radar device arranged in advance on a preset vehicle, and the initial distance image can be converted into an initial distance image by using a preset conversion method, such as a function fromPointCloud.

[0036] In the embodiment, the initial distance image of the initial three-dimensional point cloud can be obtained by using a radar device arranged in advance on a preset vehicle, and the initial distance image can be converted into an initial distance image by using a preset conversion method, such as a function fromPointCloud.

[0037] In the embodiment, the initial distance image of the initial three-dimensional point cloud can be obtained by using a radar device arranged in advance on a preset vehicle, and the initial distance image can be converted into an initial distance image by using a preset conversion method, such as a function fromPointCloud.

[0038] S102, input the depth feature map and the alignment data into a first preset model to obtain a predicted weight vector, and determine a predicted distance image according to the predicted weight vector.

[0039] In the embodiment, the initial distance image of the initial three-dimensional point cloud can be obtained by using a radar device arranged in advance on a preset vehicle, and the initial distance image can be converted into an initial distance image by using a preset conversion method, such as a function fromPointCloud.

[0040] In the embodiment, the initial distance image of the initial three-dimensional point cloud can be obtained by using a radar device arranged in advance on a preset vehicle, and the initial distance image can be converted into an initial distance image by using a preset conversion method, such as a function fromPointCloud.

[0041] S103, determining a target distance image according to the initial distance image and the predicted distance image, and determining a target three-dimensional point cloud according to a preset polynomial difference algorithm and the target distance image.

[0042] The number of points in the target three-dimensional point cloud is greater than the number of points in the initial three-dimensional point cloud, the target distance image contains all initial points in the initial distance image, and the target distance image has the same dimension as the predicted distance image.

[0043] In this embodiment, the initial distance image and the predicted distance image can be fused to obtain a target distance image, which contains all initial points in the initial distance image and part of the points in the predicted distance image. By using the preset polynomial difference algorithm to process the target distance image, the two-dimensional target distance image can be converted into a three-dimensional target three-dimensional point cloud, which contains predicted three-dimensional points and initial three-dimensional points, thereby achieving the purpose of enhancing the non-uniform point cloud.

[0044] The method for enhancing a point cloud provided in the embodiment of the application first determines a depth feature map according to an initial distance image of an initial three-dimensional point cloud and a preset feature encoder, then inputs the depth feature map and alignment data into a first preset model to obtain a predicted weight vector, determines a predicted distance image according to the predicted weight vector, determines a target distance image according to the initial distance image and the predicted distance image, and finally obtains a target three-dimensional point cloud by using a preset polynomial difference algorithm and the target distance image. Compared with the traditional method, the method uses the preset model to process the depth feature map and the alignment data of the vehicle, which can improve the accuracy of the predicted weight vector and the predicted distance image. The target distance image determined according to the predicted distance image contains all initial points in the initial distance image, which reduces the training time of the preset model in the early stage, improves the learning efficiency of the preset model, further improves the enhancement effect of the point cloud with uneven distribution, and solves the problem of poor enhancement effect of the traditional point cloud enhancement method on the point cloud with uneven distribution.

[0045] Embodiment two

[0046] Figure 2 The flowchart of the method for enhancing a point cloud provided in the second embodiment of the application is further optimized on the basis of the above-mentioned optional technical solutions, and a specific way of enhancing a point cloud is given.

[0047] Optionally, the determining the initial distance image of the initial three-dimensional point cloud comprises: processing the initial three-dimensional point cloud of the preset vehicle to obtain a distance image to be optimized according to the number of points in the laser radar ring, the vertical angle corresponding to the points and the horizontal angle of the points in the initial three-dimensional point cloud, wherein the initial three-dimensional point cloud comprises a plurality of laser radar rings; performing back projection processing on the points in the distance image to be optimized to obtain a three-dimensional point cloud to be converted; determining the Euclidean distance between the points in the three-dimensional point cloud to be converted and the corresponding points in the initial three-dimensional point cloud, and screening out the points in the three-dimensional point cloud to be converted corresponding to the Euclidean distance less than a preset distance threshold to obtain a target three-dimensional point cloud; and determining the initial distance image according to the number of points in the laser radar ring, the vertical angle corresponding to the points and the horizontal angle of the points in the target three-dimensional point cloud, wherein the total number of rows of initial points in the initial distance image is the same as the total number of rows of points in the distance image to be optimized, and the total number of columns of initial points in the initial distance image is less than the total number of columns of points in the distance image to be optimized. The advantage of this setting is that since the pixel information of adjacent points is needed in the later stage, the empty points (empty pixels) in the distance image to be optimized can be filtered out through screening, thereby avoiding the negative impact of the empty points on the determination of the predicted distance image and the target three-dimensional point cloud in the later stage.

[0048] Optionally, the determining the depth feature map according to the initial distance image and the preset feature encoder comprises: determining a first preset window range of each initial point in the initial distance image, wherein the initial point is within the first preset window range; determining, for each initial point, a relative vertical angle embedding vector according to the vertical angle corresponding to the current initial point and the current first window point, wherein the current first window point is all the initial points within the current first preset window range except the initial point corresponding to the first preset window range; determining, for each initial point, a relative distance embedding vector according to the distance between the current initial point and the current first window point; and inputting the relative distance embedding vector, the relative vertical angle embedding vector and the initial distance image into a preset feature encoder to obtain a depth feature map, wherein the preset feature encoder is determined based on an enhanced depth super-resolution network. The advantage of this setting is that by inputting the relative distance embedding vector, the relative vertical angle embedding vector and the initial distance image into the preset feature encoder, the noise points in the depth feature map can be reduced and the representation ability of the target three-dimensional point cloud for the real environment can be improved.

[0049] Optionally, the inputting the depth feature map and the alignment data into the first preset model to obtain a predicted weight vector comprises: determining alignment data at a current moment according to a predicted weight vector at a previous moment and pose information of the preset vehicle at the current moment; and inputting the depth feature map at the current moment and the alignment data at the current moment into the first preset model to obtain the predicted weight vector at the current moment, wherein the first preset model is determined based on a time self-attention mechanism and a multi-head attention mechanism. In this way, the time correlation between the same object at the previous moment and the current moment is established, and the predicted weight vector can be adjusted in real time along with the movement of the preset vehicle, thereby improving the enhancement effect of the point cloud.

[0050] As shown in Figure 2 The embodiment two of the present application provides a method for enhancing a point cloud, and specifically comprises the following steps:

[0051] S201, processing an initial three-dimensional point cloud of a preset vehicle according to a number of points in a laser radar ring in the initial three-dimensional point cloud, a vertical angle corresponding to the points, and a horizontal angle of the points, to obtain a distance image to be optimized.

[0052] The initial three-dimensional point cloud comprises a plurality of laser radar rings.

[0053] For example, the three-dimensional points in the initial three-dimensional point cloud can be projected into a two-dimensional distance image to be optimized based on radar identifiers of the points in the initial three-dimensional point cloud, and the determination of the number of rows and the number of columns of each point in the distance image to be optimized can be

[0054]

[0055] wherein x, y and z respectively represent the coordinates of the X-axis, the Y-axis and the Z-axis of the points in the initial three-dimensional point cloud, v angle represents the vertical angle corresponding to the points in the initial three-dimensional point cloud, v min represents the minimum vertical angle in the vertical angle corresponding to the points in the initial three-dimensional point cloud, h angle represents the horizontal angle of the points in the initial three-dimensional point cloud on the XY plane, h res represents the vertical angle difference between two adjacent points in the same laser radar ring of the initial three-dimensional point cloud, and n points represents the points in the initial three-dimensional point cloud and h resThe number of points in the laser radar ring, row represents the number of rows of points in the distance image to be optimized, and column represents the number of columns of points in the distance image to be optimized. According to the obtained number of rows and columns, the radar identifier of each point in the initial three-dimensional point cloud can be determined, such as the radar identifier (1, 1) representing the point on the first row and the first column in the distance image to be optimized. According to the corresponding vertical angle of each row, the corresponding relationship between the initial three-dimensional point cloud and the distance image to be optimized can be determined, such as the radar identifier of a point in the initial three-dimensional point cloud being (1, 2) and the vertical angle being 25 degrees, and the corresponding point in the distance image to be optimized being on the first row and the second column. The depth value of each point in the distance image to be optimized can be the pixel value of each point in the corresponding initial three-dimensional point cloud.

[0056] S202, performing back projection processing on the points in the distance image to be optimized to obtain a three-dimensional point cloud to be converted.

[0057] Specifically, the points in the distance image to be optimized can be back projected into three-dimensional points to obtain the three-dimensional point cloud to be converted.

[0058] S203, determining the Euclidean distance between the points in the three-dimensional point cloud to be converted and the corresponding points in the initial three-dimensional point cloud, and screening out the points in the three-dimensional point cloud to be converted corresponding to the Euclidean distance less than the preset distance threshold to obtain a target three-dimensional point cloud.

[0059] Specifically, the Euclidean distance between the points in the three-dimensional point cloud to be converted and the corresponding points in the initial three-dimensional point cloud can be calculated, and the points with smaller Euclidean distance, i.e. the points less than the preset distance threshold, are retained to obtain the target three-dimensional point cloud.

[0060] S204, determining an initial distance image according to the number of points in the laser radar ring in the target three-dimensional point cloud, the vertical angle corresponding to the points, and the horizontal angle of the points.

[0061] The total number of rows of initial points in the initial distance image is the same as the total number of rows of points in the distance image to be optimized, and the total number of columns of initial points in the initial distance image is less than the total number of columns of points in the distance image to be optimized.

[0062] Specifically, as described in step 201 above, the number of points in the laser radar ring in the target three-dimensional point cloud, the vertical angle corresponding to the points, and the horizontal angle of the points can also be used to determine the initial distance image.

[0063] S205, determining a first preset window range of each initial point in the initial distance image.

[0064] The initial point is in the first preset window range.

[0065] Exemplarily, the first preset window range can be preset as 5*5 pixels (px), and the initial point can be at a preset position in the first preset window range, such as a center position in the first preset window range.

[0066] S206, for each initial point, determining a relative vertical angle embedding vector according to the vertical angle corresponding to the current initial point and the current first window point.

[0067] wherein the current first window point is all initial points in the current first preset window range except the initial point corresponding to the first preset window range.

[0068] Specifically, for each initial point in the initial distance image, the vertical angle corresponding to the current first window point and the vertical angle corresponding to the current initial point can be processed by using a preset vector processing method, such as using a Transformer model, to obtain the relative vertical angle embedding vector of the current initial point, so that the relative vertical angle embedding vector of each initial point can be obtained.

[0069] S207, for each initial point, determining a relative distance embedding vector according to the distance between the current initial point and the current first window point.

[0070] Specifically, as described above, the distance between the current first window point and the current initial point can be converted into a relative distance embedding vector by using a Transformer model, so that the distance embedding vector of each initial point can be obtained.

[0071] S208, inputting the relative distance embedding vector, the relative vertical angle embedding vector and the initial distance image into a preset feature encoder to obtain a depth feature map.

[0072] wherein the preset feature encoder is determined based on an enhanced deep super-resolution network.

[0073] Specifically, Figure 3 is a schematic diagram of a preset feature encoder processing data, as shown in Figure 3 The relative distance embedding vector, the relative vertical angle embedding vector and the initial distance image can be input into the preset enhanced deep super-resolution network (preset feature encoder) to obtain the depth feature map.

[0074] S209, determining the alignment data at the current time according to the prediction weight vector at the last time and the pose information of the preset vehicle at the current time.

[0075] Specifically, if the current time is t, the prediction weight vector at time t-1 and the preset vehicle pose information at time t can be determined as the alignment data at time t, wherein the pose information can be in the form of a pose matrix. For image-form data, the last time can also be understood as the last frame.

[0076] S210, input the depth feature map at the current time and the alignment data at the current time into the first preset model to obtain the prediction weight vector at the current time, and determine the prediction distance image according to the prediction weight vector.

[0077] The first preset model is determined based on a time self-attention mechanism and a multi-head attention mechanism.

[0078] Specifically, if the current time is t, the depth feature map at time t and the alignment data at time t can be input into the first preset model, so that the prediction weight vector at time t and the prediction distance image can be obtained. The first preset model can include an encoding layer and a decoding layer, and the encoding layer and the decoding layer can include a multi-head attention layer, a fully connected layer with a residual connection, and a normalization layer, etc. The multi-head attention mechanism can extract feature information in multiple dimensions to obtain a more accurate prediction weight vector.

[0079] S211, determine the target distance image according to the initial distance image and the prediction distance image, and determine the target three-dimensional point cloud according to the preset polynomial difference algorithm and the target distance image.

[0080] The method for enhancing point cloud provided by the embodiment of the application first screens the points in the three-dimensional point cloud to be converted according to the Euclidean distance, and determines the initial distance image according to the points obtained after screening. Then, the relative distance embedding vector, the relative vertical angle embedding vector, and the initial distance image are input into the preset feature encoder to obtain the depth feature map. Then, the depth feature map and the alignment data are input into the first preset model to obtain the prediction weight vector, and the prediction distance image is determined according to the prediction weight vector. Then, the target distance image is determined according to the initial distance image and the prediction distance image. Finally, the target three-dimensional point cloud is obtained by using the preset polynomial difference algorithm and the target distance image. Compared with the traditional method, the method can filter out the empty points (empty pixels) in the distance image to be optimized by screening, avoiding the negative influence of the empty points on the determination of the prediction distance image and the target three-dimensional point cloud in the later stage. By inputting the relative distance embedding vector, the relative vertical angle embedding vector, and the initial distance image into the preset feature encoder, the noise points in the depth feature map can be reduced, and the representation ability of the target three-dimensional point cloud for the real environment can be improved. The time correlation between the same object in the previous time and the current time is established, and the prediction weight vector can be adjusted in real time with the movement of the preset vehicle, thereby improving the enhancement effect of the point cloud.

[0081] Embodiment three

[0082] Figure 4 A flowchart of a method for enhancing a point cloud is provided for the embodiment two of the present application. The technical solution of the embodiment of the present application is further optimized on the basis of the above-mentioned optional technical solutions, and another specific way of enhancing a point cloud is given.

[0083] Optionally, the determining the predicted distance image according to the predicted weight vector comprises: inputting the predicted weight vector into a second preset model to obtain an initial weight image, wherein the second preset model comprises a full connection layer and a classification layer, and the initial weight image comprises weight points corresponding to a plurality of initial weight values; determining a second preset window range of the weight points in the initial weight image, wherein the weight points are in the second preset window range; for each second preset window range, filtering out a preset number of second window points with the largest weight values according to the sizes of the weight values of the current second window points to obtain target weight points, wherein the current second window points are all the weight points in the current second preset window range; and performing a preset operation on the weight values corresponding to the target weight points and the corresponding depth values to obtain predicted depth values, and determining the predicted distance image according to the predicted depth values, wherein the preset operation comprises an addition operation and a multiplication operation. The advantage of this setting is that by retaining a preset number of second window points (target weight points) with the largest weight values, noise values can be avoided when determining the predicted depth values, and noise points can be avoided when determining the predicted distance image.

[0084] Optionally, the determining the target three-dimensional point cloud according to the preset polynomial difference algorithm and the target distance image comprises: determining a plurality of types of target partitions in the target distance image according to the sizes of the quotients of the radar identifiers of initial points and the vertical angles corresponding to the initial points in the target distance image, wherein the differences between the quotients corresponding to the initial points in a target partition of the same type are less than a first preset difference; and processing the initial points in the plurality of types of target partitions and the predicted points in the target distance image by using the preset polynomial difference algorithm to obtain a first target three-dimensional point cloud and a second target three-dimensional point cloud, and splicing the first target three-dimensional point cloud and the second target three-dimensional point cloud to obtain the target three-dimensional point cloud. The advantage of this setting is that by partitioning the initial points contained in the target distance image according to the sizes of the quotients of the radar identifiers and the vertical angles, the initial points that cannot be directly converted into three-dimensional points by using a linear equation can be converted into three-dimensional points.

[0085] As shown in FIG. 2, the method for enhancing a point cloud provided by the embodiment two of the present application specifically comprises the following steps: Figure 4

[0086] ​S301, processing the initial three-dimensional point cloud of the preset vehicle according to the number of points in the laser radar ring in the initial three-dimensional point cloud, the vertical angle corresponding to the points and the horizontal angle of the points, to obtain a distance image to be optimized.

[0087] S302, performing back projection processing on the points in the distance image to be optimized to obtain a three-dimensional point cloud to be converted.

[0088] S303, determining the Euclidean distance between the points in the three-dimensional point cloud to be converted and the corresponding points in the initial three-dimensional point cloud, and screening out the points in the three-dimensional point cloud to be converted corresponding to the Euclidean distance less than the preset distance threshold, to obtain a target three-dimensional point cloud.

[0089] S304, determining the initial distance image according to the number of points in the laser radar ring in the target three-dimensional point cloud, the vertical angle corresponding to the points and the horizontal angle of the points.

[0090] S305, determining a first preset window range of each initial point in the initial distance image.

[0091] S306, for each initial point, determining a relative vertical angle embedding vector according to the vertical angle corresponding to the current initial point and the current first window point. Wherein, the current first window point is all initial points in the current first preset window range except the initial point corresponding to the first preset window range.

[0092] S307, for each initial point, determining a relative distance embedding vector according to the distance between the current initial point and the current first window point.

[0093] S308, inputting the relative distance embedding vector, the relative vertical angle embedding vector and the initial distance image into a preset feature encoder to obtain a deep feature map.

[0094] S309, determining the alignment data at the current time according to the prediction weight vector at the last time and the pose information of the preset vehicle at the current time.

[0095] S310, inputting the deep feature map at the current time and the alignment data at the current time into a first preset model to obtain a prediction weight vector at the current time.

[0096] S311, inputting the prediction weight vector into a second preset model to obtain an initial weight map.

[0097] Wherein, the second preset model includes a fully connected layer and a classification layer, and the initial weight map includes weight points corresponding to a plurality of initial weight values.

[0098] Specifically, the prediction weight vector can be input into the fully connected layer and the classification layer, so that an initial weight map composed of initial weight values can be obtained, and the initial weight values are the pixel values of the points in the initial weight map.

[0099] Optional, Figure 5 This is a schematic diagram of a training sample, such as... Figure 5 As shown, during the initial training of the pre-set model, the initial sample distance image can be split into a pre-set dimension. For example, the even-numbered and odd-numbered rows in a 64*64px initial sample distance image can be split into two 32*32px images. That is, the N-dimensional initial sample distance image can be split into two N / 2-dimensional images, one of which is the target sample distance image and the other is the ground truth distance image. Inputting training samples composed of multiple different target sample distance images into the pre-set model yields multiple corresponding initial training weight maps. The difference between the initial training weight map and the corresponding ground truth distance image is determined, and the initial pre-set model is trained based on the magnitude of this difference, thus obtaining the trained pre-set model. The initial pre-set model includes a pre-set initial feature encoder, a first pre-set initial model, and a second pre-set initial model. The pre-set model includes the pre-set feature encoder, the first pre-set model, and the second pre-set model mentioned above.

[0100] S312. Within the initial weight map, determine the second preset window range for each weight point.

[0101] The weighting factor is within the second preset window range.

[0102] For example, within the initial weight map, a second preset window range can be preset for the weight points corresponding to each initial weight value. For instance, the second preset window range can be preset to 10*10px, and the weight points can be set at a position within the second preset window range, such as the center position within the second preset window range.

[0103] S313. For each second preset window range, based on the weight value of the current second window point, select the preset number of second window points with the largest weight value to obtain the target weight point.

[0104] Among them, the current second window point is all the weight points within the current second preset window range.

[0105] For example, if the preset number is 4, then for each second preset window range, the 4 points with the largest weight value in the current second window points can be selected, and these 4 points are the target weight points of the current weight points.

[0106] S314. Perform preset calculations on the weight values ​​and depth values ​​corresponding to the target weight points to obtain the predicted depth values, and determine the predicted distance image based on the predicted depth values.

[0107] The preset operations include summation and multiplication.

[0108] For example, as described above, if the initial weight value of the current weight focus is p, the target weight focus is 4 points, and the weight values corresponding to the four points are k1, k2, k3, and k4, respectively, the calculation method of the predicted depth value can be p*k1+p*k2+p*k3+p*k4, so that the predicted depth value of each weight focus can be obtained, which is the depth value of the predicted distance image point.

[0109] S315, according to the initial distance image and the predicted distance image, determine the target distance image, and according to the size of the quotient value of the radar identifier of the initial point in the target distance image and the vertical angle corresponding to the initial point, determine multiple types of target partitions in the target distance image.

[0110] Among them, the difference between the quotient values corresponding to the initial points in the target partitions of the same type is less than the first preset difference.

[0111] Specifically, the trained model can be used to integrate the initial distance image into the predicted distance image, and delete the points in the predicted distance image corresponding to the initial distance image. If the dimension of the initial distance image is 32*32px and the dimension of the predicted distance image is 64*64px, the dimension of the obtained target distance image is still 64*64px. Since it is impossible to directly convert the unevenly distributed two-dimensional points to three-dimensional point cloud through linear equation, it is impossible to directly convert the initial points in the target distance image to three-dimensional points. The quotient value of the radar identifier of the initial point in the target distance image and the vertical angle corresponding to the initial point can be calculated, and the initial points with similar quotient values can be divided into the same type of partition, so that multiple types of target partitions can be determined, such as linear partition, which represents that the quotient values of the radar identifier and the vertical angle of the initial points in the partition are almost consistent, that is, the connecting line of the initial points in the partition is close to a straight line in the coordinate system composed of the radar identifier and the vertical angle.

[0112] S316, using a preset polynomial difference value algorithm to process the initial points in the multiple types of target partitions and the predicted points in the target distance image respectively, to obtain a first target three-dimensional point cloud and a second target three-dimensional point cloud, and splicing the first target three-dimensional point cloud and the second target three-dimensional point cloud to obtain a target three-dimensional point cloud.

[0113] Specifically, the initial points in the multiple types of target partitions can be processed respectively using a preset polynomial difference value algorithm to simulate the linear expression of the initial points in the target partitions, so that the first target three-dimensional point cloud can be obtained, and the predicted points in the target distance image can be directly processed using the preset polynomial difference value algorithm, so that the second target three-dimensional point cloud can be obtained. After splicing the first target three-dimensional point cloud and the second target three-dimensional point cloud, the target three-dimensional point cloud can be obtained.

[0114] Optionally, the splicing the first target three-dimensional point cloud and the second target three-dimensional point cloud to obtain a target three-dimensional point cloud comprises: splicing the first target three-dimensional point cloud and the second target three-dimensional point cloud to obtain a to-be-determined three-dimensional point cloud; dividing the to-be-determined three-dimensional point cloud into a plurality of to-be-filtered regions; and performing filtering processing on the to-be-determined three-dimensional point cloud according to the number of to-be-filtered points in the to-be-filtered regions and the distance between the to-be-filtered points to obtain the target three-dimensional point cloud. In this way, the details in the initial three-dimensional point cloud are retained while removing outliers.

[0115] Specifically, the first target three-dimensional point cloud and the second target three-dimensional point cloud can be spliced first to obtain a to-be-determined three-dimensional point cloud, and then a preset filtering algorithm is used to divide the to-be-determined three-dimensional point cloud into a plurality of to-be-filtered regions, filter the to-be-filtered regions in which the number of to-be-filtered points exceeds a preset number, and delete the points in which the distance between the to-be-filtered points is less than a preset value, so as to complete the filtering processing on the to-be-determined three-dimensional point cloud and obtain the target three-dimensional point cloud.

[0116] Optionally, Figure 6 A flowchart for determining a target point cloud is shown in FIG. 8. As shown in FIG. 8, the to-be-determined three-dimensional point cloud can be divided into a plurality of point cloud region blocks (corresponding to the block division in FIG. 7) first, and then the plurality of point cloud region blocks are filtered simultaneously in multiple threads, and the filtered point cloud is spliced to obtain the target three-dimensional point cloud. This can save processing time and improve the efficiency of generating the target three-dimensional point cloud. Figure 6 Figure 6

[0117] ​​The method for enhancing a point cloud provided by the embodiment of the present application first screens the points in the three-dimensional point cloud to be converted according to the Euclidean distance, and determines an initial distance image according to the points obtained after the screening, then inputs the relative distance embedding vector, the relative vertical angle embedding vector and the initial distance image into a preset feature encoder to obtain a deep feature map, inputs the deep feature map and alignment data into a first preset model to obtain a predicted weight vector, screens a preset number of target weight points according to the size of the initial weight value, determines a predicted distance image according to the weight value corresponding to the target weight points, determines a target distance image according to the initial distance image and the predicted distance image, and finally partitions the initial points according to the size of the quotient of the radar identifier and the vertical angle of the initial points, processes the predicted points in the target distance image and the partitioned initial points by using a preset polynomial difference algorithm to obtain a target three-dimensional point cloud. Compared with the traditional method, the present method can avoid generating noise values when determining the predicted depth value by retaining a preset number of target weight points with the largest weight values, and can avoid generating noise points when determining the predicted distance image. By partitioning the initial points contained in the target distance image according to the size of the quotient of the radar identifier and the vertical angle, the initial points that cannot be directly converted into three-dimensional points through a linear equation can be converted into three-dimensional points, thereby improving the enhancement effect of the point cloud.

[0118] Embodiment four

[0119] Figure 7 The structure diagram of the device for enhancing a point cloud provided by the embodiment three of the present application is shown in FIG. 4. As shown in the figure, the device comprises a deep feature map determination module 401, a predicted distance image determination module 402 and a three-dimensional point cloud determination module 403, wherein: Figure 7

[0120] The deep feature map determination module is configured to determine an initial distance image of an initial three-dimensional point cloud, and determine a deep feature map according to the initial distance image and a preset feature encoder, wherein the initial three-dimensional point cloud is detected by a radar configured on a preset vehicle.

[0121] The predicted distance image determination module is configured to input the deep feature map and alignment data into a first preset model to obtain a predicted weight vector, and determine a predicted distance image according to the predicted weight vector, wherein the dimension of the predicted distance image is greater than the dimension of the initial distance image, the first preset model is determined based on a time self-attention mechanism, and the alignment data is determined according to the pose information and the historical predicted weight vector of the preset vehicle.

[0122] ​A three-dimensional point cloud determination module is configured to determine a target distance image according to the initial distance image and the predicted distance image, and determine a target three-dimensional point cloud according to a preset polynomial difference algorithm and the target distance image, wherein the number of points in the target three-dimensional point cloud is greater than the number of points in the initial three-dimensional point cloud, all initial points in the initial distance image are contained in the target distance image, and the target distance image has the same dimension as the predicted distance image.

[0123] The device for enhancing a point cloud provided by the embodiment of the application first determines a depth feature map according to an initial distance image of an initial three-dimensional point cloud and a preset feature encoder, then inputs the depth feature map and alignment data into a first preset model to obtain a predicted weight vector, determines a predicted distance image according to the predicted weight vector, determines a target distance image according to the initial distance image and the predicted distance image, and finally obtains a target three-dimensional point cloud by using a preset polynomial difference algorithm and the target distance image. Compared with a traditional method, the device can improve the accuracy of the predicted weight vector and the predicted distance image by processing the depth feature map and the alignment data of the vehicle by using the preset model, the target distance image determined according to the predicted distance image contains all initial points in the initial distance image, which reduces the training time of the preset model in the early stage, improves the learning efficiency of the preset model, further improves the enhancement effect of the point cloud with uneven distribution, and solves the problem that the traditional point cloud enhancement method has poor enhancement effect on the point cloud with uneven distribution.

[0124] Optionally, the depth feature map determination module comprises:

[0125] A distance image determination unit is configured to process an initial three-dimensional point cloud of a preset vehicle to obtain an optimized distance image according to the number of points in a laser radar ring in the initial three-dimensional point cloud, the vertical angle corresponding to the points, and the horizontal angle of the points, wherein the initial three-dimensional point cloud contains a plurality of laser radar rings.

[0126] A to-be-transformed point cloud determination unit is configured to perform back projection processing on the points in the optimized distance image to obtain a to-be-transformed three-dimensional point cloud.

[0127] A target point cloud determination unit is configured to determine the Euclidean distance between the points in the to-be-transformed three-dimensional point cloud and the corresponding points in the initial three-dimensional point cloud, and screen out the points in the to-be-transformed three-dimensional point cloud corresponding to the Euclidean distances less than a preset distance threshold to obtain a target three-dimensional point cloud.

[0128] An initial image determination unit is configured to determine an initial distance image according to a number of points in a laser radar ring in the target three-dimensional point cloud, a vertical angle corresponding to the points, and a horizontal angle of the points, wherein a total number of rows of initial points in the initial distance image is the same as a total number of rows of points in the distance image to be optimized, and a total number of columns of the initial points in the initial distance image is less than a total number of columns of points in the distance image to be optimized.

[0129] Optionally, the depth feature map determination module further comprises:

[0130] A first range determination unit is configured to determine a first preset window range of each initial point in the initial distance image, wherein the initial point is in the first preset window range.

[0131] A vertical angle vector determination unit is configured to determine, for each initial point, a relative vertical angle embedding vector according to a vertical angle corresponding to a current initial point and a current first window point, wherein the current first window point is all initial points in a current first preset window range except the initial point corresponding to the first preset window range.

[0132] A distance vector determination unit is configured to determine, for each initial point, a relative distance embedding vector according to a distance between a current initial point and a current first window point.

[0133] A depth feature map determination unit is configured to input the relative distance embedding vector, the relative vertical angle embedding vector, and the initial distance image into a preset feature encoder to obtain a depth feature map, wherein the preset feature encoder is determined based on an enhanced depth super-resolution network.

[0134] Optionally, the predicted distance image determination module comprises:

[0135] An alignment data determination unit is configured to determine alignment data at a current time according to a predicted weight vector at a previous time and pose information of the preset vehicle at the current time.

[0136] A weight vector determination unit is configured to input the depth feature map at the current time and the alignment data at the current time into a first preset model to obtain a predicted weight vector at the current time, wherein the first preset model is determined based on a temporal self-attention mechanism and a multi-head attention mechanism.

[0137] Optionally, the three-dimensional point cloud determination module comprises:

[0138] A weight map determination unit is configured to input the predicted weight vector into a second preset model to obtain an initial weight map, wherein the second preset model comprises a full connection layer and a classification layer, and the initial weight map comprises weight points corresponding to a plurality of initial weight values.

[0139] a second range determining unit, configured to determine a second preset window range of the weight points in the initial weight map, wherein the weight points are in the second preset window range;

[0140] a weight point determining unit, configured to, for each of the second preset window range, filter out a preset number of second window points with the largest weight values according to the size of the weight values of the current second window points in the current second preset window range, to obtain target weight points, wherein the current second window points are all the weight points in the current second preset window range;

[0141] a depth value determining unit, configured to perform a preset operation on the weight values and the corresponding depth values of the target weight points to obtain a predicted depth value, and determine a predicted distance image according to the predicted depth value, wherein the preset operation includes an addition operation and a multiplication operation.

[0142] Optionally, the three-dimensional point cloud determining module further comprises:

[0143] a target partition determining unit, configured to determine a plurality of types of target partitions in the target distance image according to the radar identifiers of the initial points and the size of the quotient values of the vertical angles corresponding to the initial points, wherein the difference between the quotient values corresponding to the initial points in a target partition of the same type is less than a first preset difference;

[0144] a three-dimensional point cloud determining unit, configured to process the initial points in the plurality of types of target partitions and the predicted points in the target distance image by using a preset polynomial difference value algorithm, to obtain a first target three-dimensional point cloud and a second target three-dimensional point cloud, and splice the first target three-dimensional point cloud and the second target three-dimensional point cloud to obtain a target three-dimensional point cloud.

[0145] Optionally, the splicing of the first target three-dimensional point cloud and the second target three-dimensional point cloud to obtain the target three-dimensional point cloud comprises: splicing the first target three-dimensional point cloud and the second target three-dimensional point cloud to obtain a to-be-determined three-dimensional point cloud; dividing the to-be-determined three-dimensional point cloud into a plurality of to-be-filtered regions, and performing filtering processing on the to-be-determined three-dimensional point cloud according to the number of to-be-filtered points in the to-be-filtered regions and the distance between the to-be-filtered points, to obtain the target three-dimensional point cloud.

[0146] The device for enhancing a point cloud provided in the embodiments of the present application can execute the method for enhancing a point cloud provided in any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of executing the method.

[0147] Embodiment five

[0148] Figure 8A structural diagram of an electronic device 40 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as in-vehicle head units, laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application described and / or claimed in this document.

[0149] As shown in Figure 8 The electronic device 40 includes at least one processor 41, and memory, such as read-only memory (ROM) 42, random access memory (RAM) 43, etc., communicatively connected to the at least one processor 41, where the memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes according to the computer programs stored in the read-only memory (ROM) 42 or loaded into the random access memory (RAM) 43 from the storage unit 48. Various programs and data required for the operation of the electronic device 40 can also be stored in the RAM 43. The processor 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0150] Various components in the electronic device 40 are connected to the I / O interface 45, including an input unit 46, such as a keyboard, a mouse, etc., an output unit 47, such as various types of displays, speakers, etc., a storage unit 48, such as a magnetic disk, an optical disk, etc., and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0151] The processor 41 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 41 performs various methods and processes described above, such as the method of enhancing a point cloud.

[0152] In some embodiments, the method of enhancing a point cloud can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 48. In some embodiments, parts or all of the computer program can be loaded and / or installed onto electronic device 40 via, e.g., ROM 42 and / or communication unit 49. When the computer program is loaded onto RAM 43 and executed by processor 41, one or more steps of the above-described method of enhancing a point cloud can be performed. Alternatively, in other embodiments, processor 41 can be configured to perform the method of enhancing a point cloud by other means, e.g., by way of firmware.

[0153] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, specially designed application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0154] Computer programs used to implement the methods of the application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor of the machine, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0155] The computer device provided above can be used to execute the method of enhancing a point cloud provided by any of the above embodiments, and has the corresponding functions and advantages.

[0156] Embodiment Six

[0157] In the context of the present application, the computer readable storage medium can be a tangible medium, the computer executable instructions of which, when executed by a computer processor, serve to perform the method of enhancing a point cloud, the method comprising:

[0158] determine an initial distance image of the initial three-dimensional point cloud, and determine a deep feature map according to the initial distance image and a preset feature encoder, wherein the initial three-dimensional point cloud is detected by a radar arranged on a preset vehicle;

[0159] input the deep feature map and alignment data into a first preset model to obtain a predicted weight vector, and determine a predicted distance image according to the predicted weight vector, wherein a dimension of the predicted distance image is greater than a dimension of the initial distance image, the first preset model is determined based on a time self-attention mechanism, and the alignment data is determined according to pose information of the preset vehicle and a historical predicted weight vector;

[0160] determine a target distance image according to the initial distance image and the predicted distance image, and determine a target three-dimensional point cloud according to a preset polynomial difference value algorithm and the target distance image, wherein a number of points in the target three-dimensional point cloud is greater than a number of points in the initial three-dimensional point cloud, all initial points in the initial distance image are contained in the target distance image, and a dimension of the target distance image is the same as a dimension of the predicted distance image.

[0161] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium will include one or more wires, portable computer disks, hard disk drives, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disc read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0162] The computer device provided above can be used to execute the method for enhancing a point cloud provided in any of the above embodiments, and has corresponding functions and advantages.

[0163] It is worth noting that, in the above embodiment of the device for enhancing a point cloud, each unit and module included is only divided according to functional logic, but is not limited to the above division, as long as the corresponding function can be realized; in addition, the specific names of each functional unit are only for easy mutual differentiation, and do not limit the protection scope of the present application.

[0164] Note that the above merely describes preferred embodiments of the present application and the principles of the technology applied. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, modifications and substitutions can be made without departing from the scope of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A method of enhancing a point cloud, the method comprising: The method comprises the following steps: determining an initial distance image of an initial three-dimensional point cloud, and determining a deep feature map according to the initial distance image and a preset feature encoder, wherein the initial three-dimensional point cloud is detected by a radar arranged on a preset vehicle; inputting the deep feature map and alignment data into a first preset model to obtain a predicted weight vector, and determining a predicted distance image according to the predicted weight vector, wherein the dimension of the predicted distance image is greater than that of the initial distance image, the first preset model is determined based on a time self-attention mechanism, and the alignment data is determined according to pose information and historical predicted weight vectors of the preset vehicle; determining a target distance image according to the initial distance image and the predicted distance image, and determining a target three-dimensional point cloud according to a preset polynomial difference value algorithm and the target distance image, wherein the number of points in the target three-dimensional point cloud is greater than that in the initial three-dimensional point cloud, all initial points in the initial distance image are contained in the target distance image, and the dimension of the target distance image is the same as that of the predicted distance image; wherein the determining of the deep feature map according to the initial distance image and the preset feature encoder comprises: determining a first preset window range of each initial point in the initial distance image, wherein the initial point is in the first preset window range; for each initial point, determining a relative vertical angle embedding vector according to a vertical angle corresponding to the current initial point and the current first window point, wherein the current first window point is all initial points in the current first preset window range except the initial point corresponding to the first preset window range; for each initial point, determining a relative distance embedding vector according to the distance between the current initial point and the current first window point; inputting the relative distance embedding vector, the relative vertical angle embedding vector and the initial distance image into a preset feature encoder to obtain a deep feature map, wherein the preset feature encoder is determined based on an enhanced deep super-resolution network.

2. The method of claim 1, wherein, The method comprises the following steps: processing the initial three-dimensional point cloud of the preset vehicle according to the number of points in the laser radar ring in the initial three-dimensional point cloud, the vertical angle corresponding to the points and the horizontal angle of the points to obtain a distance image to be optimized, wherein the initial three-dimensional point cloud contains a plurality of laser radar rings; performing back projection processing on the points in the distance image to be optimized to obtain a three-dimensional point cloud to be converted; determining the Euclidean distance between the points in the three-dimensional point cloud to be converted and the corresponding points in the initial three-dimensional point cloud, and screening out the points in the three-dimensional point cloud to be converted corresponding to the Euclidean distance less than a preset distance threshold to obtain a target three-dimensional point cloud; determining an initial distance image according to the number of points in the laser radar ring in the target three-dimensional point cloud, the vertical angle corresponding to the points and the horizontal angle of the points, wherein the total number of rows of initial points in the initial distance image is the same as the total number of rows of points in the distance image to be optimized, and the total number of columns of initial points in the initial distance image is less than the total number of columns of points in the distance image to be optimized.

3. The method of claim 1, wherein, The inputting the depth feature map and the alignment data into the first preset model to obtain a predicted weight vector comprises: determining alignment data at the current moment according to the predicted weight vector at the previous moment and pose information of the preset vehicle at the current moment; inputting the depth feature map at the current moment and the alignment data at the current moment into the first preset model to obtain the predicted weight vector at the current moment, wherein the first preset model is determined based on a time self-attention mechanism and a multi-head attention mechanism.

4. The method according to any one of claims 1 to 3, characterized in that, The determining a predicted distance image according to the predicted weight vector comprises: inputting the predicted weight vector into a second preset model to obtain an initial weight map, wherein the second preset model comprises a fully connected layer and a classification layer, and the initial weight map comprises weight points corresponding to a plurality of initial weight values; determining a second preset window range of the weight points in the initial weight map, wherein the weight points are in the second preset window range; for each second preset window range, filtering out a preset number of second window points with the largest weight values according to the size of the weight values of the current second window points to obtain target weight points, wherein the current second window points are all weight points in the current second preset window range; performing a preset operation on the weight values corresponding to the target weight points and the corresponding depth values to obtain a predicted depth value, and determining a predicted distance image according to the predicted depth value, wherein the preset operation comprises addition and multiplication.

5. The method of claim 1, wherein, The determining a target three-dimensional point cloud according to a preset polynomial difference value algorithm and the target distance image comprises: determining a plurality of types of target partitions in the target distance image according to the size of the quotient of the radar identifier of an initial point and the vertical angle corresponding to the initial point, wherein the difference between the quotients corresponding to the initial points in a target partition of the same type is less than a first preset difference; processing the initial points in the plurality of types of target partitions and the predicted points in the target distance image using a preset polynomial difference value algorithm to obtain a first target three-dimensional point cloud and a second target three-dimensional point cloud, and splicing the first target three-dimensional point cloud and the second target three-dimensional point cloud to obtain a target three-dimensional point cloud.

6. The method of claim 5, wherein, The splicing the first target three-dimensional point cloud and the second target three-dimensional point cloud to obtain a target three-dimensional point cloud comprises: splicing the first target three-dimensional point cloud and the second target three-dimensional point cloud to obtain a to-be-determined three-dimensional point cloud; dividing the to-be-determined three-dimensional point cloud into a plurality of to-be-filtered regions, and filtering the to-be-determined three-dimensional point cloud according to the number of to-be-filtered points in the to-be-filtered regions and the distance between the to-be-filtered points to obtain a target three-dimensional point cloud.

7. An apparatus for enhancing a point cloud, the apparatus comprising: It comprises: a depth feature map determination module configured to determine an initial distance image of an initial three-dimensional point cloud, and determine a depth feature map according to the initial distance image and a preset feature encoder, wherein the initial three-dimensional point cloud is detected by a radar configured on a preset vehicle; The prediction distance image determination module is configured to input the depth feature map and alignment data into a first preset model to obtain a prediction weight vector, and determine a prediction distance image according to the prediction weight vector, wherein a dimension of the prediction distance image is greater than a dimension of the initial distance image, and the first preset model is determined based on a time self-attention mechanism, and the alignment data is determined according to pose information of the preset vehicle and a historical prediction weight vector; The three-dimensional point cloud determination module is configured to determine a target distance image according to the initial distance image and the prediction distance image, and determine a target three-dimensional point cloud according to a preset polynomial difference value algorithm and the target distance image, wherein a number of points in the target three-dimensional point cloud is greater than a number of points in the initial three-dimensional point cloud, all initial points in the initial distance image are contained in the target distance image, and a dimension of the target distance image is the same as a dimension of the prediction distance image. The depth feature map determination module includes: The first range determination unit is configured to determine a first preset window range of each initial point in the initial distance image, wherein the initial point is in the first preset window range. The vertical angle vector determination unit is configured to determine, for each initial point, a relative vertical angle embedding vector according to a vertical angle corresponding to a current initial point and a current first window point, wherein the current first window point is all initial points in a current first preset window range except the initial point corresponding to the first preset window range. The distance vector determination unit is configured to determine, for each initial point, a relative distance embedding vector according to a distance between a current initial point and a current first window point. The depth feature map determination unit is configured to input the relative distance embedding vector, the relative vertical angle embedding vector, and the initial distance image into a preset feature encoder to obtain a depth feature map, wherein the preset feature encoder is determined based on an enhanced depth super-resolution network.

8. An electronic device, comprising: The electronic device includes: at least one processor; and a memory connected to the at least one processor in communication; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the method for enhancing a point cloud according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to execute the method for enhancing a point cloud according to any one of claims 1-6 when executed.

Images