Navigation method and related device based on dual-source information fusion

Abstract

The present invention discloses a navigation method and related devices based on dual-source information fusion. The method obtains original point cloud data and performs preprocessing; clusters the preprocessed point cloud data to obtain the coordinates of the center points of the parked vehicles on both sides of the walking path; aligns and compensates the coordinates of the center points of the parked vehicles on both sides of the walking path by an alignment and compensation method, obtains the connecting line of the center points of the parked vehicles on both sides of the walking path and processes them by an average method to obtain the navigation line of the vehicle center coordinate interpolation point; inputs the preprocessed point cloud data into a multi-scale expansion semantic segmentation model to obtain the edge information of the parked vehicles on both sides of the walking path; uses an SVM algorithm to perform binary classification on the left and right sides of the parked vehicle information to obtain the vehicle edge separation navigation line on both sides; fuses the vehicle center coordinate interpolation point navigation line and the vehicle edge separation navigation line on both sides based on the average method to obtain the real-time navigation line. The present invention can obtain an efficient and accurate real-time navigation line.

Description

Technical Field

[0001] The present invention relates to the field of planning navigation technology, and in particular to a navigation method based on dual-source information fusion and related devices. Background Art

[0002] With the rapid development of autonomous navigation technology and artificial intelligence technology, the perception of information around the moving vehicle is very important. Usually there are various obstacles on both sides of the road, such as vehicles, pedestrians, curbs, etc.

[0003] As an important sensor for automatic navigation, lidar can effectively provide three-dimensional spatial information of the surrounding area by actively emitting and receiving lasers. It has the advantages of long ranging, strong anti-interference ability and high data accuracy.

[0004] At present, there are still great limitations in the detection and perception of complex surrounding information, which poses a great threat to the safety of intelligent driving. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a navigation method and related devices based on dual-source information fusion, which can obtain efficient and accurate real-time navigation lines.

[0006] In order to achieve the above object, the technical solution of the present invention is as follows:

[0007] The navigation method based on dual-source information fusion includes the following steps:

[0008] Obtain the original point cloud data detected by the vehicle-mounted laser radar and perform preprocessing;

[0009] Perform clustering on the pre-processed point cloud data to obtain the coordinates of the center points of the parked vehicles on both sides of the walking path in the vehicle coordinate system;

[0010] By using the alignment and compensation method, the coordinates of the center points of the parked vehicles on both sides of the walking path are aligned and compensated to obtain the connecting line of the center points of the parked vehicles on both sides of the walking path;

[0011] The average method is used to process the lines connecting the center points of the vehicles parked on both sides of the walking path to obtain the navigation lines of the interpolation points of the center coordinates of the vehicles;

[0012] The preprocessed point cloud data is input into the multi-scale expansion semantic segmentation model for semantic segmentation to obtain the edge information of the parked vehicles on both sides of the walking path;

[0013] The support vector regression (SVM) algorithm is used to classify the parking vehicle information on the left and right sides. The binary classification regression model is the separation guidance line of the vehicle edges on both sides.

[0014] Based on the averaging method, the vehicle center coordinate interpolation point navigation line and the vehicle body edge separation navigation lines on both sides are fused to obtain a real-time navigation line.

[0015] Preferably, the pretreatment comprises the following steps:

[0016] Remove abnormal points in the original point cloud data through the Open3D library to obtain a valid point cloud;

[0017] The valid point cloud is converted into a vehicle body coordinate system.

[0018] Preferably, clustering the pre-processed point cloud data to obtain the coordinates of the center points of the parked vehicles on both sides of the walking path in the vehicle body coordinate system specifically includes the following steps:

[0019] The neighborhood scale range is set, and the generalized DBSCAN search domain space is used to group the points within the neighborhood scale range that can be reached from the core object into the same cluster, and a total of Nl clusters are obtained on the left and Nr clusters are obtained on the right;

[0020] Calculate the coordinates of all center points of each cluster on the left and right sides to obtain the coordinates of the center points of the parked vehicles on both sides of the walking path.

[0021] Preferably, the alignment and compensation method is used to align and compensate the coordinates of the center points of the vehicles parked on both sides of the walking path to obtain a connecting line of the center points of the vehicles parked on both sides of the walking path, which specifically includes the following steps:

[0022] It is determined in turn whether there are similar points between the coordinates of the center points of the vehicles parked on the left and the right. If so, no difference is made; if not, the center points of the corresponding sides are aligned and the difference is made;

[0023] The coordinates of the center points of the left and right parked vehicles after alignment and compensation are arranged in sequence according to the size of the x-coordinates to obtain a connecting line of the center points of the parked vehicles on both sides of the walking path.

[0024] Preferably, the multi-scale dilated semantic segmentation model includes a multi-scale dilated convolution module, a channel attention module, several residual modules, a pooling layer, a stacking deactivation layer, a fully connected layer and a regression layer connected in sequence, wherein the multi-scale dilated convolution module uses convolution kernels of different sizes to perform convolution operations on the preprocessed point cloud data, and concatnates the feature maps after convolution to obtain multi-scale features; the channel attention module assigns weights to the obtained multi-scale features through squeezing and activation operations, learns the weights between different channels, and uses the obtained weights to highlight important features and suppress unimportant features; the several residual modules include a starting residual network, an intermediate residual network and an ending residual network connected in sequence, and the output features of the ending residual network are convolved and then the corresponding elements are added to the output features of the ending residual network to obtain global features; the global features are sequentially passed through the pooling layer, the stacking deactivation layer, the fully connected layer and the regression layer to obtain the edge information of the parked vehicles on both sides of the walking path.

[0025] Preferably, the pooling layer is a global average pooling layer.

[0026] Preferably, the binary regression model f S (X), as shown below:

[0027] Y=f S (X) = A T X+B (4)

[0028] Where A and B are the coefficients and constant term of the regression model;

[0029] Taking the distance ||D|| between the edges of the left and right parking vehicles and the classification regression function as the classification standard, introducing a small slack variable ε and its dual ε*, and setting the threshold of the loss function to λ, the following optimization problem is proposed:

[0030]

[0031] Based on the above content, the present invention also discloses a navigation device based on dual-source information fusion, including: a data processing module, a first navigation line acquisition module, a second navigation line acquisition module, and a fusion module, wherein:

[0032] The data processing module is used to obtain the original point cloud data detected by the vehicle-mounted laser radar and perform preprocessing;

[0033] The first navigation line acquisition module is used to perform clustering processing on the pre-processed point cloud data to obtain the coordinates of the center points of the vehicles parked on both sides of the walking path in the vehicle body coordinate system; to align and compensate the coordinates of the center points of the vehicles parked on both sides of the walking path by an alignment and compensation method to obtain a line connecting the center points of the vehicles parked on both sides of the walking path; and to process the line connecting the center points of the vehicles parked on both sides of the walking path by an average method to obtain a navigation line of an interpolation point of the vehicle center coordinates;

[0034] The second navigation line acquisition module is used to input the preprocessed point cloud data into the multi-scale expansion semantic segmentation model for semantic segmentation to obtain the edge information of the parked vehicles on both sides of the walking path; and is used to use the support vector regression SVM algorithm to perform binary classification on the information of the parked vehicles on the left and right sides. The binary classification regression model is the navigation line for separating the edges of the vehicles on both sides.

[0035] The fusion module is used to fuse the vehicle center coordinate interpolation point navigation line and the vehicle body edge separation navigation lines on both sides based on the averaging method to obtain a real-time navigation line.

[0036] Based on the above content, the present invention further discloses a computer device, including: a memory for storing a computer program; and a processor for implementing any of the above methods when executing the computer program.

[0037] Based on the above content, the present invention further discloses a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, any of the methods described above is implemented.

[0038] Based on the above technical scheme, the beneficial effects of the present invention are as follows: the present invention is based on a navigation method and related devices based on dual-source information fusion, the method uses the Open3D library to remove abnormal points in the original point cloud obtained by the vehicle-mounted radar, and converts the valid point cloud into the vehicle body coordinate system; the GDBSCAN clustering algorithm is used to obtain the coordinates of the center points of the parked vehicles on both sides of the walking path; the alignment and compensation method is used to align and compensate the coordinates of the center points of the parked vehicles on the upper and lower sides; the average method is used to obtain the navigation line of the vehicle body center coordinate interpolation point; based on multi-scale dilated convolution semantic segmentation, and the classifier obtained by fusion SVM binary classification is used as the navigation line for separating the edges of the vehicle bodies on both sides; but considering that the navigation line of the vehicle body center coordinate interpolation point is obtained based on the vehicle body center position information, the vehicle body edge information is not considered; and the vehicle body edge separation navigation line on both sides obtained by semantic segmentation based on multi-scale dilated convolution and fusion SVM binary classification is a navigation line generated by the vehicle body edge information, and the vehicle body center position information is not considered. Therefore, the two are fused to obtain an efficient and accurate real-time navigation line. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1is a flowchart of a navigation method based on dual-source information fusion in an embodiment;

[0040] Figure 2 is a schematic diagram of generating a navigation line of a vehicle body center coordinate interpolation point in a navigation method based on dual-source information fusion in an embodiment;

[0041] Figure 3 is a schematic diagram of the structure of a multi-scale dilated convolutional semantic segmentation model in an embodiment;

[0042] Figure 4 The present invention is a schematic diagram of generating navigation lines separating the edges of vehicles on both sides in a navigation method based on dual-source information fusion in an embodiment. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0044] like Figure 1 As shown, the navigation method based on dual-source information fusion in this embodiment specifically includes the following steps:

[0045] Step 1: Preprocessing of original point cloud data. Take the original point cloud data detected by the vehicle-mounted laser radar, use the Open3D library to remove abnormal points in the original point cloud data, and convert the original point cloud in the laser radar coordinate system to the vehicle body coordinate system. Let the i-th coordinate point in the original point cloud be The i-th point in the vehicle coordinate system is The conversion formula is as follows:

[0046]

[0047] Where (Δx, Δy, Δz,) is the translation factor.

[0048] Step 2: Use the GDBSCAN clustering algorithm to find the center coordinates of the vehicles parked on both sides of the walking path. The coordinates of the center point of the vehicle on the left side of the walking path are marked as The coordinates of the center point of the vehicle on the right side of the walking path are marked as The process of this step is as follows:

[0049] 2.1 Set the neighborhood scale to δ, use the generalized DBSCAN search domain space, and group the points within the neighborhood scale δ that can be reached from the core object into the same cluster. Finally, we can get a total of N on the left. l Cluster, there are N on the right r cluster;

[0050] 2.2 Calculate the center point of each cluster The coordinates are calculated as follows:

[0051]

[0052] The coordinates obtained are the center coordinates of the vehicles parked on the left and right sides of the walking path. and

[0053] Step 3: Use the alignment and compensation method to align and compensate the coordinates of the center points of the left and right parking vehicles.

[0054] First align the center point of the left parking body to make up the difference, then align the center point of the right parking body to make up the difference, and finally arrange the coordinate point sequences on the left and right sides in order. The specific implementation steps are as follows:

[0055] 3.1 Take out the first point P1 at the center of the left parking vehicle Cl The x-coordinate Query the center point of the right vehicle body The x-coordinate Is there any Coordinate points with similar distances, i.e.

[0056] 3.2 If there is no similar point, interpolate the coordinate point on the right side in Then take Find the y-coordinate of the point on the left of the two adjacent points, and then go to step 3.4;

[0057] 3.3 If there are similar points, do not make up the difference and go to step 3.4;

[0058] 3.4 Repeat 3.1-3.3 until the N on the left is l After traversing all the center points, go to step 3.5;

[0059] 3.5 Use the same method to align and compensate the left center point with the center point of the right parking vehicle as the reference;

[0060] 3.6 After the left and right sides are compensated, they are re-arranged. The coordinate points of the parking bodies on the left and right sides are arranged in order according to the size of the x coordinates, and are recorded as At this time, the number of coordinate points on the left and right sides is the same, i = 1, 2, ..., N c .

[0061] Step 4: Determine the vehicle center coordinate interpolation point navigation line, see Figure 2 .

[0062] The vehicle center coordinate interpolation point navigation line is the central axis of the left and right parking vehicle center point lines in step 3. After step 3, the number of center points on the left and right sides is the same. The x-direction coordinates on the left and right sides are either the same or the difference is very small, then the coordinates of the i-th navigation point of the center coordinate interpolation point navigation line are

[0063]

[0064] Step 5: Perform semantic segmentation based on multi-scale dilated convolution to obtain the edge information of the vehicle body.

[0065] Based on the residual network, a multi-scale expansion semantic segmentation model is proposed, and its model structure is as follows: Figure 3 As shown in the figure, first, the preprocessed point cloud radar is connected in series with the channel attention mechanism through a multi-scale dilation convolution to form a multi-scale dilation module for feature extraction. The multi-scale dilation convolution can use convolution kernels with different dilation rates to extract multi-scale features, avoiding the incomplete expression of feature information at a single scale and causing the loss of weak features. The channel attention mechanism can assign weights to the obtained multi-scale features, learn the weights between different channels, and use the obtained weights to highlight important features and suppress unimportant features; then the feature map obtained after the multi-scale dilation module is passed through multiple residual modules; in order to obtain features with stronger expressive power, the extracted features are compressed and aggregated using GAP (Global Average Pooling), and a random inactivation (dropout) layer is added before the fully connected layer to prevent overfitting problems; finally, regression prediction is performed through the regression layer. In the figure, "Conv 3×3, 2, 32" means that the convolution kernel size is 3×3, the step size is 2, the number of channels is 32, and BN (Batch Normalization) and Relu (Rectified Linear Unit) are performed before each convolution.

[0066] Through this method, the parked vehicle body and the surrounding environment are semantically divided, and the edge information of the parked vehicle body on the left and right sides of the walking path is obtained.

[0067] Step 6: Use the support vector regression SVM algorithm to classify the parking vehicle information on the left and right sides. The binary classification regression model is the separation guidance line of the vehicle edges on both sides. The general form is as follows:

[0068] Y=f S (X) = A T X+B (4)

[0069] Where A and B are the coefficients and constant term of the regression model.

[0070] The coordinate point clouds on the left and right sides of the vehicle body are Taking the distance ||D|| between the edges of the left and right parking vehicles and the classification regression function as the classification standard, introducing a small slack variable ε and its dual ε*, and setting the threshold of the loss function to λ, the following optimization problem is proposed:

[0071]

[0072] By optimizing the above model, we can obtain the classification regression function f S (X), this function is the guide line separating the edges of the two sides of the vehicle body, see Figure 4 .

[0073] Step 7: Dual information navigation line fusion.

[0074] Since the interpolation point of the center coordinate of the vehicle body is based on the center position information of the vehicle body, the edge information of the vehicle body is not considered; and the vehicle edge separation navigation line obtained by semantic segmentation based on multi-scale dilated convolution and fusion of SVM binary classification is a navigation line generated by the edge information of the vehicle body, without considering the center position information of the vehicle body, and the two navigation lines are complementary. Here, the navigation line of the center coordinate interpolation point of the vehicle body and the navigation line of the vehicle edge separation on both sides are fused. The specific steps are as follows:

[0075] 7.1 Perform quintic spline curve fitting on the vehicle center coordinate interpolation navigation point obtained in step 3 to obtain the center coordinate interpolation navigation line f Z (P);

[0076] 7.2 f Z (P) and f obtained in step 6 S (P) is averaged according to the corresponding coordinate points, and the resulting navigation line f(P) is calculated as follows:

[0077]

[0078] 7.3 Divide the navigation line into N points, then the coordinates of the i-th navigation point are:

[0079]

[0080] y i =f(x i )(7)

[0081] It should be understood that, although the various steps in the above-mentioned flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above-mentioned flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0082] The above description is only a preferred implementation of a navigation method and related device based on dual-source information fusion disclosed by the present invention, and is not intended to limit the protection scope of the embodiments of this specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of this specification shall be included in the protection scope of the embodiments of this specification.

Claims

1. A navigation method based on dual-source information fusion, characterized in that: The steps include: Obtain the original point cloud data detected by the vehicle-mounted laser radar and perform preprocessing; Perform clustering on the pre-processed point cloud data to obtain the coordinates of the center points of the parked vehicles on both sides of the walking path in the vehicle coordinate system; By using the alignment and compensation method, the coordinates of the center points of the parked vehicles on both sides of the walking path are aligned and compensated to obtain the connecting line of the center points of the parked vehicles on both sides of the walking path; The average method is used to process the lines connecting the center points of the vehicles parked on both sides of the walking path to obtain the navigation lines of the interpolation points of the center coordinates of the vehicles; The preprocessed point cloud data is input into the multi-scale expansion semantic segmentation model for semantic segmentation to obtain the edge information of the parked vehicles on both sides of the walking path. The multi-scale expansion semantic segmentation model includes a multi-scale expansion convolution module, a channel attention module, a plurality of residual modules, a pooling layer, a stacking inactivation layer, a fully connected layer and a regression layer connected in sequence. The multi-scale expansion convolution module uses convolution kernels of different sizes to perform convolution operations on the preprocessed point cloud data, and concatinates the feature maps after convolution to obtain multi-scale features. The channel attention module performs convolution operations on the preprocessed point cloud data, and concatinates the feature maps after convolution to obtain multi-scale features. The module distributes weights to the obtained multi-scale features through squeezing and activation operations, learns weights between different channels, and uses the obtained weights to highlight important features and suppress unimportant features; the plurality of residual modules include a starting residual network, an intermediate residual network and an ending residual network connected in sequence, the output features of the ending residual network are convolved and then the corresponding elements are added to the output features of the ending residual network to obtain global features; the global features are sequentially passed through a pooling layer, a stacking inactivation layer, a fully connected layer and a regression layer to obtain edge information of the parked vehicles on both sides of the walking path; The support vector regression (SVM) algorithm is used to classify the parking vehicle information on the left and right sides. The binary classification regression model is the separation guidance line of the vehicle edges on both sides. Based on the averaging method, the vehicle center coordinate interpolation point navigation line and the vehicle body edge separation navigation lines on both sides are fused to obtain a real-time navigation line.

2. The navigation method based on dual-source information fusion according to claim 1, characterized in that: The pre-processing comprises the following steps: Remove abnormal points in the original point cloud data through the Open3D library to obtain a valid point cloud; The valid point cloud is converted into a vehicle body coordinate system.

3. The navigation method based on dual-source information fusion according to claim 1, characterized in that: The clustering process of the pre-processed point cloud data to obtain the coordinates of the center points of the parked vehicles on both sides of the walking path in the vehicle body coordinate system specifically includes the following steps: Set the neighborhood scale range, use the generalized DBSCAN search domain space, and group the points within the neighborhood scale range that can be reached from the core object into the same cluster, and get a total of N on the left. l Cluster, there are N on the right r cluster; Calculate the coordinates of all center points of each cluster on the left and right sides to obtain the coordinates of the center points of the parked vehicles on both sides of the walking path.

4. The navigation method based on dual-source information fusion according to claim 3 is characterized in that: The alignment and compensation method is used to align and compensate the coordinates of the center points of the vehicles parked on both sides of the walking path to obtain a connecting line of the center points of the vehicles parked on both sides of the walking path, specifically including the following steps: It is determined in turn whether there are similar points between the coordinates of the center points of the vehicles parked on the left and the right. If so, no difference is made; if not, the center points of the corresponding sides are aligned and the difference is made; The coordinates of the center points of the left and right parked vehicles after alignment and compensation are arranged in sequence according to the size of the x-coordinates to obtain a connecting line of the center points of the parked vehicles on both sides of the walking path.

5. The navigation method based on dual-source information fusion according to claim 1, characterized in that: The pooling layer is a global average pooling layer.

6. The navigation method based on dual-source information fusion according to claim 1, characterized in that: The binary regression model is as follows: Y=f S (X)=A T X+B (4) Where A and B are the coefficients and constant term of the regression model; Taking the distance ||D|| between the edges of the left and right parking vehicles and the classification regression function as the classification standard, introducing a small slack variable ε and its dual ε*, and setting the threshold of the loss function to λ, the following optimization problem is proposed:

7. A navigation device based on dual-source information fusion, characterized in that: include: Data processing module, first navigation line acquisition module, second navigation line acquisition module, fusion module, wherein, The data processing module is used to obtain the original point cloud data detected by the vehicle-mounted laser radar and perform preprocessing; The first navigation line acquisition module is used to perform clustering processing on the pre-processed point cloud data to obtain the coordinates of the center points of the vehicles parked on both sides of the walking path in the vehicle body coordinate system; to align and compensate the coordinates of the center points of the vehicles parked on both sides of the walking path by an alignment and compensation method to obtain a line connecting the center points of the vehicles parked on both sides of the walking path; and to process the line connecting the center points of the vehicles parked on both sides of the walking path by an average method to obtain a navigation line of an interpolation point of the vehicle center coordinates; The second navigation line acquisition module is used to input the preprocessed point cloud data into the multi-scale expansion semantic segmentation model for semantic segmentation to obtain the edge information of the parked vehicles on both sides of the walking path; it is used to use the support vector regression SVM algorithm to classify the parked vehicle information on the left and right sides. The binary classification regression model is the navigation line separating the edges of the vehicles on both sides. The multi-scale expansion semantic segmentation model includes a multi-scale expansion convolution module, a channel attention module, a plurality of residual modules, a pooling layer, a stacking deactivation layer, a fully connected layer and a regression layer connected in sequence. The multi-scale expansion convolution module uses convolution kernels of different sizes to perform convolution operations on the preprocessed point cloud data. The feature graphs are concatinated to obtain multi-scale features; the channel attention module assigns weights to the obtained multi-scale features through squeezing and activation operations, learns weights between different channels, and uses the obtained weights to highlight important features and suppress unimportant features; the plurality of residual modules include a starting residual network, an intermediate residual network, and an ending residual network connected in sequence, and the output features of the ending residual network are convolved and then added to the output features of the ending residual network by corresponding elements to obtain global features; the global features are sequentially passed through a pooling layer, a stacking inactivation layer, a fully connected layer, and a regression layer to obtain edge information of the parked vehicles on both sides of the walking path; The fusion module is used to fuse the vehicle center coordinate interpolation point navigation line and the vehicle body edge separation navigation lines on both sides based on the averaging method to obtain a real-time navigation line.

8. A computer device, characterized in that: The device comprises: a memory for storing a computer program; A processor, configured to implement the method according to any one of claims 1 to 6 when executing the computer program.

9. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images