Road tunnel scene detection method based on millimeter wave radar point cloud

Abstract

The invention belongs to the technical field of radar detection in automatic driving, and particularly relates to a road tunnel scene detection method based on millimeter-wave radar point clouds, which comprises the following steps: acquiring point cloud data on the left and right sides of a road by using a vehicle millimeter-wave radar; n pieces of point cloud data in the limited space range are taken from the left side and the right side respectively; collecting 2N pieces of point cloud data collected left and right into a 4N-dimensional vector X; inputting the vector X into a machine learning reasoning model constructed based on SVM or DNN for training; and deploying the trained model to an ECU, an MCU or a domain controller of a vehicle end, and performing real-time reasoning according to point cloud data acquired in real time to obtain a final tunnel scene type value. According to the technical scheme, the characteristic that false point clouds are generated due to back-and-forth reflection of millimeter wave radar rays in a tunnel scene is fully utilized, the process is simple, parameters needing to be debugged are few, and the detection accuracy of the technical scheme can be further improved along with more and more collected and labeled data.

Description

Technical Field

[0001] The present invention belongs to the technical field of radar detection in autonomous driving, and particularly relates to a method for detecting road tunnel scenes based on millimeter-wave radar point clouds. Background Art

[0002] The detection of specific road environments or road scenes is one of the key issues concerned in the field of autonomous driving perception. For example, road scenes such as bridges, gantries, uphill and downhill sections, curves, and tunnels will affect the driving behavior of human drivers. Similarly, the perception of this information is also very important for autonomous driving. Especially for tunnel scenes, the light is relatively dim, the detection ability of cameras becomes weak, resulting in more false detections, the GPS signal will also be missing, and there are walls all around, and the millimeter-wave radar point clouds are also very scattered. Therefore, it is necessary to adjust the algorithms of autonomous driving, especially the perception algorithms, for special processing, that is, the algorithm needs to be switched after entering the tunnel so that the autonomous driving vehicle can stably complete driving in the tunnel. Thus, the detection of tunnel scenes is particularly important, especially in high-speed scenarios.

[0003] At present, the detection schemes for tunnel scenes on roads are mainly the following three schemes:

[0004] 1) Detection based on cameras, which is a relatively common scheme. Generally, it is implemented by deep learning, detecting images. Generally, a separate model is used to detect the scene, and usually, it does not share the same model with obstacle detection. The advantages of this detection scheme are that the technology is relatively mature and there are many optional models; the disadvantage is that it is based on a deep neural network, consuming a large amount of computing power.

[0005] 2) Detection based on lidar. Lidar can obtain the height information of target objects and has an advantage in obtaining the accuracy of tunnels. The advantages of this detection scheme are relatively high accuracy, and the disadvantages are very high costs. The hardware cost of lidar itself is very high, the amount of point cloud data is large, and the cost of data processing is also very high.

[0006] 3) Scheme based on high-precision maps. It uses inertial navigation for positioning and matches the position information with high-precision maps to obtain tunnel information. The advantages of this detection scheme are simple application and low computing consumption. The disadvantages are that the signal of inertial navigation equipment is very poor in mountainous areas, especially almost no signal inside the tunnel, and it is difficult to obtain accurate and real-time tunnel information, and the drawing cost of high-precision maps itself is very high.

[0007] In view of the above problems, the existing patent CN111699408 A discloses a tunnel scene detection method and a millimeter-wave radar. The technical solution disclosed in this patent is to detect the tunnel scene through the millimeter-wave radar point cloud. The stationary millimeter-wave radar point cloud within a preset area is obtained by the in-vehicle millimeter-wave radar, the point cloud is clustered, and it is determined whether the vehicle has entered the tunnel according to the length of the clustering area; when the vehicle enters the tunnel, the false point cloud generated by the back-and-forth reflection in the tunnel is removed according to the preset suppression condition, the stability of the millimeter-wave radar is improved, the clustering effect is enhanced, and thus the accuracy of tunnel detection is improved.

[0008] The disadvantages of this technical solution are as follows:

[0009] First, the steps of detecting the tunnel scene are relatively cumbersome. It is necessary to cluster the millimeter-wave radar point cloud first and then make a judgment according to the length of the clustering target. The quality of the clustering method will seriously affect the final judgment result. In addition, the judgment of false point cloud is added, increasing the steps and the number of parameters to be debugged.

[0010] Second, when suppressing false targets (or point clouds), it is necessary to obtain the signal intensity threshold of the millimeter-wave radar. For non-millimeter-wave radar manufacturers, that is, ordinary millimeter-wave radar users, it may not be possible to obtain the signal intensity value of the millimeter-wave radar, and this threshold also needs to be adjusted, further increasing the application threshold.

[0011] Third, all the parameters (various thresholds) of this method must be manually debugged, and the application process is relatively cumbersome. The increase in the amount of collected data does not necessarily improve the detection accuracy.

[0012] Fourth, this method does not effectively utilize the characteristic of the back-and-forth reflection of the millimeter-wave radar rays in the tunnel scene, that is, it does not effectively utilize the characteristic of excessive false targets. Instead, it is necessary to remove this feature, which may reduce the detection effect. Summary of the Invention

[0013] The purpose of the present invention is to provide a road tunnel scene detection method based on millimeter-wave radar point cloud to solve the following problems:

[0014] 1. It can accurately obtain the tunnel scene information on the road, including four scenarios: the entrance of the tunnel, the exit of the tunnel, inside the tunnel, and outside the tunnel.

[0015] 2. It can complete the detection of the tunnel scene with relatively low software and hardware costs and data acquisition and annotation costs, and can obtain the tunnel scene information relatively quickly with less computing power consumption.

[0016] 3. It can effectively utilize the characteristic of the back-and-forth reflection of the millimeter-wave radar rays in the tunnel scene.

[0017] To achieve the above object, the present application is implemented through the following technical solutions:

[0018] A method for detecting road tunnel scenes based on millimeter-wave radar point clouds, comprising the following steps:

[0019] S1. Use the millimeter-wave radar of the host vehicle to collect point cloud data on both sides of the road;

[0020] S2. For the point cloud data collected in step S1, take N point cloud data within a limited space range on each of the left and right sides;

[0021] S3. Combine the 2N point cloud data collected on the left and right in step S2 into a 4N-dimensional vector X;

[0022] S4. Input the vector X into a machine learning inference model constructed based on SVM or DNN for training;

[0023] S5. Deploy the trained model to the ECU, MCU or domain controller on the vehicle side, and perform real-time inference based on the real-time collected point cloud data to obtain the final tunnel scene type value.

[0024] Further, in step S1, using the millimeter-wave radar of the host vehicle to collect point cloud data on both sides of the road, including collecting the point cloud data of the millimeter-wave radar in tunnel scenes, non-tunnel scenes, tunnel entrance and tunnel exit on the road, extracting some of the data, and performing cleaning, removing the point cloud data behind the vehicle body, removing the dynamic point cloud data, retaining the point cloud data on both sides of the road, and performing type annotation.

[0025] Further, in step S2, taking N point cloud data within a limited space range on each of the left and right sides is the forward view range of the host vehicle, and intercepting the point cloud data within the rectangles on both sides at a set distance.

[0026] Further, in step S3, the arrangement method of the 2N point cloud data is determined by the arrangement method of the points inside the vector X during the training of the machine learning model, and after the arrangement order is determined, the arrangement order cannot be changed during the subsequent model training and inference stages.

[0027] Further, in step S5, the classification results obtained after the inference operation include tunnel entrance, tunnel exit, inside the tunnel and outside the tunnel.

[0028] Further, the method for determining the tunnel scene type value is that when the obtained tunnel scene type value is within the set threshold range, it is determined that the point cloud data distribution category is the tunnel entrance, when it is greater than or equal to the maximum threshold, it is determined that the point cloud data distribution category is inside the tunnel, and when it is less than the minimum threshold, it is determined that the point cloud data distribution category is outside the tunnel.

[0029] Furthermore, when not on a highway, the detection accuracy is improved by enhancing the capabilities of the model.

[0030] The beneficial effects of the present invention are as follows:

[0031] 1. Using a millimeter-wave radar for tunnel scene detection on the road has a low cost, much lower than that of a lidar.

[0032] 2. The point cloud of the millimeter-wave radar has a much lower data volume compared to the point cloud of the lidar and the image of the camera, and the data transmission cost and processing cost consumed are much lower.

[0033] 3. If using traditional machine learning methods (SVM) for data classification, the required computing power is much lower than that of deep learning. It can run on an ordinary CPU, runs quickly and in real-time. Even if using a DNN (deep neural network), the required network structure will not be particularly complex because the data to be processed is not large and there are not many features.

[0034] 4. The millimeter-wave radar is not affected by light and rain / fog weather. The tunnel detection stability of this solution is not affected by any weather environment. The camera is affected by light and rain / fog weather, and the lidar is affected by rain / fog weather.

[0035] 5. Compared with the existing millimeter-wave radar tunnel detection solutions, this technical solution makes full use of the characteristics that false point clouds (or targets) are generated by the back-and-forth reflection of millimeter-wave radar rays in the tunnel scene. It uses fewer steps, the process is not complex, and the parameters to be debugged are not many. Moreover, as the collected and labeled data increases, the detection accuracy of this technical solution will be further improved and the stability will also be better. That is to say, it can achieve the improvement of the performance of this solution driven by data. The machine learning model used is mature and has a large space for adjustment and improvement at the same time. Description of the Drawings

[0036] Figure 1 This is a diagram showing the technical solution of the present invention. This diagram marks the distribution of millimeter-wave radar point clouds at the tunnel entrance. This solution uses a vehicle-mounted front millimeter-wave radar to obtain point clouds. Triangles represent millimeter-wave radar point clouds, the thick blue line represents the road edge on the highway, and the black dashed box represents the point cloud area to be screened. There are two screening areas on the left and right, and N points are collected in each dashed box.

[0037] Figure 2 It is a schematic diagram of the point cloud distribution when entering the tunnel entrance.

[0038] Figure 3 It is a schematic diagram of the point cloud distribution when exiting the tunnel entrance.

[0039] Figure 4 It is a schematic diagram of the point cloud distribution when inside the tunnel.

[0040] Figure 5 It is a schematic diagram of the point cloud distribution when outside the tunnel.

[0041] Figure 6 It is a flowchart of the technical solution of the present invention. Specific embodiments

[0042] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings. The following embodiments are merely exemplary and can only be used to explain and illustrate the technical solution of the present invention, rather than being construed as a limitation of the technical solution of the present invention.

[0043] The present application provides a method for detecting road tunnel scenes based on millimeter-wave radar point clouds. The point cloud data on the left and right sides of the road (the part close to the road edge) is collected by an in-vehicle millimeter-wave radar to classify the point cloud data. The classification categories are four types: entering the tunnel entrance, exiting the tunnel entrance, inside the tunnel, and outside the tunnel. As Figures 1 to 5 shown, the detection of tunnel scenes is achieved through the classification of point clouds. The following is a specific introduction.

[0044] The present application uses a 2D millimeter-wave radar (if it is a 4D millimeter-wave radar, it is also feasible. The scheme is the same, and only the height information needs to be removed and converted into 2D). There are no special regulations on the installation position of the millimeter-wave radar on the vehicle itself, as long as it can ensure the acquisition of point cloud data on the left and right sides of the road. As Figure 1 shown, a forward millimeter-wave radar is used here to acquire the point cloud data on both sides of the road.

[0045] The point cloud information of the millimeter-wave radar includes the position information of the point cloud (represented in the form of polar coordinates or Cartesian coordinates), signal strength, and velocity information. After the signal emitted by the millimeter-wave radar is repeatedly reflected by the inner wall in the tunnel, there will be many false targets in the generated point cloud distribution, and relatively dense point clouds will be generated at the road edge part. This will make the point cloud distribution inside the tunnel very different from the point cloud distribution outside the tunnel. As Figure 1 shown, it is precisely based on this difference in point cloud distribution that provides conditions for us to classify the point clouds. Figures 2 to 5 The millimeter-wave radar point cloud distributions in four scenarios of entering the tunnel entrance, exiting the tunnel entrance, inside the tunnel, and outside the tunnel are respectively shown. Obviously, as long as the collected millimeter-wave radar point clouds are classified using an algorithm, the purpose of tunnel scene detection can be achieved.

[0046] One point that needs to be noted is that the collected point clouds should reflect the information of the road, mainly static point clouds with an absolute speed very small (close to 0 or 0). When there are dynamic obstacles beside, the obtained dynamic point cloud data should be discarded when performing tunnel scene detection.

[0047] The classification algorithm adopted in this application is a classification algorithm in machine learning and is supervised, such as SVM (Support Vector Machine) and DNN (Deep Neural Network), and the data used for training needs to be labeled.

[0048] Reference Figure 6 As shown, this application provides a method for detecting road tunnel scenes based on millimeter-wave radar point clouds, including the following steps:

[0049] S1. Collect data, build a model and train it. Use the millimeter-wave radar on the vehicle to collect the point cloud data on the left and right sides of the road. As mentioned before, the number and position of the millimeter radars used are not limited, as long as the point cloud data on the left and right sides of the road can be collected normally, and then filter out the static point cloud data.

[0050] Collect the point cloud data of the millimeter-wave radar for tunnel scenes, non-tunnel scenes, tunnel entrance and tunnel exit on the road, extract some of the data, and clean it, remove the part behind the vehicle body, remove the dynamic point cloud data, retain the data on the left and right sides of the road, and perform type annotation. Each frame of data is annotated with a certain type, that is, one of the 4 types. The 4 types are 4 scenes. As for which 4 integer values, it can be defined relatively freely. In this application, it is defined as 0, 1, 2, 3, which represent outside the tunnel, inside the tunnel, tunnel entrance and tunnel exit respectively.

[0051] S2. As Figure 1 shown, perform an interception within a certain range on the collected data, take N points within the defined spatial range on the left and right respectively. The point-taking range and the number of points taken need to be adjusted according to repeated experiments. For the front view range of the vehicle, intercept the points within the rectangles on both sides of a set distance, that is, Figure 1 the points within the dashed box in, take N points on each side. The point-taking range and the number of points taken need to be adjusted according to repeated experiments.

[0052] S3. Build a vector: Assemble the 2N collected points into a 4N-dimensional vector X. Since each point contains two coordinates, horizontal and vertical (here it describes Cartesian coordinates, and of course it can also be polar coordinates, with the same principle), the final dimension of vector X is 4N. The arrangement order of the 2N points is determined by the arrangement order of the points inside vector X during the training of the machine learning model to be used later. It can be from left to right first, or from right to left first, or take one point on the left and one point on the right for arrangement. Within one point, the vertical coordinate can be in front or the horizontal coordinate can be in front. After determining one arrangement order, the arrangement order cannot be changed during the subsequent model training and inference stages. For example, the following is an arrangement method of the elements inside a certain vector X:

[0053] X = ( ),

[0054] where x represents the abscissa of a certain point, y represents the ordinate, and left and right in the upper right corner of the element represent the points from the left rectangular frame and the right rectangular frame respectively. Of course, not only position information may be used in the process of constructing the vector, and signal strength can also be added. However, some millimeter-wave radars may not output the signal strength of the point cloud. Here, only the construction of the vector using the point cloud position information is described to ensure the generality of the method.

[0055] S4. Classification: Input the vector X into the machine learning inference model constructed based on SVM (Support Vector Machine) or DNN (Deep Neural Network). After performing inference operations in the ECU, MCU or domain control device, obtain the classification result, which is divided into four categories in total, such as Figures 2 to 5 shown as entering the tunnel entrance, exiting the tunnel entrance, inside the tunnel, and outside the tunnel.

[0056] Specifically, for training the machine learning model, construct a multi-class (4-class) machine learning model f(x) based on SVM or DNN (Deep Neural Network). Train the model f(x) with the collected vector data and classification labels. (The input variable of f(x) is the vector similar to the previously mentioned sorted vector X), and perform repeated parameter tuning and retraining.

[0057] S5. Deploy the trained model to the ECU, MCU or domain controller on the vehicle side, and perform real-time inference based on the real-time collected data. That is, on the vehicle side, organize the point cloud data collected by the millimeter-wave radar in real time according to the previously sorted data rules (including the interception range and arrangement rules) to obtain a vector X with a dimension of 4N. After inputting X into the machine learning model f(x), obtain the final tunnel scene type value.

[0058] The above steps only describe the process steps after training the machine learning model. The data annotation problem is also worthy of attention, especially the definition of the annotation type. How to define the transition scenes of entering the tunnel entrance and exiting the tunnel entrance between outside the tunnel and inside the tunnel. In this application, the defined threshold is set as an adjustable parameter and is debugged during the actual training and testing process.

[0059] When entering the tunnel entrance, such as Figure 1The dotted box for intercepting point clouds shown. Assume that the ratio of the length of the square occupied by the part inside the tunnel to the length of the entire square is R. When R is less than 80% (the threshold is denoted as β) and greater than 20% (the threshold is denoted as α), we determine that the category of the point cloud distribution is the entrance of the tunnel. When R is not less than 80% (i.e., β), we determine it as inside the tunnel. When R is not greater than 20% (i.e., α), we determine it as outside the tunnel. Similarly, for the exit of the tunnel, we process the scenario in a similar way. Note that the 80% and 20% here (i.e., β and α) are adjustable thresholds, which need to be adjusted in combination with the actual training and test situations and are not fixed parameters.

[0060] On the other hand, if the tunnel detection method is only used on highways, relatively simple models can be enabled, such as SVM and simple DNN. On non-highways, the characteristics of the tunnel point cloud distribution will be different, and the ability of the model needs to be enhanced to improve the detection accuracy, such as recording multi-frame information for classification or recording and extracting historical information in the model.

[0061] The above is the preferred embodiment of the present invention. The basic principle and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification only illustrates the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of the present invention claimed is defined by the appended claims and their equivalents.

Claims

1. A road tunnel scene detection method based on millimeter-wave radar point cloud, characterized in that It includes the following steps: S1. Use the vehicle's millimeter-wave radar to collect point cloud data on both the left and right sides of the road; S2. For the point cloud data collected in step S1, take N point cloud data within the defined spatial range on both the left and right sides; S3. Combine the 2N point cloud data collected on the left and right in step S2 into a 4N-dimensional vector X; S4. Input the vector X into a machine learning inference model constructed based on SVM or DNN for training; S5. Deploy the trained model to the vehicle's ECU, MCU, or domain controller, and perform real-time inference based on the real-time collected point cloud data to obtain the final tunnel scene type value.

2. The method for detecting road tunnel scenarios based on millimeter-wave radar point cloud according to claim 1, characterized in that, In step S1, using the vehicle's millimeter-wave radar to collect point cloud data on both the left and right sides of the road includes collecting the point cloud data of the millimeter-wave radar for tunnel scenes, non-tunnel scenes, tunnel entrance, and tunnel exit on the road, extracting some of the data, and cleaning it, removing the point cloud data behind the vehicle body and the dynamic point cloud data, retaining the point cloud data on both sides of the road, and performing type annotation.

3. The method for detecting road tunnel scenes based on millimeter-wave radar point cloud according to claim 2, wherein The types include tunnel entrance, tunnel exit, inside the tunnel, and outside the tunnel.

4. The method for detecting road tunnel scenes based on millimeter-wave radar point cloud according to claim 1, characterized in that In step S2, taking N point cloud data within the defined spatial range on both the left and right sides is the front view range of the vehicle, and intercepting the point cloud data within the rectangles on both sides at a set distance.

5. The method for detecting road tunnel scenes based on millimeter-wave radar point cloud according to claim 1, wherein, In step S3, the arrangement of the 2N point cloud data is determined by the arrangement of the internal points of the vector X during the training of the machine learning model, and after the arrangement order is determined, the arrangement order cannot be changed during the subsequent model training and inference stages.

6. The method for detecting road tunnel scenarios based on millimeter-wave radar point cloud according to claim 1, characterized in that In step S5, after the inference operation, the classification results are obtained, including tunnel entrance, tunnel exit, inside the tunnel, and outside the tunnel.

7. The method for detecting road tunnel scenarios based on millimeter-wave radar point clouds according to claim 6, wherein The method for determining the tunnel scene type value is that when the obtained tunnel scene type value is within the set threshold range, it is determined that the point cloud data distribution category is the tunnel entrance; when it is greater than or equal to the maximum threshold, it is determined that the point cloud data distribution category is inside the tunnel; when it is less than the minimum threshold, it is determined that the point cloud data distribution category is outside the tunnel.

8. The method for detecting road tunnel scenarios based on millimeter-wave radar point clouds according to claim 1, wherein, It also includes improving the detection accuracy by enhancing the model's ability when not on the highway.

Images