A tunnel scene recognition method and device based on 4D millimeter wave radar point cloud

Abstract

The present invention discloses a tunnel scene recognition method and device based on 4D millimeter wave radar point cloud, which relates to the technical field of vehicle driving scene recognition. The method includes obtaining the stationary point trace information in the near area and the far area set in front of the vehicle in real time based on the 4D millimeter wave radar; calculating the z-direction clustering length and variance of the stationary point traces in the far area, and the probability density of the y-direction mean of the stationary point traces in the near area and the number of point traces located outside the cluster after clustering; judging whether the current scene state of the vehicle is in the tunnel: if so, based on the z-direction clustering length and variance, whether to drive out of the tunnel in the future is determined to update the scene state; if not, based on the z-direction clustering length and variance, and the probability density of the y-direction mean and the number of point traces located outside the cluster after clustering, whether to drive into the tunnel in the future is determined to update the scene state. The present invention can effectively improve the accuracy of millimeter wave radar for tunnel scene recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle driving scene recognition, and in particular to a tunnel scene recognition method and device based on 4D millimeter wave radar point cloud. Background Art

[0002] Tunnels are engineering structures buried in the ground and are a form of human use of underground space. Tunnels can be divided into traffic tunnels, hydraulic tunnels, municipal tunnels, mining tunnels, etc. Traffic tunnels are particularly common for mountain roads. With the advancement of technology, vehicles often identify the surrounding driving scenes during driving, thereby assisting the driver in driving the vehicle. Currently, the detection and identification of driving scenes around the vehicle is generally performed through millimeter-wave radar.

[0003] However, the current millimeter-wave radar scene recognition mainly relies on two-dimensional (longitudinal distance, lateral distance) point cloud data for judgment, lacking height information. This recognition method mainly relies on longitudinal (vehicle driving direction) clustering results to distinguish scenes, which has certain limitations, resulting in low scene recognition accuracy in actual applications. Summary of the invention

[0004] The present application provides a tunnel scene recognition method and device based on 4D millimeter-wave radar point cloud, which can effectively improve the accuracy of millimeter-wave radar in tunnel scene recognition.

[0005] In a first aspect, an embodiment of the present application provides a tunnel scene recognition method based on 4D millimeter wave radar point cloud, which specifically includes the following steps:

[0006] Based on 4D millimeter wave radar, it can obtain the static point information in the near and far areas in front of the vehicle in real time;

[0007] Calculate the z-direction cluster length and variance of the stationary points in the distant area, as well as the probability density of the y-direction mean of the stationary points in the near area and the number of points outside the cluster after clustering;

[0008] Determine whether the vehicle is currently in a tunnel:

[0009] If yes, then based on the z-direction cluster length and variance, a decision is made on whether to exit the tunnel in the future to update the scene state;

[0010] If not, based on the z-direction cluster length and variance, the probability density of the y-direction mean, and the number of points outside the cluster after clustering, a determination is made as to whether to enter a tunnel in the future to update the scene state.

[0011] In combination with the first aspect, in one implementation, before acquiring the stationary point track information in the near area and the far area set in front of the vehicle in real time based on the 4D millimeter wave radar, it also includes:

[0012] A three-dimensional coordinate system is established with the vehicle as the coordinate origin, wherein the vehicle's traveling direction is the x-axis direction, the vehicle's lateral right direction is the y-axis direction, and the vehicle's vertical upward direction is the z-axis direction.

[0013] In combination with the first aspect, in one implementation, the real-time acquisition of stationary point track information in a near area and a far area set in front of the vehicle comprises the following specific steps:

[0014] Based on the detection range of the 4D millimeter-wave radar, determine the near and far areas set in front of the vehicle;

[0015] According to the detection results of the 4D millimeter-wave radar, the number of stationary points in the distant area and the coordinate values ​​of the stationary points in the z-axis direction are counted, and the number of stationary points whose coordinate values ​​in the z-axis direction are greater than the set height value is screened;

[0016] Count the number of stationary points in the nearby area and their coordinate values ​​in the y-axis direction.

[0017] In combination with the first aspect, in one implementation, the z-direction cluster length and variance of the stationary point traces in the distant area, and the probability density of the y-direction mean of the stationary point traces in the near area and the number of points outside the cluster after clustering are calculated, wherein the calculation of the z-direction cluster length and variance of the stationary point traces in the distant area specifically includes the following steps:

[0018] The stationary points whose coordinate values ​​in the z-axis direction are greater than the set height value and are screened in the distant area are clustered;

[0019] Traverse the coordinate values ​​of all stationary points participating in the clustering in the distant area in the z-axis direction, and take the difference between the maximum coordinate value and the minimum coordinate value as the cluster length;

[0020] The mean and variance of the z-axis coordinate values ​​of the stationary point traces participating in clustering in the distant area are calculated.

[0021] In combination with the first aspect, in one implementation, the z-direction cluster length and variance of the stationary points in the distant area, and the probability density of the y-direction mean of the stationary points in the near area and the number of points outside the cluster after clustering are calculated, wherein the probability density of the y-direction mean of the stationary points in the near area and the number of points outside the cluster after clustering are calculated specifically include:

[0022] Traverse the coordinate values ​​of all stationary points in the nearby area in the y-axis direction, and take the difference between the maximum coordinate value and the minimum coordinate value as the span length;

[0023] According to the positive and negative coordinate values ​​of the stationary point traces in the near area in the y-axis direction, the stationary point traces in the near area are divided into two parts, wherein the stationary point traces with negative coordinate values ​​belong to the first part, and the stationary point traces with positive coordinate values ​​belong to the second part;

[0024] The probability density of the coordinate values ​​of the first part of the stationary point trace in the y-axis direction and the probability density of the coordinate values ​​of the second part of the stationary point trace in the y-axis direction are calculated based on the truncated probability density function;

[0025] Based on the size of the coordinate value in the y-axis direction, the first part of the stationary traces are sorted and clustered to obtain the number of stationary traces located on the left side outside the cluster;

[0026] Based on the size of the coordinate value in the y-axis direction, the second part of the stationary traces are sorted and clustered to obtain the number of stationary traces located on the right side outside the cluster.

[0027] In combination with the first aspect, in one implementation, the determination of whether to enter a tunnel in the future to update the scene state is performed based on the z-direction cluster length and variance, the probability density of the y-direction mean, and the number of points located outside the cluster after clustering, and the specific steps include:

[0028] The near-distance scene judgment condition and the far-distance scene judgment condition for entering the tunnel are set, and after each acquisition operation of the stationary point track information in the near area and the far area set in front of the vehicle, a determination is made as to whether the judgment conditions are met;

[0029] Based on the judgment result, if both the short-range scene judgment condition and the long-range scene judgment condition for entering the tunnel are met, the tunnel entry score corresponding to the current acquisition operation is recorded as 1, otherwise it is recorded as 0;

[0030] Determine whether the accumulated tunnel entry score is greater than a first set score after performing the first preset number of acquisition operations:

[0031] If yes, the scene state of the vehicle is updated to be in the tunnel;

[0032] If not, determine whether the current cumulative number of determinations of the determination condition is greater than the first preset number:

[0033] - If the number is greater than the first preset number, the scene state of the vehicle will remain unchanged, and it will still be outside the tunnel;

[0034] -If it is not greater than the first preset number, the scene state of the vehicle is updated to unknown.

[0035] In combination with the first aspect, in one implementation,

[0036] When the span length is greater than the set span length threshold, the probability density of the coordinate values ​​of the first part of the stationary point traces in the y-axis direction is less than the set probability density threshold, the probability density of the coordinate values ​​of the second part of the stationary point traces in the y-axis direction is less than the set probability density threshold, the number of stationary point traces located on the left side outside the cluster is greater than the first set number threshold, and the number of stationary point traces located on the right side outside the cluster is greater than the first set number threshold, it is determined that the close-range scene judgment condition for entering the tunnel is met, otherwise, it is determined that the close-range scene judgment condition for entering the tunnel is not met;

[0037] When the number of stationary points in the distant area is greater than the second set number threshold, the number of stationary points whose coordinate values ​​in the z-axis direction are greater than the set height value obtained by screening is greater than the third set number threshold, the cluster length is greater than the set cluster length threshold, the mean of the z-axis coordinate values ​​of the stationary points participating in the clustering in the distant area is greater than the set mean threshold, and the variance of the z-axis coordinate values ​​of the stationary points participating in the clustering in the distant area is greater than the set variance threshold, then it is determined that the long-distance scene judgment condition for entering the tunnel is met; otherwise, it is determined that the long-distance scene judgment condition for entering the tunnel is not met.

[0038] In combination with the first aspect, in one implementation, the step of determining whether to exit a tunnel in the future to update a scene state based on the z-direction cluster length and variance includes:

[0039] The long-distance scene judgment condition for exiting the tunnel is set, and after each acquisition operation of the stationary point track information in the near area and the far area set in front of the vehicle, a determination is made as to whether the judgment condition is satisfied;

[0040] Based on the judgment result, if the long-distance scene judgment condition of exiting the tunnel is met, the exiting tunnel score corresponding to the current acquisition operation is recorded as 1, otherwise it is recorded as 0;

[0041] Determine whether the accumulated tunnel entry score is greater than a second set score after performing the acquisition operation for a second preset number of times:

[0042] If yes, the scene state of the vehicle is updated to be outside the tunnel;

[0043] If not, the scene state of the vehicle remains unchanged, that is, still in the tunnel.

[0044] In combination with the first aspect, in one implementation,

[0045] When the number of stationary points in the distant area is less than the second set number threshold, the number of stationary points whose coordinate values ​​in the z-axis direction are greater than the set height value obtained by screening is less than the third set number threshold, the cluster length is less than the set cluster length threshold, the mean of the z-axis coordinate values ​​of the stationary points participating in the clustering in the distant area is less than the set mean threshold, and the variance of the z-axis coordinate values ​​of the stationary points participating in the clustering in the distant area is less than the set variance threshold, then it is determined that the long-distance scene judgment condition for exiting the tunnel is met; otherwise, it is determined that the long-distance scene judgment condition for exiting the tunnel is not met.

[0046] In a second aspect, an embodiment of the present application provides a tunnel scene recognition device based on 4D millimeter wave radar point cloud, comprising:

[0047] An acquisition module is used to acquire stationary point information in a set near area and far area in front of the vehicle in real time based on a 4D millimeter wave radar;

[0048] A calculation module, which is used to calculate the z-direction cluster length and variance of the stationary points in the distant area, as well as the probability density of the y-direction mean of the stationary points in the near area and the number of points outside the cluster after clustering;

[0049] A judgment module, which is used to judge whether the current scene state of the vehicle is in a tunnel, and if so, drive the first execution module to work, and if not, drive the second execution module to work;

[0050] A first execution module, which is used to determine whether to exit the tunnel in the future based on the z-direction cluster length and variance to update the scene state;

[0051] The second execution module is used to determine whether to enter a tunnel in the future to update the scene state based on the z-direction cluster length and variance, the probability density of the y-direction mean and the number of points outside the cluster after clustering.

[0052] The beneficial effects brought by the technical solution provided in the embodiments of the present application include:

[0053] Based on the 4D millimeter-wave radar, the information of stationary points in the near and far areas in front of the vehicle is obtained in real time, and corresponding calculations are performed. When the vehicle is currently in a tunnel, the z-direction clustering length and variance of the stationary points in the far area are used to determine whether the vehicle will exit the tunnel in the future to update the scene state. When the vehicle is currently outside the tunnel, the z-direction clustering length and variance of the stationary points in the far area, as well as the probability density of the y-direction mean of the stationary points in the near area and the number of points outside the cluster after clustering, are used to determine whether the vehicle will enter the tunnel in the future to update the scene state. The height information of the 4D radar points is used to consider the longitudinal results while adding the vertical distance and the high-altitude longitudinal clustering results to effectively improve the accuracy of the millimeter-wave radar in tunnel scene recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is a flow chart of a tunnel scene recognition method based on 4D millimeter wave radar point cloud in an embodiment of the present invention;

[0055] Figure 2 A schematic diagram of the division of the near area and the far area in front of the vehicle;

[0056] Figure 3 The following is a flow chart for judging exiting a tunnel;

[0057] Figure 4 This is a flow chart for judging entering a tunnel;

[0058] Figure 5 This is a schematic diagram of the hardware structure of the tunnel scene recognition device based on 4D millimeter wave radar point cloud;

[0059] Figure 6 This is a schematic diagram of the structure of a tunnel scene recognition device based on 4D millimeter-wave radar point cloud. DETAILED DESCRIPTION

[0060] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0061] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0062] First, see Figure 1 As shown, a tunnel scene recognition method based on 4D millimeter wave radar point cloud provided by an embodiment of the present invention specifically includes the following steps:

[0063] S1: Based on 4D millimeter wave radar, the stationary point track information in the near and far areas in front of the vehicle is obtained in real time. The 4D millimeter wave radar is installed on the vehicle. When the vehicle is driving, the 4D millimeter wave radar installed on the vehicle can obtain the stationary point track information in the near and far areas in front of the vehicle in real time.

[0064] For details, see Figure 2 As shown, the rectangle within the first set distance in front of the vehicle can be regarded as the near area, that is, Figure 2As shown in area A in FIG; the rectangle within the second set distance in front of the vehicle and outside the first set distance is regarded as the near area, such as Figure 2 As shown in area B in FIG. 1 , the near area and the far area in front of the vehicle are divided.

[0065] In the present invention, before obtaining the stationary point track information in the near area and the far area set in front of the vehicle in real time based on the 4D millimeter wave radar, it also includes: establishing a three-dimensional coordinate system with the vehicle as the coordinate origin, wherein the vehicle's driving direction is the x-axis direction, the vehicle's lateral right direction is the y-axis direction, and the vehicle's vertical upward direction is the z-axis direction.

[0066] For details, see Figure 2 As shown, Figure 2 The vehicle in the figure is used as the coordinate origin of the three-dimensional coordinate system, the vehicle's driving direction is used as the positive direction of the x-axis, the vehicle's lateral right direction is used as the positive direction of the y-axis, and the vehicle's vertical upward direction is used as the positive direction of the z-axis to establish a three-dimensional coordinate system, which is convenient for subsequent calculations.

[0067] In the present invention, the stationary point trace information in the near area and the far area set in front of the vehicle is obtained in real time, and the specific steps include:

[0068] S101: Based on the detection range of the 4D millimeter-wave radar, determine the near area and the far area set in front of the vehicle; in actual application, the first set distance and the second set distance can be determined according to the detection range of the radar, and then the area boundaries of the near area and the far area can be determined.

[0069] S102: according to the detection result of the 4D millimeter wave radar, the number of stationary point traces in the distant area and the coordinate values ​​of the stationary point traces in the z-axis direction are obtained by counting, and the number of stationary point traces whose coordinate values ​​in the z-axis direction are greater than the set height value is obtained by screening;

[0070] That is, for the acquisition of stationary point information in the distant area, specifically, the number n1 of stationary point traces in the distant area and the coordinate values ​​of these stationary point traces in the z-axis direction, that is, the z values, are obtained and stored, and then the number n2 of stationary point traces whose z values ​​are greater than the set height value among these stationary point traces is counted.

[0071] S103: Counting the number of stationary points in the nearby area and the coordinate values ​​of the stationary points in the y-axis direction.

[0072] That is, for obtaining the stationary point information in the near area, specifically, obtaining the number n3 of the stationary point information in the near area, and then counting the coordinate values ​​of these stationary point information in the y-axis direction, that is, the y values, and storing them.

[0073] S2: Calculate the z-direction cluster length and variance of the stationary points in the distant area, as well as the probability density of the y-direction mean of the stationary points in the near area and the number of points outside the cluster after clustering;

[0074] In the present invention, the z-direction cluster length and variance of the stationary points in the distant area, as well as the probability density of the y-direction mean of the stationary points in the near area and the number of points outside the cluster after clustering are calculated. The specific steps for calculating the z-direction cluster length and variance of the stationary points in the distant area include:

[0075] S201: clustering the stationary points whose coordinate values ​​in the z-axis direction are greater than the set height value and are screened in the distant area;

[0076] That is, the stationary point traces whose coordinate values ​​in the z-axis direction in the distant area are greater than the set height value are regarded as stationary point traces that meet the requirements for clustering, that is, the stationary point trace n2 is clustered as a point that meets the requirements.

[0077] S202: traverse the coordinate values ​​of all stationary points participating in the clustering in the distant area in the z-axis direction, and take the difference between the maximum coordinate value and the minimum coordinate value as the cluster length;

[0078] That is, the coordinate values ​​of all stationary point traces n2 in the z-axis direction are traversed to obtain the maximum coordinate value and the minimum coordinate value, and the difference between the maximum coordinate value and the minimum coordinate value is used as the cluster length l1.

[0079] S203: Calculate the mean and variance of the coordinate values ​​of the stationary point traces participating in the clustering in the distant area in the z-axis direction.

[0080] That is, first obtain the coordinate values ​​of all stationary point traces n2 in the z-axis direction, and then calculate the mean and variance σ of the obtained coordinate values.

[0081] In the present invention, the z-direction cluster length and variance of the stationary points in the distant area, as well as the probability density of the y-direction mean of the stationary points in the near area and the number of points located outside the cluster after clustering are calculated. The specific steps for calculating the probability density of the y-direction mean of the stationary points in the near area and the number of points located outside the cluster after clustering include:

[0082] S211: traverse the coordinate values ​​of all the stationary points in the near area in the y-axis direction, and take the difference between the maximum coordinate value and the minimum coordinate value as the span length;

[0083] That is, the coordinate values ​​of all the stationary points in the nearby area in the y-axis direction are traversed to obtain the maximum coordinate value and the minimum coordinate value, and then the difference between the maximum coordinate value and the minimum coordinate value is taken as the span length l2.

[0084] S212: dividing the stationary point traces in the near area into two parts according to the positive and negative coordinate values ​​of the stationary point traces in the y-axis direction in the near area, wherein the stationary point traces with negative coordinate values ​​belong to the first part, and the stationary point traces with positive coordinate values ​​belong to the second part;

[0085] That is, according to the positive and negative coordinate values ​​of the stationary point traces in the near area in the y-axis direction, the stationary point traces with negative y-axis coordinate values ​​are divided into the first part of the stationary point traces LeftY, and the stationary point traces with positive y-axis coordinate values ​​are divided into the second part of the stationary point traces RightY.

[0086] S213: Calculating the probability density of the coordinate values ​​of the first part of the stationary point trace in the y-axis direction and the probability density of the coordinate values ​​of the second part of the stationary point trace in the y-axis direction based on the truncated probability density function;

[0087] By truncating the probability density function, we first calculate the y value of the maximum probability density of the first part of the stationary point trace LeftY and the second part of the stationary point trace RightY, and then obtain the probability density p1 of the y-axis coordinate value of the first part of the stationary point trace and the probability density p2 of the y-axis coordinate value of the second part of the stationary point trace.

[0088] S214: sorting and clustering the first part of the stationary traces based on the size of the coordinate values ​​in the y-axis direction, and obtaining the number of stationary traces located on the left side outside the cluster;

[0089] That is, according to the size of the coordinate value in the y-axis direction, the stationary point traces in the first part of the stationary point traces LeftY are sorted and clustered to obtain the number of stationary point traces located on the left side outside the cluster.

[0090] S215: Sort and cluster the second part of stationary traces based on the size of the coordinate value in the y-axis direction to obtain the number of stationary traces located on the right side outside the cluster.

[0091] That is, according to the size of the coordinate value in the y-axis direction, the stationary point traces in the second part of the stationary point traces RightY are sorted and clustered to obtain the number of stationary point traces located on the right side outside the cluster.

[0092] S3: Determine whether the current scene state of the vehicle is in a tunnel. If so, go to S4; if not, go to S5;

[0093] That is, first determine whether the vehicle's current scene state is in a tunnel. If so, then determine whether the vehicle has exited the tunnel, and then update the vehicle's scene state; if not, then determine whether the vehicle has entered the tunnel, and then update the vehicle's scene state.

[0094] S4: Based on the z-direction cluster length and variance, determining whether to exit the tunnel in the future to update the scene state;

[0095] In the present invention, based on the z-direction cluster length and variance, a decision is made on whether to exit the tunnel in the future to update the scene state, and the specific steps include:

[0096] S401: setting a long-distance scene judgment condition for exiting a tunnel, and determining whether the judgment condition is satisfied after each acquisition operation of stationary point trace information in a near area and a far area set in front of the vehicle;

[0097] It should be noted that the stationary point track information in the near area and the far area in front of the vehicle is set in batches and continuously. Every time the stationary point track information in front of the vehicle is obtained, a long-distance scene judgment condition determination operation for exiting the tunnel will be performed.

[0098] S402: Based on the determination result, if the long-distance scene determination condition for exiting the tunnel is satisfied, the exiting tunnel score corresponding to the current acquisition operation is recorded as 1, otherwise it is recorded as 0;

[0099] Specifically, in each operation of determining the long-distance scene judgment condition for exiting the tunnel, if the long-distance scene judgment condition for exiting the tunnel is met in the current determination, the current score for exiting the tunnel is recorded as 1; if the long-distance scene judgment condition for exiting the tunnel is not met in the current determination, the current score for exiting the tunnel is recorded as 0.

[0100] S403: Determine whether the accumulated tunnel exit score is greater than a second set score after performing the acquisition operation for a second preset number of times:

[0101] If yes, the scene state of the vehicle is updated to be outside the tunnel;

[0102] If not, the scene state of the vehicle remains unchanged, that is, still in the tunnel.

[0103] That is, after the second preset number of determination operations are performed continuously, it is determined whether the accumulated tunnel exit score is greater than the second set score. If so, the vehicle scene state is updated to be outside the tunnel; if not, the vehicle scene state remains unchanged and is still inside the tunnel.

[0104] In the present invention, when the number of stationary points in the distant area is less than the second set number threshold, the number of stationary points whose coordinate values ​​in the z-axis direction are greater than the set height value obtained by screening is less than the third set number threshold, the cluster length is less than the set cluster length threshold, the mean of the z-axis coordinate values ​​of the stationary points participating in the clustering in the distant area is less than the set mean threshold, and the variance of the z-axis coordinate values ​​of the stationary points participating in the clustering in the distant area is less than the set variance threshold, then it is determined that the long-distance scene judgment condition for exiting the tunnel is met, otherwise, it is determined that the long-distance scene judgment condition for exiting the tunnel is not met.

[0105] Specifically, the long-distance scene judgment conditions for exiting the tunnel include: 1. The number n1 of stationary points in the distant area is less than the second set number threshold; 2. The number n2 of stationary points whose z value is greater than the set height value is less than the third set number threshold; 3. The cluster length l1 is less than the set cluster length threshold; 4. The mean of the coordinate values ​​of all stationary points n2 in the z-axis direction is less than the set mean threshold; 5. The variance σ of the coordinate values ​​of all stationary points n2 in the z-axis direction is less than the set variance threshold. When the above conditions 1 to 5 are all met, it is determined that the long-distance scene judgment conditions for exiting the tunnel are met, otherwise, it is determined that the long-distance scene judgment conditions for exiting the tunnel are not met.

[0106] The following combination Figure 3 , the process of judging whether to exit a tunnel is described in detail.

[0107] S41: Whether the long-distance scene condition of exiting the tunnel is satisfied, if yes, go to S42, if not, go to S43;

[0108] S42: Record the score of this tunnel exit as 1, and go to S44;

[0109] S43: Record the tunnel exit score as 0, and go to S44;

[0110] S44: after the second preset number of determination operations are performed continuously, whether the accumulated tunnel exit score is greater than the second set score, if so, go to S45, if not, go to S46;

[0111] S45: The scene state is updated to being outside the tunnel, and the process ends;

[0112] S46: The scene state remains unchanged, still in the tunnel, and ends.

[0113] S5: Based on the cluster length and variance in the z direction, the probability density of the mean in the y direction and the number of points outside the cluster after clustering, a determination is made as to whether to enter a tunnel in the future to update the scene state.

[0114] In the present invention, based on the z-direction cluster length and variance, the probability density of the y-direction mean, and the number of points outside the cluster after clustering, a determination is made as to whether to enter a tunnel in the future to update the scene state. The specific steps include:

[0115] S501: setting a near-distance scene judgment condition and a far-distance scene judgment condition for entering a tunnel, and determining whether the judgment condition is satisfied after each acquisition operation of stationary point trace information in the near area and the far area set in front of the vehicle;

[0116] It should be noted that the stationary point track information in the near area and the far area in front of the vehicle is set in batches and continuously. Every time the stationary point track information in front of the vehicle is obtained, a near-range scene judgment condition and a far-range scene judgment condition judgment operation for entering the tunnel will be performed.

[0117] S502: Based on the determination result, if both the short-distance scene determination condition and the long-distance scene determination condition for entering the tunnel are satisfied, the tunnel entry score corresponding to the current acquisition operation is recorded as 1, otherwise it is recorded as 0;

[0118] Specifically, in each judgment operation of the close-range scene judgment conditions and the long-range scene judgment conditions for entering the tunnel, if both the close-range scene judgment conditions and the long-range scene judgment conditions for entering the tunnel are met in the current judgment, then the current score for entering the tunnel is recorded as 1; otherwise, the current score for entering the tunnel is recorded as 0.

[0119] S503: Determine whether the accumulated tunnel entry score is greater than a first set score after performing the first preset number of acquisition operations:

[0120] If yes, the scene state of the vehicle is updated to be in the tunnel;

[0121] If not, determine whether the current cumulative number of determinations of the determination condition is greater than the first preset number:

[0122] - If the number is greater than the first preset number, the scene state of the vehicle will remain unchanged, and it will still be outside the tunnel;

[0123] -If it is not greater than the first preset number, the scene state of the vehicle is updated to unknown.

[0124] That is, after performing the first preset number of determination operations in succession, it is determined whether the accumulated score for entering the tunnel is greater than the first set score. If so, the scene state of the vehicle is updated to be in the tunnel; if not, it is determined whether the accumulated determination operations are greater than the first preset number of times. If greater than the first preset number of times, the scene state of the vehicle will be kept unchanged and still be outside the tunnel; if not greater than the first preset number of times, the scene state of the vehicle will be updated to unknown.

[0125] In the present invention, when the span length is greater than the set span length threshold, the probability density of the y-axis coordinate values ​​of the first part of the stationary point traces is less than the set probability density threshold, the probability density of the y-axis coordinate values ​​of the second part of the stationary point traces is less than the set probability density threshold, the number of stationary point traces located on the left side outside the cluster is greater than the first set number threshold, and the number of stationary point traces located on the right side outside the cluster is greater than the first set number threshold, then it is determined that the close-range scene judgment condition for entering the tunnel is met; otherwise, it is determined that the close-range scene judgment condition for entering the tunnel is not met.

[0126] That is, the close-range scene judgment conditions for entering the tunnel include: 1. The span length l2 is greater than the set span length threshold; 2. The probability density p1 of the coordinate values ​​of the first part of the stationary point traces in the y-axis direction is less than the set probability density threshold; 3. The probability density p2 of the coordinate values ​​of the second part of the stationary point traces in the y-axis direction is less than the set probability density threshold; 4. The number of stationary point traces located on the left side outside the cluster is greater than the first set number threshold; 5. The number of stationary point traces located on the right side outside the cluster is greater than the first set number threshold. When the above conditions 1 to 5 are all met, it is determined that the close-range scene judgment conditions for entering the tunnel are met, otherwise, it is determined that the close-range scene judgment conditions for entering the tunnel are not met.

[0127] In the present invention, when the number of stationary points in the distant area is greater than the second set number threshold, the number of stationary points whose coordinate values ​​in the z-axis direction are greater than the set height value obtained by screening is greater than the third set number threshold, the cluster length is greater than the set cluster length threshold, the mean of the z-axis coordinate values ​​of the stationary points participating in the clustering in the distant area is greater than the set mean threshold, and the variance of the z-axis coordinate values ​​of the stationary points participating in the clustering in the distant area is greater than the set variance threshold, then it is determined that the long-distance scene judgment condition for entering the tunnel is met, otherwise, it is determined that the long-distance scene judgment condition for entering the tunnel is not met.

[0128] That is, the long-distance scene judgment conditions for entering the tunnel include: 1. The number of stationary point traces n1 in the distant area is greater than the second set number threshold; 2. The number of stationary point traces n2 with z values ​​greater than the set height value in the stationary point traces is greater than the third set number threshold; 3. The cluster length l1 is greater than the set cluster length threshold; 4. The mean value of the coordinate values ​​of all stationary point traces n2 in the z-axis direction is greater than the set mean threshold; 5. The variance σ of the coordinate values ​​of all stationary point traces n2 in the z-axis direction is greater than the set variance threshold. When the above conditions 1 to 5 are all met, it is determined that the long-distance scene judgment conditions for entering the tunnel are met, otherwise, it is determined that the long-distance scene judgment conditions for entering the tunnel are not met.

[0129] The following combination Figure 4 , the judgment process of entering a tunnel is explained.

[0130] S51: Determine whether the close-range scene determination condition for entering the tunnel is met, if so, go to S52, if not, go to S53;

[0131] S52: Determine whether the long-distance scene determination condition for entering the tunnel is met, if so, go to S54, if not, go to S53;

[0132] S53: Record the tunnel entry score as 0;

[0133] S54: Record the tunnel entry score as 1, and go to S55;

[0134] S55: After the first preset number of determination operations are performed continuously, whether the accumulated tunnel entry score is greater than the first set score, if so, go to S56, if not, go to S57;

[0135] S56: The scene state is updated to being in the tunnel, and the process ends;

[0136] S57: Determine whether the cumulative number of determination operations is greater than a first preset number, if yes, go to S58, if no, go to S59;

[0137] S58: Keep the scene state of the vehicle unchanged, that is, still outside the tunnel, and end;

[0138] S59: The scene status is updated to unknown, and the process ends.

[0139] The tunnel scene recognition method based on 4D millimeter-wave radar point cloud of the embodiment of the present invention obtains the stationary point track information in the near area and the far area set in front of the vehicle in real time based on the 4D millimeter-wave radar, and performs corresponding calculations. When the vehicle is currently in the tunnel, based on the z-direction clustering length and variance of the stationary point tracks in the far area, it is determined whether to exit the tunnel in the future to update the scene state. When the vehicle is currently outside the tunnel, based on the z-direction clustering length and variance of the stationary point tracks in the far area, and the probability density of the y-direction mean of the stationary point tracks in the near area and the number of points located outside the cluster after clustering, it is determined whether to enter the tunnel in the future to update the scene state. The height information of the 4D radar point tracks is used, and while considering the longitudinal results, the vertical distance and the high-altitude longitudinal clustering results are added to effectively improve the accuracy of the millimeter-wave radar for tunnel scene recognition.

[0140] On the second aspect, an embodiment of the present application provides a tunnel scene recognition device based on 4D millimeter-wave radar point cloud. The tunnel scene recognition device based on 4D millimeter-wave radar point cloud can be a personal computer (PC), a laptop computer, a server, or other device with data processing capabilities.

[0141] Reference Figure 5 , Figure 5 The hardware structure diagram of the tunnel scene recognition device based on 4D millimeter wave radar point cloud involved in the embodiment of the present application is shown in FIG. In the embodiment of the present application, the tunnel scene recognition device based on 4D millimeter wave radar point cloud may include a processor, a memory, a communication interface and a communication bus.

[0142] The communication bus may be of any type and is used to interconnect the processor, the memory, and the communication interface.

[0143] The communication interface includes input / output (I / O) interface, physical interface and logical interface, etc., which are used to realize the interconnection of devices inside the tunnel scene recognition device based on 4D millimeter wave radar point cloud, and the interface used to realize the interconnection between the tunnel scene recognition device based on 4D millimeter wave radar point cloud and other devices (such as other computing devices or user devices). The physical interface can be an Ethernet interface, a fiber optic interface, an ATM interface, etc.; the user device can be a display (Display), a keyboard (Keyboard), etc.

[0144] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0145] The processor may be a general-purpose processor, which may call a tunnel scene recognition program based on 4D millimeter-wave radar point cloud stored in a memory, and execute the tunnel scene recognition method based on 4D millimeter-wave radar point cloud provided in an embodiment of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the tunnel scene recognition program based on 4D millimeter-wave radar point cloud is called may refer to the various embodiments of the tunnel scene recognition method based on 4D millimeter-wave radar point cloud of the present application, which will not be repeated here.

[0146] Those skilled in the art will understand that Figure 5 The hardware structure shown in the figure does not constitute a limitation on the present application, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0147] Thirdly, see Figure 6 As shown, an embodiment of the present invention further provides a tunnel scene recognition device based on 4D millimeter wave radar point cloud, including an acquisition module, a calculation module, a judgment module, a first execution module and a second execution module.

[0148] The acquisition module is used to acquire the stationary point information in the near and far areas set in front of the vehicle in real time based on the 4D millimeter wave radar; the calculation module is used to calculate the z-direction clustering length and variance of the stationary point traces in the far area, as well as the probability density of the y-direction mean of the stationary point traces in the near area and the number of point traces located outside the cluster after clustering; the judgment module is used to judge whether the current scene state of the vehicle is in a tunnel, if so, drive the first execution module to work, if not, drive the second execution module to work; the first execution module is used to determine whether to exit the tunnel in the future based on the z-direction clustering length and variance to update the scene state; the second execution module is used to determine whether to enter the tunnel in the future based on the z-direction clustering length and variance, the probability density of the y-direction mean and the number of point traces located outside the cluster after clustering to update the scene state.

[0149] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0150] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit "first", "second" and "third" to different types.

[0151] In the description of the embodiments of the present application, "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a specific way.

[0152] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; the “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0153] In some processes described in the embodiments of the present application, multiple operations or steps that appear in a specific order are included, but it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or in parallel, and the sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0154] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.

[0155] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A tunnel scene recognition method based on 4D millimeter wave radar point cloud, characterized in that: The specific steps include: Based on 4D millimeter wave radar, it can obtain the static point information in the near and far areas in front of the vehicle in real time; Calculate the z-direction cluster length and variance of the stationary points in the distant area, as well as the probability density of the y-direction mean of the stationary points in the near area and the number of points outside the cluster after clustering; Determine whether the vehicle is currently in a tunnel: If yes, then based on the z-direction cluster length and variance, a decision is made on whether to exit the tunnel in the future to update the scene state; If not, based on the z-direction cluster length and variance, the probability density of the y-direction mean, and the number of points outside the cluster after clustering, a decision is made on whether to enter a tunnel in the future to update the scene state; A three-dimensional coordinate system is established with the vehicle as the coordinate origin, wherein the vehicle's travel direction is the x-axis direction, the vehicle's lateral rightward direction is the y-axis direction, and the vehicle's vertical upward direction is the z-axis direction; Among them, for the calculation of the z-direction cluster length and variance of the stationary point traces in the distant area, the specific steps include: The stationary points whose coordinate values ​​in the z-axis direction are greater than the set height value and are screened in the distant area are clustered; Traverse the coordinate values ​​of all stationary points participating in the clustering in the distant area in the z-axis direction, and take the difference between the maximum coordinate value and the minimum coordinate value as the cluster length; Calculate the mean and variance of the z-axis coordinate values ​​of the stationary point traces participating in clustering in the distant area; Among them, for the probability density of the y-direction mean of the stationary points in the nearby area and the number of points outside the cluster after clustering, the specific steps include: Traverse the coordinate values ​​of all stationary points in the nearby area in the y-axis direction, and take the difference between the maximum coordinate value and the minimum coordinate value as the span length; According to the positive and negative coordinate values ​​of the stationary point traces in the near area in the y-axis direction, the stationary point traces in the near area are divided into two parts, wherein the stationary point traces with negative coordinate values ​​belong to the first part, and the stationary point traces with positive coordinate values ​​belong to the second part; The probability density of the coordinate values ​​of the first part of the stationary point trace in the y-axis direction and the probability density of the coordinate values ​​of the second part of the stationary point trace in the y-axis direction are calculated based on the truncated probability density function; Based on the size of the coordinate value in the y-axis direction, the first part of the stationary traces are sorted and clustered to obtain the number of stationary traces located on the left side outside the cluster; Based on the size of the coordinate value in the y-axis direction, the second part of the stationary traces are sorted and clustered to obtain the number of stationary traces located on the right side outside the cluster.

2. The tunnel scene recognition method based on 4D millimeter wave radar point cloud according to claim 1, characterized in that: The real-time acquisition of the stationary point track information in the near area and the far area set in front of the vehicle includes the following specific steps: Based on the detection range of the 4D millimeter-wave radar, determine the near and far areas set in front of the vehicle; According to the detection results of the 4D millimeter-wave radar, the number of stationary points in the distant area and the coordinate values ​​of the stationary points in the z-axis direction are counted, and the number of stationary points whose coordinate values ​​in the z-axis direction are greater than the set height value is screened; Count the number of stationary points in the nearby area and their coordinate values ​​in the y-axis direction.

3. The tunnel scene recognition method based on 4D millimeter wave radar point cloud according to claim 2, characterized in that: Based on the cluster length and variance in the z direction, the probability density of the mean in the y direction and the number of points outside the cluster after clustering, whether to enter the tunnel in the future is determined to update the scene state, and the specific steps include: The near-distance scene judgment condition and the far-distance scene judgment condition for entering the tunnel are set, and after each acquisition operation of the stationary point track information in the near area and the far area set in front of the vehicle, a determination is made as to whether the judgment conditions are met; Based on the judgment result, if both the short-range scene judgment condition and the long-range scene judgment condition for entering the tunnel are met, the tunnel entry score corresponding to the current acquisition operation is recorded as 1, otherwise it is recorded as 0; Determine whether the accumulated tunnel entry score is greater than a first set score after performing the first preset number of acquisition operations: If yes, the scene state of the vehicle is updated to be in the tunnel; If not, determine whether the current cumulative number of determinations of the determination condition is greater than the first preset number: - If the number is greater than the first preset number, the scene state of the vehicle will remain unchanged, and it will still be outside the tunnel; -If it is not greater than the first preset number, the scene state of the vehicle is updated to unknown.

4. The tunnel scene recognition method based on 4D millimeter wave radar point cloud according to claim 3, characterized in that: When the span length is greater than the set span length threshold, the probability density of the coordinate values ​​of the first part of the stationary point traces in the y-axis direction is less than the set probability density threshold, the probability density of the coordinate values ​​of the second part of the stationary point traces in the y-axis direction is less than the set probability density threshold, the number of stationary point traces located on the left side outside the cluster is greater than the first set number threshold, and the number of stationary point traces located on the right side outside the cluster is greater than the first set number threshold, it is determined that the close-range scene judgment condition for entering the tunnel is met, otherwise, it is determined that the close-range scene judgment condition for entering the tunnel is not met; When the number of stationary points in the distant area is greater than the second set number threshold, the number of stationary points whose coordinate values ​​in the z-axis direction are greater than the set height value obtained by screening is greater than the third set number threshold, the cluster length is greater than the set cluster length threshold, the mean of the z-axis coordinate values ​​of the stationary points participating in the clustering in the distant area is greater than the set mean threshold, and the variance of the z-axis coordinate values ​​of the stationary points participating in the clustering in the distant area is greater than the set variance threshold, then it is determined that the long-distance scene judgment condition for entering the tunnel is met; otherwise, it is determined that the long-distance scene judgment condition for entering the tunnel is not met.

5. The tunnel scene recognition method based on 4D millimeter wave radar point cloud according to claim 2, characterized in that: The step of determining whether to exit the tunnel in the future based on the z-direction cluster length and variance to update the scene state includes: The long-distance scene judgment condition for exiting the tunnel is set, and after each acquisition operation of the stationary point track information in the near area and the far area set in front of the vehicle, a determination is made as to whether the judgment condition is satisfied; Based on the judgment result, if the long-distance scene judgment condition of exiting the tunnel is met, the exiting tunnel score corresponding to the current acquisition operation is recorded as 1, otherwise it is recorded as 0; Determine whether the accumulated tunnel entry score is greater than a second set score after performing the acquisition operation for a second preset number of times: If yes, the scene state of the vehicle is updated to be outside the tunnel; If not, the scene state of the vehicle remains unchanged, that is, still in the tunnel.

6. The tunnel scene recognition method based on 4D millimeter wave radar point cloud according to claim 5, characterized in that: When the number of stationary points in the distant area is less than the second set number threshold, the number of stationary points whose coordinate values ​​in the z-axis direction are greater than the set height value obtained by screening is less than the third set number threshold, the cluster length is less than the set cluster length threshold, the mean of the z-axis coordinate values ​​of the stationary points participating in the clustering in the distant area is less than the set mean threshold, and the variance of the z-axis coordinate values ​​of the stationary points participating in the clustering in the distant area is less than the set variance threshold, then it is determined that the long-distance scene judgment condition for exiting the tunnel is met; otherwise, it is determined that the long-distance scene judgment condition for exiting the tunnel is not met.

7. A tunnel scene recognition device based on 4D millimeter wave radar point cloud, characterized in that: include: An acquisition module is used to acquire stationary point information in a set near area and far area in front of the vehicle in real time based on a 4D millimeter wave radar; A calculation module, which is used to calculate the z-direction cluster length and variance of the stationary points in the distant area, as well as the probability density of the y-direction mean of the stationary points in the near area and the number of points outside the cluster after clustering; A judgment module, which is used to judge whether the current scene state of the vehicle is in a tunnel, and if so, drive the first execution module to work, and if not, drive the second execution module to work; A first execution module, which is used to determine whether to exit the tunnel in the future based on the z-direction cluster length and variance to update the scene state; A second execution module, which is used to determine whether to enter a tunnel in the future to update the scene state based on the z-direction cluster length and variance, the probability density of the y-direction mean, and the number of points located outside the cluster after clustering; A three-dimensional coordinate system is established with the vehicle as the coordinate origin, wherein the vehicle's travel direction is the x-axis direction, the vehicle's lateral rightward direction is the y-axis direction, and the vehicle's vertical upward direction is the z-axis direction; Among them, for the calculation of the z-direction cluster length and variance of the stationary point traces in the distant area, the specific steps include: The stationary points whose coordinate values ​​in the z-axis direction are greater than the set height value and are screened in the distant area are clustered; Traverse the coordinate values ​​of all stationary points participating in the clustering in the distant area in the z-axis direction, and take the difference between the maximum coordinate value and the minimum coordinate value as the cluster length; Calculate the mean and variance of the z-axis coordinate values ​​of the stationary point traces participating in clustering in the distant area; Among them, for the probability density of the y-direction mean of the stationary points in the nearby area and the number of points outside the cluster after clustering, the specific steps include: Traverse the coordinate values ​​of all stationary points in the nearby area in the y-axis direction, and take the difference between the maximum coordinate value and the minimum coordinate value as the span length; According to the positive and negative coordinate values ​​of the stationary point traces in the near area in the y-axis direction, the stationary point traces in the near area are divided into two parts, wherein the stationary point traces with negative coordinate values ​​belong to the first part, and the stationary point traces with positive coordinate values ​​belong to the second part; The probability density of the coordinate values ​​of the first part of the stationary point trace in the y-axis direction and the probability density of the coordinate values ​​of the second part of the stationary point trace in the y-axis direction are calculated based on the truncated probability density function; Based on the size of the coordinate value in the y-axis direction, the first part of the stationary traces are sorted and clustered to obtain the number of stationary traces located on the left side outside the cluster; Based on the size of the coordinate value in the y-axis direction, the second part of the stationary traces are sorted and clustered to obtain the number of stationary traces located on the right side outside the cluster.

Images