Millimeter wave radar high-voltage line intelligent detection and trend prediction method

Abstract

The application provides a millimeter wave radar high-voltage line intelligent detection and trend prediction method, including the following steps: step one, taking the radar echo map in an antenna scanning cycle as the input of a convolutional neural network; step two, setting a first threshold, and retaining the line point detection result greater than the first threshold; step three, using the line point detection result as the input, using a multi-curve fitting method, and obtaining the preliminary detection result of multiple power lines; step four, performing pre-order traversal on the preliminary detection result of each power line, sequentially connecting each node obtained through the traversal, and obtaining the power line detection result in the form of a line segment; step five, performing a smoothing operation on each power line detection result sequence in the form of a line segment; step six, setting a second threshold, removing the power line in the form of a line segment with a confidence degree or length less than the second threshold, and outputting the remaining power line in the form of a line segment as the detection result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of radar collision avoidance, and in particular to a millimeter wave radar high-voltage line intelligent detection and trend prediction method. BACKGROUND

[0002] Helicopter collision with power lines is a global problem. In the statistics of flight accidents in recent years, low-altitude obstacles such as chimneys and high-voltage lines have become the main factors threatening flight safety, and high-voltage lines are often the most difficult to find in poor visibility, and therefore become the most dangerous obstacle in low-altitude flight. Because the wavelength of millimeter wave is close to the size of power line, the power line echo has the most obvious Bragg effect in the millimeter wave band, and is less affected by climate and light, so the millimeter wave collision avoidance radar system has become the main choice for collision avoidance detection of low-altitude aircraft.

[0003] The existing millimeter wave high-voltage line detection algorithm part uses a method of first extracting the positions of suspected power lines and power towers through constant false alarm detection, then extracting power lines using a straight line detection method such as Hough transform, and finally using a feature classifier for classification. Ma Qirong et al. proposed a high-voltage line detection algorithm that jointly uses Hough transform and a support vector machine classifier. The power mean, average peak interval, and variance of the power line are extracted to complete power line detection, but this method uses Hough transform to detect straight lines, and is powerless in the case of changes in the trend of the power line. Chinese invention patent CN106529416A first divides the millimeter wave echo image into blocks, extracts straight lines in each divided region using Hough transform, and extracts line segment features to determine whether each line segment is a power line using a decision tree. But the detection result of this method is a separate straight line segment rather than a complete high-voltage line, and it is also unable to detect changes in the trend of the power line within the block region. Chinese invention patent CN107561509A uses constant false alarm detection to extract suspected power line points in the millimeter wave echo image, and uses Kalman filter algorithm to connect the line points to complete power line position detection, and uses support vector machine algorithm to complete power line recognition and determination. But this method only conducts experiments on simulation data, and the millimeter wave radar echo is calculated according to the theoretical RCS of the power line in the simulation data, which cannot well adapt to the case where there are a large number of clutter areas in the actual scene and the Bragg effect is not obvious. SUMMARY

[0004] Therefore, the embodiments of the present application provide a millimeter wave radar high-voltage line intelligent detection and trend prediction method to improve the obstacle avoidance capability of low-altitude flight of helicopters and protect the safety of flight personnel and passengers.

[0005] The embodiment of the present specification provides the following technical scheme: a millimeter wave radar high-voltage line intelligent detection and trend prediction method, comprising the following steps: step one, taking the radar echo map in one antenna scanning cycle as the input of the convolutional neural network; step two, setting a first threshold, and retaining the line point detection result greater than the first threshold; step three, using the line point detection result as the input, using the multi-curve fitting method, obtaining the preliminary detection result of the plurality of power lines; step four, performing pre-order traversal on the preliminary detection result of each power line, sequentially connecting each node obtained by traversal to obtain the power line detection result in the form of a line segment; step five, performing smoothing operation on each power line detection result in the form of a line segment; step six, setting a second threshold, removing the power line in the form of a line segment with a confidence or length less than the second threshold, and outputting the remaining power line in the form of a line segment as the detection result.

[0006] Further, step one comprises: setting the classification loss function as l c =-2|c p -c g | λ c g logc p , wherein c p , c g are the category prediction confidence and the real category result respectively, and λ is a parameter for balancing difficult samples.

[0007] Further, step one further comprises: setting the angle prediction to adopt an absolute value loss function , wherein a p , a g are the angle prediction result and the real line point tangent angle respectively, d is the distance of the point from the nearest line point in the label, and D is a parameter for adjusting the angle loss.

[0008] Further, step three comprises:

[0009] Step 3.1, a new empty tree is created, and all line point detection results greater than the first threshold are represented in the form of a linked list;

[0010] Step 3.2, the first line point in the linked list is sequentially compared with the size of the nodes in the root node of each red-black tree layer by layer, until the closest node in the red-black tree is found;

[0011] Step 3.3, the distance, angle difference degree, position difference degree and position interval in different directions of the line point detection result greater than the first threshold and the closest node are calculated.

[0012] Step 3.4, according to the calculation result in step 3.3, when the distance, the angle difference degree, the position difference degree and the direction interval all meet the set conditions, the current line point is merged with the node in the tree, and step 3.7 is jumped to;

[0013] Step 3.5, when only the distance does not meet the set condition, the line point is inserted into the tree as a leaf node, the red-black tree is operated by left rotation or right rotation according to the balance of the current tree, and step 3.8 is jumped to.

[0014] Step 3.6, the line point is compared with the next tree, and step 3.2 is jumped to, if all the trees have been compared, the line point is placed at the tail of the linked list first, and then step 3.2 is jumped to.

[0015] Step 3.7, it is verified whether the node in the tree after merging and its adjacent node still meet the conditions in step 3.4, if yes, step 3.8 is jumped to, otherwise step 3.6 is jumped to;

[0016] Step 3.8, the line point is removed from the linked list, and then step 3.2 is jumped to;

[0017] Step 3.9, if the linked list does not complete any merging or inserting operation in a tree in one cycle, a new tree is built with the first line point in the linked list as the root node;

[0018] Step 3.10, the above steps are cycled until the linked list is empty.

[0019] Compared with the prior art, the above at least one technical scheme adopted by the embodiment of the present application can achieve at least the following beneficial effects: the convolutional neural network is used to automatically extract the peak value features in the radar echo map, and the problem that in the traditional CFAR peak value detection, the detection threshold is relied on, and it is impossible to reduce the false alarm while reducing the missed detection is overcome. Further, the present method relies on the prediction of the direction of the line point, and the problem that the existing power line detection method can only detect straight lines is overcome, and accurate detection can still be realized under the condition that the power line direction changes. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creating laborious work.

[0021] Figure 1 is a flowchart of the embodiment of the present application;

[0022] Figure 2Fig. 1 is a schematic diagram of a power line detection algorithm flow in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0024] It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0025] As shown in Figure 1 and Figure 2 , the embodiment of the present application provides a millimeter wave radar high-voltage line intelligent detection and trend prediction method, which specifically includes the following steps:

[0026] Step 1: The radar echo map in one antenna scanning cycle is taken as the input of the CNN network. The CNN network adopts the ENet network model. The feature extraction part of the network is composed of 7 stage modules, and each stage module is composed of a plurality of bottleneck structures. The outputs of the feature extraction network are connected to two branches respectively, and each branch includes two fully connected layers respectively to predict the line point confidence at each pixel point and the tangent angle at the line point. The tangent angle adopts radian system and the value range is [0, π). The line point confidence prediction adopts a classification prediction mode, and the cross entropy is used as the loss function. In order to improve the prediction accuracy of the network for difficult samples, a calculation method similar to the focal loss function is adopted, but the sampling probability of positive and negative samples can be set during the training process, and the sample imbalance problem is not easy to occur, so the calculation item for relieving the class imbalance degree in the focal loss is deleted. The calculation formula of the loss function l c in the class prediction is:

[0027] l c = -2|c p -c g | λ c g logc p (1)

[0028] Where c p ,c g are the class prediction confidence and the true class result respectively, and λ is a parameter for balancing difficult samples, which is set to 2 in the method.

[0029] The angle prediction adopts the absolute value loss function. Since the tangent angle prediction result at the non-power line point is meaningless, the angle loss function l a is set to be negatively correlated with the distance between the point and the nearest line point in the label during training, and the calculation formula is:

[0030]

[0031] Where a p ,a g These are the angle prediction result and the actual line point tangent angle, respectively. d is the distance between the point and the nearest line point in the label, and D is a parameter for adjusting the angle loss, which is set to 30 in the method.

[0032] Step 2: Set a threshold of 0.5, remove points with confidence scores less than or equal to the threshold, and only retain the detection results of line points with confidence scores greater than the threshold. To further reduce the computational load, randomly retain 25% of these remaining line points for subsequent calculation steps;

[0033] Step 3: Using the filtered line point confidence and angle prediction results as input, the designed multi-curve fitting method is used to obtain the preliminary detection results of the power lines. The multi-curve fitting method is based on red-black trees, and its specific method is as follows:

[0034] (3.1) First, create an empty tree and represent all line points in the form of a linked list;

[0035] (3.2) Starting from the root node of each red-black tree, the first node in the linked list is compared with the size of nodes in the red-black tree layer by layer until the closest node in the tree is found. The calculation method for the size of the node and the node in the tree is as follows: First, calculate the perpendicular line from the node in the tree, expressed in the standard form of a straight line. Then, substitute the node into the formula for calculating the perpendicular line. If the result is positive, the node is considered to be greater than the node in the tree; otherwise, the node is considered to be less than the node in the tree.

[0036] (3.3) Calculate the distance, angular difference, positional difference, and positional interval in different directions between the line point and the nearest node; where the distance is divided into the projected distance d relative to the tangent direction of the node. x and the projected distance d relative to the tangent of the node in the perpendicular direction y Angular difference d ang The formula for calculating the difference between the predicted tangent line at a line point and the predicted angle between two adjacent nodes in the tree is as follows:

[0037] d ang =|aa left |+|aa right |-|a left -a right | (3)

[0038] Among them a, a left ,a right These represent the angles of the line point and the tree nodes located to its left and right, respectively. Positional difference d posThe smoothness between the line point to be inserted and the two adjacent nodes is measured, and is defined as the difference between the angle formed by the line point and the two adjacent nodes and π. pos and d ang Both are expressed in radian system.

[0039] The position interval in different directions calculates the distance between the line point and the farthest merged point of the node in the tree in different directions, and by eliminating the points that are too far from the merged point set of the current tree node, the purpose of suppressing false alarm interference within a certain range around the correct line point is achieved. In the present application, a circle of 360° is evenly divided into 36 directions, each direction covers an area of 10°, and only the point that is farthest from the node center point among the points in the merged point set that fall in the direction is retained. Since the position of the node center will change every time the merging is performed, in order to reduce the calculation amount, the position of the node center point is always used as the center point of the first point in the node when calculating the position interval in different directions.

[0040] (3.4) According to the calculation result in 3.3, when the tangent direction projection distance is less than 20 and the tangent perpendicular direction projection distance is less than 30, if the angle difference degree is less than 0.6 and the position difference degree is less than 0.8, the line point is merged with the node in the tree, and jumps to 3.7. The merging is performed in a weighted manner, and the weight is the line point confidence degree predicted by the CNN model in step 1. The node position and angle update formulas are respectively:

[0041]

[0042] Where conf poi is the line point confidence degree, x poi , y poi , and θ poi are the horizontal and vertical coordinates of the line point and the tangent direction respectively. x old , x new , y old , y new are the horizontal coordinates of the node in the tree before and after the update and the vertical coordinates before and after the update respectively. θ new is the tangent direction of the node in the tree after the update.

[0043] (3.5) If the tangent direction projection distance is less than 40 and the tangent perpendicular direction projection distance is less than 50, but the distance does not meet the merging requirement, and the remaining indicators meet the requirements, the line point is inserted into the tree as a leaf node of the current tree node, the red-black tree is rotated left or right according to the balance of the current tree, and jumps to 3.8;

[0044] (3.6) The line point is compared with the next tree, and jumps to 3.2. If all the trees have been compared, the line point is first placed at the tail of the linked list, and then jumps to 3.2;

[0045] (3.7) check whether the node in the merged tree and its adjacent nodes still satisfy the angle difference and position difference conditions in 3.4, if yes, jump to 3.8, otherwise jump to 3.6;

[0046] (3.8) remove the line point from the linked list, and jump to 3.2;

[0047] (3.9) if the linked list does not complete any merging or inserting operation in a loop, create a new tree with the first line point in the linked list as the root node;

[0048] (3.10) repeat the above process until the linked list is empty;

[0049] Step 4: perform a pre-order traversal on each tree obtained in step 3, and connect each node obtained in the traversal in sequence to obtain a preliminary detection result of the power line in the form of a line segment;

[0050] Step 5: perform a smoothing operation on each power line detection result sequence. When smoothing, the end points at the two ends of the line are kept unchanged, and for the intermediate nodes, the smoothing depends on the positions of the nodes before and after the intermediate nodes. Let the coordinates of the point to be smoothed and its left and right nodes be (x1, y1), (x0, y0) and (x2, y2) respectively, and the calculation formula is:

[0051]

[0052] In the formula, v is an intermediate variable, is the coordinate of the smoothed line point.

[0053] Step 6: calculate the sum of the confidence degrees of all line points contained in each power line, and arrange the power line sequences in descending order of confidence degree. Accumulate the confidence degrees of the power line sequences, and when the accumulated confidence degree accounts for more than a threshold (0.8 in the method) of the total confidence degree of all power lines, delete the remaining power lines in the sequence and delete the power lines with a length less than 80, so as to suppress the interference of false alarms in the detection result of the peak value detection model.

[0054] It should be noted that the red-black tree can effectively maintain the balance of the tree and the efficiency of insertion and search when there are many nodes, so the structure based on the red-black tree is used in the power line fitting part of the method, but any ordered data structure can be used to replace the red-black tree to realize the curve fitting function in step 3. At the same time, the image segmentation network model with dense point prediction can also realize the peak value detection function based on CNN.

[0055] The advantages of the embodiment of the application are:

[0056] (1) The power line points are pre-extracted by using a convolutional neural network (CNN). In the traditional power line detection method, the constant false alarm detection method is used to extract the power line points and tower points. These methods need to set the detection threshold in advance. If the detection threshold is set too high, it is easy to miss detection, and if the detection threshold is too low, it is easy to produce a large number of false alarms. In this method, the power line points are automatically extracted by using the deep learning method, without the need to set the detection threshold in advance, and the accuracy of power line point detection can be significantly improved.

[0057] (2) By predicting the tangent direction of the power line point, the position of the power line can be detected without using the Hough transform, avoiding the assumption that most existing millimeter wave radar power line detection methods rely on the power line being a straight line. At the same time, this method can still achieve high recognition rate without the power line discrimination step in the traditional method, avoiding excessive reliance on prior knowledge of high-voltage lines, and has strong scene applicability.

[0058] The above is only a specific embodiment of the present application, which cannot limit the scope of the application. Therefore, the replacement of equivalent components or equivalent changes and modifications made within the scope of the present application should still fall within the scope of the present patent. In addition, the technical features in the present application can be freely combined with each other, and the technical features can be freely combined with each other.

Claims

1. A millimeter wave radar high-voltage line intelligent detection and trend prediction method, characterized in that, The method comprises the following steps: Step one: taking the radar echo map in an antenna scanning cycle as the input of the CNN network, wherein the CNN network adopts an ENet network model, the feature extraction part of the network is composed of 7 stage modules, each stage module is composed of a plurality of bottleneck structures, the output of the feature extraction network is connected to two branches, each branch comprises two fully connected layers, respectively predicting the line point confidence at each pixel point and the tangent angle at the line point, the line point confidence prediction adopts a classification prediction mode, and cross entropy is used as the loss function; Step two: setting a first threshold, and retaining the line point detection result greater than the first threshold; Step three: using the line point detection result as input, using a multi-curve fitting method based on a red-black tree to obtain the preliminary detection result of a plurality of power lines; Step four: performing pre-order traversal on the preliminary detection result of each power line, and sequentially connecting each node obtained by traversal to obtain the power line detection result in the form of a line segment; Step five: performing smoothing operation on each power line detection result sequence in the form of a line segment; Step six: setting a second threshold, removing the line segment form power line whose confidence or length is less than the second threshold, and outputting the remaining line segment form power line as the detection result; The step one comprises: setting a classification loss function as wherein, are a class prediction confidence and a real class result, respectively, is a parameter balancing difficult samples. The step one further comprises: setting angle prediction to adopt an absolute value loss function wherein are the angle prediction result and the real line point tangent angle respectively, d is the distance of the point from the nearest line point in the label, and D is a parameter for adjusting the angle loss.

2. The millimeter wave radar high-voltage line intelligent detection and direction prediction method according to claim 1, characterized in that, The step three comprises: Step 3.1: a new empty tree is created, and all line point detection results greater than the first threshold are represented in the form of a linked list; Step 3.2: the first line point in the linked list is compared with the nodes in the root node of each red-black tree layer by layer, until the closest node in the red-black tree is found; Step 3.3: the distance, angle difference, position difference and position interval in different directions between the line point detection result greater than the first threshold and the closest node are calculated; Step 3.4: according to the calculation result in step 3.3, when the distance, angle difference, position difference and direction interval all meet the set conditions, the current line point and the node in the tree are merged, and step 3.7 is jumped to; Step 3.5: when only the distance does not meet the set conditions, the line point is inserted into the tree as a leaf node, and the red-black tree is rotated left or right according to the balance of the current tree, and step 3.8 is jumped to; Step 3.6: the line point is compared with the next tree, and step 3.2 is jumped to, if all the trees have been compared, the line point is placed at the tail of the linked list, and then step 3.2 is jumped to; Step 3.7: the nodes in the merged tree and their adjacent nodes are checked to see whether they still meet the conditions in step 3.4, if yes, step 3.8 is jumped to, otherwise step 3.6 is jumped to; Step 3.8: the line point is removed from the linked list, and then step 3.2 is jumped to; Step 3.9: if the linked list does not complete any merging or inserting into the tree in one cycle, a new tree is created with the first line point in the linked list as the root node; Step 3.10: the above steps are cycled until the linked list is empty.

Images