Unmanned aerial vehicle air traffic collision prediction method

Abstract

The invention discloses an unmanned aerial vehicle air traffic collision prediction method, and belongs to the technical field of image processing. A camera carried by an unmanned aerial vehicle is used for collecting an environment image in the advancing direction, a contour map is obtained through contour extraction, and a closed contour is marked. Then, the linearity of contour pixel points on the inner side and the outer side of the closed contour is obtained, a contour linear ratio vector is obtained according to the linearity, and the matched closed contour is found out by calculating the similarity of the closed contour on the vector on the contour map at each moment. Performing binarization processing on the matched closed contour to obtain a matched contour binary image, and constructing an inner side and outer side confidence feature map according to the linearity; and processing the feature map by using a three-channel convolutional neural network to realize classification of the matched closed contours, and judging whether a collision risk exists based on the group of matched closed contours when the unmanned aerial vehicle is classified, thereby providing an effective guarantee for safe flight of the unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for predicting air traffic collisions of unmanned aerial vehicles. Background Art

[0002] Drone air traffic safety is one of the most critical research areas in modern aviation technology. With the widespread application of drones in civil and commercial fields, the complexity of air traffic is increasing, and collision prevention has become a core challenge to ensure flight safety. At present, existing drone obstacle avoidance technologies mainly rely on traditional visual recognition methods. These methods mainly use neural networks to identify drones, but achieving high-precision drone recognition requires a huge neural network, and when the network structure is lightweight, the accuracy of aerial drone recognition will drop significantly.

[0003] Especially in the rapidly changing aerial environment, the performance shortcomings of lightweight neural networks are more prominent, and traditional visual recognition methods are unable to extract target contour features. Since such neural networks cannot conduct in-depth analysis of complex and changing aerial scenes, when faced with drones in different flight postures, it is difficult to accurately capture their unique contour features, effectively filter out interference information, and accurately extract the key features of target drones, resulting in low drone recognition accuracy and low drone air traffic collision prediction accuracy. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, a method for predicting UAV air traffic collisions provided by the present invention solves the problem that when a lightweight neural network is used to identify UAVs, the accuracy of UAV identification is low, resulting in low accuracy in predicting UAV air traffic collisions.

[0005] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a method for predicting air traffic collision of unmanned aerial vehicles, comprising the following steps:

[0006] The camera carried by the UAV is used to collect the environment image in the forward direction, and the contour is extracted to obtain a contour map, and the closed contours on the contour map are marked;

[0007] Obtain linearity for inner contour pixels and outer contour pixels on each closed contour;

[0008] According to the ratio of the inner contour pixel points and the outer contour pixel points of each closed contour in terms of linearity, a contour linear ratio vector is obtained;

[0009] Calculate the similarity of each closed contour on the contour graph at each moment on the contour linear ratio vector, and find multiple groups of matching closed contours;

[0010] Binarization processing is performed on the matching closed contour on the contour map to obtain a matching contour binary map;

[0011] According to the linearity of the inner contour pixel points and the outer contour pixel points of the matching closed contour of the contour map, a linear confidence value is obtained, and an inner confidence feature map and an outer confidence feature map are constructed;

[0012] A three-channel convolutional neural network is used to process the matching contour binary map, the inner confidence feature map and the outer confidence feature map to obtain the classification of the matching closed contour;

[0013] When the matched closed contours are classified as drones, it is determined whether there is a collision risk based on the set of matched closed contours.

[0014] Furthermore, the process of obtaining linearity includes:

[0015] Extracting contour pixel points of the inner edge and contour pixel points of the outer edge of the closed contour to obtain an inner closed contour and an outer closed contour;

[0016] Taking each contour pixel point in the inner closed contour or the outer closed contour as the center, multiple contour pixel points in the neighborhood are taken to obtain a contour segment;

[0017] In a contour segment, subtract the i+1th contour pixel from the ith contour pixel to obtain the ith vector, where i is a positive integer;

[0018] Calculate the angle between adjacent vectors;

[0019] The variance of each angle of a contour is taken as the linearity of the contour pixel at the center.

[0020] Furthermore, the formula for calculating the angle is: , where θ i is the i-th angle, v i is the i-th vector, v i+1 is the i+1th vector, arccos is the inverse cosine, i is a positive integer, and || is the modulo operation.

[0021] Furthermore, the process of obtaining the contour linear ratio vector is:

[0022] Arrange the linearity of the inner contour pixel points on the closed contour from large to small to obtain the inner linearity sequence;

[0023] Arrange the linearity of the outer contour pixel points on the closed contour from large to small to obtain an outer linearity sequence;

[0024] The inner linearity sequence is divided into N segments, and the inner linearity of each segment is averaged to obtain the inner mean vector, where N is a positive integer;

[0025] Divide the outer linearity sequence into N segments, take the average of the outer linearity of each segment, and obtain the outer mean vector;

[0026] Divide the outer mean vector by the inner mean vector bitwise to obtain the contour linear ratio vector.

[0027] Furthermore, the process of finding multiple sets of matching closed contours includes:

[0028] Randomly select a contour image from the contour group images at multiple consecutive moments, and calculate the similarity between the contour linear ratio vector of the nth closed contour on the contour image and the contour linear ratio vector of each closed contour on each of the remaining contour images, where the initial value of n is 1;

[0029] Take the maximum similarity among all similarities corresponding to the remaining contour image;

[0030] When the maximum similarities on all remaining contour images are greater than the similarity threshold, the corresponding closed contours on each contour image are grouped into a group of matching closed contours;

[0031] Add 1 to n and repeat the matching process until all closed contours on any selected contour map are traversed, and multiple groups of matching closed contours are obtained.

[0032] Furthermore, the process of obtaining the matching contour binary image includes: setting pixel values ​​of pixel points belonging to the matching closed contour in the contour image to 1, and setting pixel values ​​of other pixel points to 0, to obtain the matching contour binary image.

[0033] Furthermore, the process of constructing the inner confidence feature map and the outer confidence feature map includes:

[0034] According to the linearity of the inner contour pixel point of the matching closed contour of the contour map, the linear confidence value of the inner contour pixel point is calculated, the linear confidence value is used to replace the original pixel value, and the pixel values ​​of other pixels on the contour map are set to 0 to obtain the inner confidence feature map;

[0035] According to the linearity of the outer contour pixel points of the matching closed contour of the contour map, the linear confidence value of the outer contour pixel points is calculated; the linear confidence value replaces the original pixel value, and the pixel values ​​of other pixels on the contour map are set to 0 to obtain the outer confidence feature map.

[0036] Furthermore, the process of calculating the linear confidence value is: subtracting the linearity of the contour pixel point from the maximum linearity to obtain the linearity difference, and taking the ratio of the linearity difference to the maximum linearity as the linear confidence value.

[0037] Furthermore, the three-channel convolutional neural network includes: a first convolution block, a second convolution block, a third convolution block, an adder A1, an adder A2, a first residual neural network, a second residual neural network, a Concat layer, a CNN network and a fully connected layer;

[0038] The input end of the first convolution block is used to input the inner confidence feature map; the input end of the second convolution block is used to input the matching contour binary map; the input end of the third convolution block is used to input the outer confidence feature map;

[0039] The first input terminal of adder A1 is connected to the output terminal of the first convolution block, and the second input terminal thereof is connected to the output terminal of the second convolution block; the first input terminal of adder A2 is connected to the output terminal of the third convolution block, and the second input terminal thereof is connected to the output terminal of the second convolution block; the output terminal of adder A1 is connected to the input terminal of the first residual neural network; the output terminal of adder A2 is connected to the input terminal of the second residual neural network; the input terminal of the Concat layer is respectively connected to the output terminal of the first residual neural network and the output terminal of the second residual neural network, and the output terminal thereof is connected to the input terminal of the CNN network; the input terminal of the fully connected layer is connected to the output terminal of the CNN network, and the output terminal thereof serves as the output terminal of the three-channel convolutional neural network.

[0040] Furthermore, when the matched closed contour is classified as a drone, the corresponding group of matched closed contours is extracted, and the area ratio of each matched closed contour is calculated. When any area ratio is greater than the area ratio threshold, there is a collision risk.

[0041] The beneficial effects of the present invention are:

[0042] 1. The present invention extracts the contour of the environmental image and obtains the closed contour on the contour map to distinguish the target from the background, and then obtains the linearity of the inner contour pixel points and the outer contour pixel points of each closed contour to reflect the edge characteristics of the closed contour.

[0043] 2. The present invention can determine which closed contours belong to the imaging results of the same drone at different times by comparing the contour linear ratio vector similarity of closed contours on contour maps at different times. Thus, the corresponding closed contours belonging to the same area are grouped together, and the contour of the suspected drone is extracted on each contour map, further mining the drone contour and eliminating the interference of other contours.

[0044] 3. The present invention further performs binarization processing on the matching closed contour on the contour map to obtain a matching contour binary map, so that the drone contour is prominent. Then, according to the linearity of the inner contour pixel points and the outer contour pixel points of the matching closed contour, a linear confidence value is obtained, which can further quantify the characteristics of the drone contour. Since the drone contour is relatively regular, the linear confidence value can well reflect the characteristics of its contour.

[0045] 4. The matching contour binary map provides the contour information of the target, and the inner and outer confidence feature maps are supplemented from the perspective of linear features inside and outside respectively, enriching the input feature dimension of the network. Compared with the single input feature of lightweight neural networks, this multi-channel input method enables the network to learn more comprehensive and in-depth feature representations, improving the recognition accuracy of drones and thus enhancing the prediction accuracy of UAV air traffic collisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flowchart of a method for predicting UAV air traffic collisions;

[0047] Figure 2 is a schematic diagram of the positions of the contour pixel points of the inner edge and the contour pixel points of the outer edge;

[0048] Figure 3 is a schematic diagram of the structure of a three-channel convolutional neural network;

[0049] Figure 4 is a schematic diagram of the structure of a residual neural network. DETAILED DESCRIPTION OF THE INVENTION

[0050] The following describes the specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions made using the concept of the present invention are within the scope of protection.

[0051] As Figure 1 shown, a method for predicting UAV air traffic collisions includes the following steps:

[0052] Use the camera mounted on the UAV to collect the environmental image in the forward direction, extract the contour to obtain the contour map, and label the closed contours on the contour map;

[0053] Obtain the linearity of the inner contour pixel points and the outer contour pixel points on each closed contour;

[0054] According to the ratio of the inner contour pixel points and the outer contour pixel points of each closed contour in terms of linearity, obtain the contour linear ratio vector;

[0055] Calculate the similarity of each closed contour on the contour map at each moment in the contour linear ratio vector, and find multiple groups of matching closed contours;

[0056] Perform binarization processing on the matching closed contours on the contour map to obtain the matching contour binary map;

[0057] According to the linearity of the inner contour pixel points and the outer contour pixel points of the matching closed contour of the contour map, a linear confidence value is obtained, and an inner confidence feature map and an outer confidence feature map are constructed;

[0058] A three-channel convolutional neural network is used to process the matching contour binary map, the inner confidence feature map and the outer confidence feature map to obtain the classification of the matching closed contour;

[0059] When the matched closed contours are classified as drones, it is determined whether there is a collision risk based on the set of matched closed contours.

[0060] In this embodiment, the internal hollowing method is used to extract the contour of the environmental image, that is, in the environmental image after grayscale processing, with each pixel point as the center, when the pixel values ​​in the 3×3 neighborhood are the same as the pixel value at the center, the pixel point at the center is a non-contour point, and all non-contour points in the environmental image are marked, and the pixel values ​​of the non-contour points are set to 0 to obtain an initial contour map, and then the initial contour map is eroded to separate the interconnected contours to obtain the desired contour map.

[0061] In this embodiment, the process of obtaining linearity includes:

[0062] Extracting contour pixel points of the inner edge and contour pixel points of the outer edge of the closed contour to obtain an inner closed contour and an outer closed contour;

[0063] Taking each contour pixel point in the inner closed contour or the outer closed contour as the center, multiple contour pixel points in the neighborhood are taken to obtain a contour segment, wherein the length of a contour segment can be 5 or 7. When the length is 5, 2 contour pixel points are obtained in the neighborhood on both sides in addition to the center. When the length is 7, 3 contour pixel points are obtained in the neighborhood on both sides in addition to the center.

[0064] In a contour segment, subtract the i+1th contour pixel from the ith contour pixel to obtain the ith vector, where i is a positive integer;

[0065] Calculate the angle between adjacent vectors;

[0066] The variance of each angle of a contour is taken as the linearity of the contour pixel at the center.

[0067] The i-th vector is: v i =(x i+1 -x i , y i+1 -y i ), where v i is the i-th vector, x i is the horizontal coordinate of the i-th contour pixel, y i is the ordinate of the i-th contour pixel, xi+1 is the horizontal coordinate of the i+1th contour pixel, y i+1 is the ordinate of the i+1th contour pixel. The formula for calculating the angle is: , where θ i is the i-th angle, v i is the i-th vector, v i+1 is the i+1th vector, arccos is the inverse cosine, i is a positive integer, and || is the modulo operation.

[0068] The present invention extracts the inner closed contour and the outer closed contour. In order to evaluate the linearity of each pixel point, the pixel point is taken as the center, and multiple contour pixel points in the neighborhood range are extracted to form a contour. The linearity of a contour is used as the linearity of the pixel point at the center. The angle between adjacent vectors is calculated, and the variance of the angle is used as the linearity to quantify the linearity of the contour in the local area. The variance can reflect the discrete degree of the angle between adjacent vectors. A small variance indicates that the contour has a high degree of linearity and a smooth trend in the local area; a large variance indicates that the contour has a large bending change in the area, which can accurately reflect the linear characteristics of the contour.

[0069] In this embodiment, when the internal hollowing method is used to extract the contour, the pixel values ​​at the edge present a gradual characteristic, which makes the contour line relatively wide in the width dimension. The specific method for determining the inner and outer edge contour pixel points of the closed contour is as follows: traverse from the internal area surrounded by the closed contour, for any pixel point with a non-zero pixel value, if there is a pixel point with a zero pixel value in the 3×3 neighborhood centered on it, then the central pixel point is identified as the inner edge contour pixel point; similarly, traverse from the outer area of ​​the closed contour, when there is a pixel point with a zero pixel value in the 3×3 neighborhood centered on a pixel point with a non-zero pixel value, the central pixel point is determined to be the outer edge contour pixel point, such as Figure 2 For easy observation, Figure 2 The area with pixel value 0 is represented by white. Figure 2 The medium black area is the outline area.

[0070] A closed contour refers to an area in an image that is composed of a series of continuous pixels and connected end to end. In this embodiment, the process of marking a closed contour includes: extracting a single connected area (such as Figure 2 As shown in the inner white area in the figure, the outer contour of the simply connected region is marked as a closed contour.

[0071] In this embodiment, the process of obtaining the contour linear ratio vector is:

[0072] Arrange the linearity of the inner contour pixel points on the closed contour from large to small to obtain the inner linearity sequence;

[0073] Arrange the linearity of the outer contour pixel points on the closed contour from large to small to obtain an outer linearity sequence;

[0074] The inner linearity sequence is divided into N segments, and the inner linearity of each segment is averaged to obtain the inner mean vector, where N is a positive integer;

[0075] Divide the outer linearity sequence into N segments, take the average of the outer linearity of each segment, and obtain the outer mean vector;

[0076] Divide the outer mean vector by the inner mean vector bitwise to obtain the contour linear ratio vector.

[0077] In this embodiment, N is 3, and the inner linearity sequence and the outer linearity sequence are evenly divided into 3 segments of data. The specific size of N can be adjusted, and the linearity is arranged from large to small to achieve partition comparison. The sequence is divided into N segments. On the one hand, the amount of data is reduced and the calculation complexity is reduced; on the other hand, the mean operation can weaken the influence of noise to a certain extent, so that the obtained feature vector is more stable and representative.

[0078] The present invention obtains a contour linearity ratio vector by dividing the outer mean vector by the inner mean vector in phase, and further explores the difference relationship between the inner and outer contour linearities.

[0079] In this embodiment, the process of finding multiple sets of matching closed contours includes:

[0080] Randomly select a contour image from the contour group images at multiple consecutive moments, and calculate the similarity between the contour linear ratio vector of the nth closed contour on the contour image and the contour linear ratio vector of each closed contour on each of the remaining contour images, where the initial value of n is 1;

[0081] Take the maximum similarity among all similarities corresponding to the remaining contour image;

[0082] When the maximum similarities on all remaining contour images are greater than the similarity threshold, the corresponding closed contours on each contour image are grouped into a group of matching closed contours;

[0083] Add 1 to n and repeat the matching process until all closed contours on any selected contour map are traversed, and multiple groups of matching closed contours are obtained.

[0084] In the present invention, any contour map is taken as a reference contour map, and the contour linear ratio vectors of each closed contour in the reference contour map are calculated for similarity with the contour linear ratio vectors of each closed contour on other contour maps. Since there are multiple closed contours on each of the other contour maps, for one closed contour in the reference contour map, there will be multiple similarities corresponding to each of the remaining contour maps. The maximum similarity is selected, and it is determined whether all the maximum similarities corresponding to the remaining contour maps are greater than the similarity threshold. If so, the closed contours corresponding to the respective maximum similarities are a group of matching closed contours, indicating that the contour shape has always existed in the images at each consecutive moment. If not, the contour shape does not always exist at each consecutive moment and may be an area in the background.

[0085] In the present invention, only when the maximum similarities on all the remaining contour maps are greater than the threshold, the corresponding closed contours are grouped into a group of matching closed contours, which can effectively filter out contours with low similarities, avoid mis-matching irrelevant contours, further improve the accuracy of matching, and ensure that the contours in the finally obtained group of matching closed contours truly belong to the same target.

[0086] In this embodiment, the cosine similarity is used as the similarity, and the similarity threshold is between 0.5 and 0.9. The specific threshold can be adjusted in real time according to experiments or experience.

[0087] In this embodiment, the process of obtaining the matching contour binary map includes: setting the pixel value of the pixel points belonging to the matching closed contours in the contour map to 1, and setting the pixel values of other pixel points to 0, to obtain the matching contour binary map.

[0088] In this embodiment, the process of constructing the inner confidence feature map and the outer confidence feature map includes:

[0089] According to the linearity of the inner contour pixel points of the matching closed contours in the contour map, calculate the linear confidence value of the inner contour pixel points, replace the original pixel value with the linear confidence value, and set the pixel values of other pixel points on the contour map to 0, to obtain the inner confidence feature map;

[0090] According to the linearity of the outer contour pixel points of the matching closed contours in the contour map, calculate the linear confidence value of the outer contour pixel points; replace the original pixel value with the linear confidence value, and set the pixel values of other pixel points on the contour map to 0, to obtain the outer confidence feature map.

[0091] In this embodiment, the process of calculating the linear confidence value is: subtract the linearity of the contour pixel points from the maximum linearity to obtain the linearity difference, and use the ratio of the linearity difference to the maximum linearity as the linear confidence value.

[0092] The present invention calculates the linear confidence values ​​of the inner and outer contour pixels and replaces the original pixel values, while setting other pixels to 0, thereby highlighting the inner and outer linear features of the matching closed contour.

[0093] The present invention uses the ratio of the difference between the maximum linearity and the linearity of the contour pixel point to the maximum linearity as the linear confidence value, which reflects the reliability of the linear characteristics of the contour pixel point. The higher the linear confidence value, the more prominent the contour linear characteristics of the pixel point are and the more they meet expectations.

[0094] like Figure 3 As shown, the three-channel convolutional neural network includes: a first convolution block, a second convolution block, a third convolution block, an adder A1, an adder A2, a first residual neural network, a second residual neural network, a Concat layer, a CNN network and a fully connected layer;

[0095] The input end of the first convolution block is used to input the inner confidence feature map; the input end of the second convolution block is used to input the matching contour binary map; the input end of the third convolution block is used to input the outer confidence feature map;

[0096] The first input terminal of adder A1 is connected to the output terminal of the first convolution block, and the second input terminal thereof is connected to the output terminal of the second convolution block; the first input terminal of adder A2 is connected to the output terminal of the third convolution block, and the second input terminal thereof is connected to the output terminal of the second convolution block; the output terminal of adder A1 is connected to the input terminal of the first residual neural network; the output terminal of adder A2 is connected to the input terminal of the second residual neural network; the input terminal of the Concat layer is respectively connected to the output terminal of the first residual neural network and the output terminal of the second residual neural network, and the output terminal thereof is connected to the input terminal of the CNN network; the input terminal of the fully connected layer is connected to the output terminal of the CNN network, and the output terminal thereof serves as the output terminal of the three-channel convolutional neural network.

[0097] In this embodiment, the first residual neural network and the second residual neural network both include: a first convolutional layer, a first ReLU layer, a second convolutional layer, an adder A3 and a second ReLU layer, such as Figure 4 shown.

[0098] In this embodiment, the first convolution block, the second convolution block, and the third convolution block all include: a convolution layer, an activation function layer, and a normalization layer. The convolution layers in the first convolution block, the second convolution block, and the third convolution block all use a convolution kernel of size 3×3.

[0099] The present invention realizes the integration of multi-source features by processing and fusing the inner confidence feature map, matching contour binary map, and outer confidence feature map with convolution blocks, and significantly improves the accuracy of target recognition. At the same time, the residual neural network is introduced to effectively overcome the problem of gradient disappearance in network training. The two-way features are then spliced ​​through the Concat layer, and the deep processing of CNN and the fully connected layer is used to achieve high-precision recognition of drones in the air environment.

[0100] The three images input to the three-channel convolutional neural network come from the same contour image.

[0101] The three-channel convolutional neural network of the present invention has much fewer specific convolutional layers than the existing YOLO neural network for target recognition, significantly reduces the number of parameters, and has lower computational overhead.

[0102] In this embodiment, when the matching closed contour is classified as a drone, the corresponding group of matching closed contours is extracted, and the area ratio of each matching closed contour is calculated. When any area ratio is greater than the area ratio threshold, there is a collision risk and an alarm is performed. Alternatively, the area change speed of the group of matching closed contours can be calculated. When the area change speed is greater than the area change speed threshold, there is a collision risk and an alarm is performed. In this embodiment, the area is represented by the number of pixels occupied by the area. The area change speed threshold and the area ratio threshold are specifically set according to needs or experiments.

[0103] In this embodiment, in the group of matching closed contours, the area difference is obtained by subtracting the number of pixels occupied by the initial matching closed contour area from the number of pixels occupied by the latest matching closed contour area, and the area change rate is obtained by dividing the area difference by the interval time.

[0104] The present invention extracts the contour of the environment image and obtains the closed contour on the contour map to distinguish the target from the background, and then obtains the linearity of the inner contour pixel points and the outer contour pixel points of each closed contour to reflect the edge characteristics of the closed contour.

[0105] Relying solely on the linearity of the inner or outer contour pixels to describe a closed contour is easily disturbed by environmental factors, such as lighting changes, background noise, etc. However, by calculating the ratio of the two, the influence of these interference factors can be offset to a certain extent. Because when the lighting or background changes, the linearity of the inner and outer contour pixels will be affected at the same time, but the ratio between them is relatively stable. At the same time, in the environmental images taken at different times, the size of the drone in the environmental image is different. The ratio of the two is not affected by the size, and the contour linear ratio vector will remain stable.

[0106] The present invention can determine which closed contours belong to the imaging results of the same drone at different times by comparing the contour linear ratio vector similarity of closed contours on contour maps at different times. Thus, the corresponding closed contours belonging to the same area are grouped together, and the contour of the suspected drone is extracted on each contour map, further mining the drone contour and eliminating the interference of other contours.

[0107] The present invention further performs binarization processing on the matching closed contour on the contour map to obtain a matching contour binary map, so that the drone contour is prominent. Then, according to the linearity of the inner contour pixel points and the outer contour pixel points of the matching closed contour, a linear confidence value is obtained, which can further quantify the characteristics of the drone contour. Since the drone contour is relatively regular, the linear confidence value can well reflect the characteristics of its contour.

[0108] The present invention adopts a three-channel convolutional neural network to process the matching contour binary map, the inner confidence feature map and the outer confidence feature map to obtain the classification of the matching closed contour, realize the comprehensive three-aspect contour features of the suspected drone, realize the accurate classification of the matching closed contour, and improve the drone recognition accuracy.

[0109] The matching contour binary map provides the contour information of the target, and the inner and outer confidence feature maps supplement it from the perspective of internal and external linear features, respectively, enriching the input feature dimension of the network. Compared with the single input feature of the lightweight neural network, this multi-channel input method enables the network to learn a more comprehensive and in-depth feature representation, improve the accuracy of drone recognition, and thus improve the accuracy of drone air traffic collision prediction.

[0110] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for predicting UAV air traffic collision, characterized in that: The following steps are involved: The camera carried by the UAV is used to collect the environment image in the forward direction, and the contour is extracted to obtain a contour map, and the closed contours on the contour map are marked; Obtain linearity for inner contour pixels and outer contour pixels on each closed contour; According to the ratio of the inner contour pixel points and the outer contour pixel points of each closed contour in terms of linearity, a contour linear ratio vector is obtained; Calculate the similarity of each closed contour on the contour graph at each moment on the contour linear ratio vector, and find multiple groups of matching closed contours; Binarization processing is performed on the matching closed contour on the contour map to obtain a matching contour binary map; According to the linearity of the inner contour pixel points and the outer contour pixel points of the matching closed contour of the contour map, a linear confidence value is obtained, and an inner confidence feature map and an outer confidence feature map are constructed; A three-channel convolutional neural network is used to process the matching contour binary map, the inner confidence feature map and the outer confidence feature map to obtain the classification of the matching closed contour; When the matched closed contours are classified as drones, it is determined whether there is a collision risk based on the set of matched closed contours.

2. The method for predicting UAV air traffic collision according to claim 1, characterized in that: The process of obtaining linearity includes: Extracting contour pixel points of the inner edge and contour pixel points of the outer edge of the closed contour to obtain an inner closed contour and an outer closed contour; Taking each contour pixel point in the inner closed contour or the outer closed contour as the center, multiple contour pixel points in the neighborhood are taken to obtain a contour segment; In a contour segment, subtract the i+1th contour pixel from the ith contour pixel to obtain the ith vector, where i is a positive integer; Calculate the angle between adjacent vectors; The variance of each angle of a contour is taken as the linearity of the contour pixel at the center.

3. The method for predicting UAV air traffic collision according to claim 2, characterized in that: The formula for calculating the angle is: , where θ i is the i-th angle, v i is the i-th vector, v i+1 is the i+1th vector, arccos is the inverse cosine, i is a positive integer, and || is the modulo operation.

4. The method for predicting UAV air traffic collision according to claim 1, characterized in that: The process of obtaining the contour linear ratio vector: Arrange the linearity of the inner contour pixel points on the closed contour from large to small to obtain the inner linearity sequence; Arrange the linearity of the outer contour pixel points on the closed contour from large to small to obtain an outer linearity sequence; The inner linearity sequence is divided into N segments, and the inner linearity of each segment is averaged to obtain the inner mean vector, where N is a positive integer; Divide the outer linearity sequence into N segments, take the average of the outer linearity of each segment, and obtain the outer mean vector; Divide the outer mean vector by the inner mean vector bitwise to obtain the contour linear ratio vector.

5. The method for predicting UAV air traffic collision according to claim 1, characterized in that: The process of finding multiple sets of matching closed contours includes: Randomly select a contour image from the contour group images at multiple consecutive moments, and calculate the similarity between the contour linear ratio vector of the nth closed contour on the contour image and the contour linear ratio vector of each closed contour on each of the remaining contour images, where the initial value of n is 1; Take the maximum similarity among all similarities corresponding to the remaining contour image; When the maximum similarities on all remaining contour images are greater than the similarity threshold, the corresponding closed contours on each contour image are grouped into a group of matching closed contours; Add 1 to n and repeat the matching process until all closed contours on any selected contour map are traversed, and multiple groups of matching closed contours are obtained.

6. The method for predicting UAV air traffic collision according to claim 1, characterized in that: The process of obtaining the matching contour binary image includes: setting the pixel values ​​of the pixel points belonging to the matching closed contour in the contour image to 1, and setting the pixel values ​​of other pixel points to 0, to obtain the matching contour binary image.

7. The method for predicting UAV air traffic collision according to claim 1, characterized in that: The process of constructing the inner confidence feature map and the outer confidence feature map includes: According to the linearity of the inner contour pixel point of the matching closed contour of the contour map, the linear confidence value of the inner contour pixel point is calculated, the linear confidence value is used to replace the original pixel value, and the pixel values ​​of other pixels on the contour map are set to 0 to obtain the inner confidence feature map; According to the linearity of the outer contour pixel points of the matching closed contour of the contour map, the linear confidence value of the outer contour pixel points is calculated; the linear confidence value replaces the original pixel value, and the pixel values ​​of other pixels on the contour map are set to 0 to obtain the outer confidence feature map.

8. The method for predicting UAV air traffic collision according to claim 7, characterized in that: The process of calculating the linear confidence value is as follows: the linearity difference is obtained by subtracting the linearity of the contour pixel point from the maximum linearity, and the ratio of the linearity difference to the maximum linearity is taken as the linear confidence value.

9. The method for predicting UAV air traffic collision according to claim 1, characterized in that: The three-channel convolutional neural network includes: a first convolution block, a second convolution block, a third convolution block, an adder A1, an adder A2, a first residual neural network, a second residual neural network, a Concat layer, a CNN network and a fully connected layer; The input end of the first convolution block is used to input the inner confidence feature map; the input end of the second convolution block is used to input the matching contour binary map; the input end of the third convolution block is used to input the outer confidence feature map; The first input end of adder A1 is connected to the output end of the first convolution block, and the second input end thereof is connected to the output end of the second convolution block; the first input end of adder A2 is connected to the output end of the third convolution block, and the second input end thereof is connected to the output end of the second convolution block; the output end of adder A1 is connected to the input end of the first residual neural network; the output end of adder A2 is connected to the input end of the second residual neural network; the input end of the Concat layer is respectively connected to the output end of the first residual neural network and the output end of the second residual neural network, and the output end thereof is connected to the input end of the CNN network; the input end of the fully connected layer is connected to the output end of the CNN network, and the output end thereof serves as the output end of the three-channel convolutional neural network.

10. The method for predicting UAV air traffic collision according to claim 1, characterized in that: When the matched closed contour is classified as a drone, the corresponding group of matched closed contours is extracted, and the area ratio of each matched closed contour is calculated. When any area ratio is greater than the area ratio threshold, there is a collision risk.

Images