A method for predicting collisions in unmanned aerial vehicle air traffic

Abstract

The present invention discloses a method for predicting air traffic collisions of unmanned aerial vehicles, belonging to the technical field of image processing. In the present invention, the environmental image in the forward direction is collected by a camera carried by the unmanned aerial vehicle, and the contour map is obtained through contour extraction and the closed contours are labeled. Subsequently, the linearity of the inner and outer contour pixel points of the closed contour is obtained, and based on this, the contour linear ratio vector is obtained. By calculating the similarity of the closed contours on the contour map at each moment on this vector, the matching closed contours are found. The matching closed contours are binarized to obtain a matching contour binary map, and at the same time, the inner and outer confidence feature maps are constructed based on the linearity. The above feature maps are processed by a three-channel convolutional neural network to realize the classification of the matching closed contours. When the classification is an unmanned aerial vehicle, it is judged whether there is a collision risk based on this group of matching closed contours, providing an effective guarantee for the safe flight of the unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method for predicting air traffic collisions of unmanned aerial vehicles (UAVs). Background Art

[0002] The air traffic safety of UAVs is one of the most critical research fields in modern aviation technology. With the widespread application of UAVs in civilian and commercial fields, the complexity of air traffic is increasing day by day, and collision prevention has become the core challenge to ensure flight safety. Currently, the existing UAV obstacle avoidance technologies mainly rely on traditional visual recognition methods. These methods mainly use neural networks to identify UAVs, but achieving high-precision UAV recognition requires a huge neural network. When the network structure is lightweight, the recognition accuracy of UAVs in the air will decrease significantly.

[0003] Especially in a rapidly changing air environment, the performance shortcoming of lightweight neural networks becomes more prominent, and traditional visual recognition methods are unable to extract target contour features effectively. Since such neural networks cannot perform in-depth analysis on complex and changing air scenarios, it is difficult to accurately capture the unique contour features of UAVs with different flight postures, unable to effectively filter out interference information, and accurately extract the key features of the target UAV, resulting in low UAV recognition accuracy and low prediction accuracy of UAV air traffic collisions. Summary of the Invention

[0004] Aiming at the above deficiencies in the prior art, the method for predicting UAV air traffic collisions provided by the present invention solves the problems of low UAV recognition accuracy and low prediction accuracy of UAV air traffic collisions when using lightweight neural networks for UAV recognition.

[0005] To achieve the above invention objective, the technical solution adopted by the present invention is: a method for predicting UAV air traffic collisions, including the following steps:

[0006] Collect the environmental images in the forward direction by using a camera mounted on the UAV, extract the contours to obtain a contour map, and label the closed contours on the contour map;

[0007] Obtain the linearity of the inner contour pixels and outer contour pixels on each closed contour;

[0008] According to the ratio of the inner contour pixels and outer contour pixels of each closed contour in terms of linearity, obtain a contour linear ratio vector;

[0009] Calculate the similarity of each closed contour on the contour map at each moment in the contour linear ratio vector, and find multiple groups of matching closed contours;

[0010] Binarize the matching closed contours on the contour map to obtain a matching contour binary map;

[0011] Respectively, obtain linear confidence values based on the linearity of the inner contour pixels and outer contour pixels of the matching closed contours of the contour map, and construct an inner confidence feature map and an outer confidence feature map;

[0012] Use a three-channel convolutional neural network to process the matching contour binary map, the inner confidence feature map, and the outer confidence feature map to obtain the classification of the matching closed contours;

[0013] When the classification of the matching closed contours is a drone, based on this set of matching closed contours, determine whether there is a collision risk.

[0014] Further, the process of obtaining the linearity includes:

[0015] Extract the contour pixels of the inner edge and the outer edge of the closed contour to obtain an inner closed contour and an outer closed contour;

[0016] Taking each contour pixel in the inner closed contour or the outer closed contour as the center, and taking multiple contour pixels within the neighborhood range to obtain a section of contour;

[0017] Subtract the (i + 1)-th contour pixel from the i-th contour pixel in a section of contour to obtain the i-th vector, where i is a positive integer;

[0018] Calculate the angle between adjacent vectors;

[0019] Take the variance of the angles of a section of contour as the linearity of the contour pixel at the center.

[0020] Further, the formula for calculating the angle is: , where θ i is the i-th angle, v i is the i-th vector, v i+1 is the (i + 1)-th vector, arccos is the inverse cosine, i is a positive integer, and | | is the modulus operation.

[0021] Further, the process of obtaining the contour linear ratio vector:

[0022] Arrange the linearity of the inner contour pixels on the closed contour from largest to smallest to obtain an inner linearity sequence;

[0023] Arrange the linearity of the outer contour pixels on the closed contour from largest to smallest to obtain an outer linearity sequence;

[0024] Divide the inner linearity sequence into N segments, take the average value of each segment of the inner linearity to obtain an inner average vector, where N is a positive integer;

[0025] Divide the outer linearity sequence into N segments, take the average value of the outer linearity for each segment, and obtain the outer average vector;

[0026] Divide the outer average vector and the inner average vector bit by bit to obtain the contour linearity ratio vector.

[0027] Further, the process of finding multiple groups of matching closed contours includes:

[0028] Randomly select a contour map from the contour maps at multiple consecutive moments, calculate the similarity between the contour linearity ratio vector of the nth closed contour on this contour map and the contour linearity ratio vectors of each closed contour on each of the remaining contour maps, and the initial value of n is 1;

[0029] Take the maximum similarity among all the similarities corresponding to the remaining one contour map;

[0030] When the maximum similarities on all the remaining contour maps are greater than the similarity threshold, classify the corresponding closed contours on each contour map into a group of matching closed contours;

[0031] Increment n by 1, repeat the matching process until all the closed contours on the randomly selected contour map are traversed, and obtain multiple groups of matching closed contours.

[0032] Further, the process of obtaining the matching contour binary map includes: setting the pixel value of the pixel points belonging to the matching closed contours in the contour map to 1, and setting the pixel value of other pixel points to 0, to obtain the matching contour binary map.

[0033] Further, the process of constructing the inner confidence feature map and the outer confidence feature map includes:

[0034] According to the linearity of the inner contour pixel points of the matching closed contours of the contour map, calculate the linear confidence value of this inner contour pixel point, replace the original pixel value with the linear confidence value, and set the pixel values of other pixel points on the contour map to 0, to obtain the inner confidence feature map;

[0035] According to the linearity of the outer contour pixel points of the matching closed contours of the contour map, calculate the linear confidence value of this outer contour pixel point; replace the original pixel value with the linear confidence value, and set the pixel values of other pixel points on the contour map to 0, to obtain the outer confidence feature map.

[0036] Further, the process of calculating the linear confidence value is: subtract the linearity of the contour pixel point from the maximum linearity to obtain the linearity difference, and take the ratio of the linearity difference to the maximum linearity as the linear confidence value.

[0037] Further, the three-channel convolutional neural network includes: a first convolutional block, a second convolutional block, a third convolutional block, adder A1, adder A2, a first residual neural network, a second residual neural network, a Concat layer, a CNN network, and a fully connected layer;

[0038] The input end of the first convolutional block is used to input the inner confidence feature map; the input end of the second convolutional block is used to input the matching contour binary map; the input end of the third convolutional block is used to input the outer confidence feature map;

[0039] The first input end of adder A1 is connected to the output end of the first convolutional block, and its second input end is connected to the output end of the second convolutional block; the first input end of adder A2 is connected to the output end of the third convolutional block, and its second input end is connected to the output end of the second convolutional block; the output end of adder A1 is connected to the input end of the first residual neural network; the output end of adder A2 is connected to the input end of the second residual neural network; the input ends of the Concat layer are respectively connected to the output ends of the first residual neural network and the second residual neural network, and its output end is connected to the input end of the CNN network; the input end of the fully connected layer is connected to the output end of the CNN network, and its output end serves as the output end of the three-channel convolutional neural network.

[0040] Further, when the classification of the matching closed contour is a drone, extract the corresponding group of matching closed contours, calculate the area ratio for each matching closed contour, and when any area ratio is greater than the area ratio threshold, there is a collision risk.

[0041] The beneficial effects of the present invention are as follows:

[0042] 1. The present invention extracts the contours from the environmental image and obtains the closed contours on the contour map, realizing the separation of the target from the background. Then, the linearity of the inner contour pixel points and the outer contour pixel points of each closed contour is obtained to reflect the edge features of the closed contour.

[0043] 2. By comparing the similarity of the contour linear ratio vectors of the closed contours on the contour maps at different times, the present invention can determine which closed contours belong to the imaging results of the same drone at different times. Thus, the corresponding closed contours belonging to the same area are formed into a group, realizing the extraction of the suspected drone contours on each contour map, further excavating the drone contours, and eliminating the interference of other contours.

[0044] 3. The present invention then performs binary processing on the matching closed contours on the contour map to obtain the matching contour binary map, making the drone contours prominent. Then, according to the linearity of the inner contour pixel points and the outer contour pixel points of the matching closed contours, the linear confidence value is obtained, which can further quantify the features of the drone contours. Since the drone contours are relatively regular, the linear confidence value can well reflect the characteristics of its contours.

[0045] 4. The matching contour binary map provides the contour information of the target, and the inner and outer confidence feature maps are supplemented from the perspectives of internal and external linear features respectively, enriching the input feature dimension of the network. Compared with the single input feature of lightweight neural networks, this multi-channel input method enables the network to learn more comprehensive and in-depth feature representations, improving the recognition accuracy of drones and thus enhancing the prediction accuracy of drone air traffic collisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flowchart of a method for predicting drone air traffic collisions;

[0047] Figure 2 is a schematic diagram of the positions of the contour pixel points of the inner edge and the contour pixel points of the outer edge;

[0048] Figure 3 is a schematic diagram of the structure of a three-channel convolutional neural network;

[0049] Figure 4 is a schematic diagram of the structure of a residual neural network. DETAILED DESCRIPTION OF THE INVENTION

[0050] The following describes the specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions made using the concept of the present invention are within the scope of protection.

[0051] As Figure 1 shown, a method for predicting drone air traffic collisions includes the following steps:

[0052] Use a camera mounted on the drone to collect the environmental image in the forward direction, extract the contour to obtain a contour map, and label the closed contours on the contour map;

[0053] Obtain the linearity of the inner contour pixel points and the outer contour pixel points on each closed contour;

[0054] According to the ratio of the inner contour pixel points and the outer contour pixel points of each closed contour in terms of linearity, obtain a contour linear ratio vector;

[0055] Calculate the similarity of each closed contour on the contour map at each moment in the contour linear ratio vector, and find multiple groups of matching closed contours;

[0056] Perform binary processing on the matching closed contours on the contour map to obtain a matching contour binary map;

[0057] According to the linearity of the inner contour pixels and the outer contour pixels of the matching closed contour in the contour map respectively, a linear confidence value is obtained, and an inner confidence feature map and an outer confidence feature map are constructed;

[0058] A three-channel convolutional neural network is used to process the matching contour binary map, the inner confidence feature map and the outer confidence feature map to obtain the classification of the matching closed contour;

[0059] When the classification of the matching closed contour is a drone, based on this set of matching closed contours, it is judged whether there is a collision risk.

[0060] In this embodiment, the internal hollowing method is used to extract the contour of the environmental image, that is, in the environmental image after gray processing, when the pixel values within the 3×3 neighborhood range centered on each pixel point are the same as the pixel value at the center, the pixel point at the center is a non-contour point, all non-contour points in the environmental image are marked, and the pixel values of the non-contour points are set to 0 to obtain an initial contour map, and then the initial contour map is subjected to an erosion operation to separate the connected contours to obtain the required contour map.

[0061] In this embodiment, the process of obtaining the linearity includes:

[0062] The contour pixels of the inner edge and the outer edge of the closed contour are extracted to obtain the inner closed contour and the outer closed contour;

[0063] Taking multiple contour pixels within the neighborhood range centered on each contour pixel in the inner closed contour or the outer closed contour to obtain a section of contour. Among them, the length of a section of contour can be 5 or 7. When it is 5, 2 contour pixels in the neighborhood on both sides are obtained except for the center. When it is 7, 3 contour pixels in the neighborhood on both sides are obtained except for the center;

[0064] Subtracting the (i + 1)-th contour pixel from the i-th contour pixel in a section of contour to obtain the i-th vector, where i is a positive integer;

[0065] Calculating the angle between adjacent vectors;

[0066] Taking the variance of the angles of a section of contour as the linearity of the contour pixel at the center.

[0067] The i-th vector is: v i =(x i+1 -x i , y i+1 -y i ), where v i is the i-th vector, x i is the abscissa of the i-th contour pixel, y i is the ordinate of the i-th contour pixel, xi+1 is the abscissa of the (i + 1)-th contour pixel point, y i+1 is the ordinate of the (i + 1)-th contour pixel point. The formula for calculating the included angle is: , where θ i is the i-th included angle, v i is the i-th vector, v i+1 is the (i + 1)-th vector, arccos is the inverse cosine, i is a positive integer, and | | is the modulus operation.

[0068] In the present invention, the inner closed contour and the outer closed contour are extracted. In order to evaluate the linearity of each pixel point, taking this pixel point as the center, multiple contour pixel points within the neighborhood range are extracted to form a section of contour. The linearity of a section of contour is used as the linearity of the pixel point at the center. By calculating the included angle between adjacent vectors and using the variance of the included angle as the linearity, the linear degree of the contour in the local area is quantified. The variance can reflect the dispersion degree of the included angle between adjacent vectors. A small variance indicates a high linear degree of the contour in this local area and a relatively smooth trend; a large variance indicates a large bending change of the contour in this area, which can accurately reflect the linear feature situation of the contour.

[0069] In this embodiment, during the process of extracting the contour by using the internal hollowing method, due to the gradual change characteristic of the pixel values at the edge, the contour line is relatively wide in the width dimension. To determine the inner and outer edge contour pixel points of the closed contour, the specific method is as follows: Traverse from the internal area enclosed by the closed contour. For any pixel point with a non-zero pixel value, if there is a pixel point with a zero pixel value in its 3×3 neighborhood centered on it, then this central pixel point is identified as the inner edge contour pixel point; similarly, traverse from the external area of the closed contour. When there is a pixel point with a zero pixel value in the 3×3 neighborhood centered on a pixel point with a non-zero pixel value, this central pixel point is determined as the outer edge contour pixel point, as shown in the schematic diagram of Figure 2 . For the convenience of observation, Figure 2 the area with a pixel value of 0 in Figure 2 is represented by white, and the black area in

[0070] A closed contour refers to a region in an image composed of a series of continuous pixel points that are connected end to end. In this embodiment, the process of labeling the closed contour includes: On the contour map, extract the single-connected region formed by each pixel value of 0 (such as the internal white area shown in Figure 2 ), and label the outer contour of the single-connected region as the closed contour.

[0071] In this embodiment, the process of obtaining the contour linear ratio vector:

[0072] Arrange the linearities of the inner contour pixel points on the closed contour from large to small to obtain the inner linearity sequence;

[0073] Arrange the linearity of the outer contour pixel points on the closed contour from large to small to obtain the outer linearity sequence;

[0074] Divide the inner linearity sequence into N segments, and take the average value of the inner linearity of each segment to obtain the inner average vector, where N is a positive integer;

[0075] Divide the outer linearity sequence into N segments, and take the average value of the outer linearity of each segment to obtain the outer average vector;

[0076] Divide the outer average vector by the inner average vector bit by bit to obtain the contour linear ratio vector.

[0077] In this embodiment, N is taken as 3, and both the inner linearity sequence and the outer linearity sequence are divided into 3 segments of data. The specific size of N can be adjusted. Arrange the linearity from large to small to achieve partition comparison. Divide the sequence into N segments. On the one hand, it reduces the amount of data and the computational complexity; on the other hand, the averaging operation can weaken the influence of noise to a certain extent, making the obtained feature vector more stable and representative.

[0078] The present invention obtains the contour linear ratio vector by dividing the outer average vector by the inner average vector bit by bit, further exploring the differential relationship between the inner and outer contour linearities.

[0079] In this embodiment, the process of finding multiple groups of matching closed contours includes:

[0080] Randomly select a contour map from the contour group maps at consecutive multiple moments, and calculate the similarity between the contour linear ratio vector of the nth closed contour on this contour map and the contour linear ratio vectors of each closed contour on each of the remaining contour maps. The initial value of n is 1;

[0081] Take the maximum similarity among all the similarities corresponding to the remaining one contour map;

[0082] When the maximum similarities on all the remaining contour maps are greater than the similarity threshold, classify the corresponding closed contours on each contour map into a group of matching closed contours;

[0083] Increment n by 1 and repeat the matching process until all the closed contours on the randomly selected contour map are traversed to obtain multiple groups of matching closed contours.

[0084] In the present invention, any one contour map is taken as a reference contour map, and the contour linear ratio vectors of each closed contour in the reference contour map are calculated for similarity with the contour linear ratio vectors of each closed contour on other contour maps. Since there are multiple closed contours on each of the other contour maps, for one closed contour in the reference contour map, there will be multiple similarities corresponding to each of the remaining contour maps. The maximum similarity is selected, and it is determined whether all the maximum similarities corresponding to the remaining contour maps are greater than the similarity threshold. If so, the closed contours corresponding to the respective maximum similarities are a group of matching closed contours, indicating that the contour shape has always existed in the images at each consecutive moment. If not, the contour shape does not always exist at each consecutive moment and may be an area in the background.

[0085] In the present invention, only when the maximum similarities on all the remaining contour maps are greater than the threshold, the corresponding closed contours are grouped into a group of matching closed contours, which can effectively filter out the contours with low similarities, avoid mis-matching irrelevant contours, further improve the accuracy of matching, and ensure that the contours in the finally obtained group of matching closed contours truly belong to the same target.

[0086] In this embodiment, the similarity is the cosine similarity, and the similarity threshold is between 0.5 and 0.9. The specific threshold can be adjusted in real time according to experiments or experience.

[0087] In this embodiment, the process of obtaining the matching contour binary map includes: setting the pixel value of the pixel points belonging to the matching closed contours in the contour map to 1, and setting the pixel values of other pixel points to 0 to obtain the matching contour binary map.

[0088] In this embodiment, the process of constructing the inner confidence feature map and the outer confidence feature map includes:

[0089] According to the linearity of the inner contour pixel points of the matching closed contours in the contour map, calculate the linear confidence value of the inner contour pixel points, replace the original pixel value with the linear confidence value, and set the pixel values of other pixel points on the contour map to 0 to obtain the inner confidence feature map;

[0090] According to the linearity of the outer contour pixel points of the matching closed contours in the contour map, calculate the linear confidence value of the outer contour pixel points; replace the original pixel value with the linear confidence value, and set the pixel values of other pixel points on the contour map to 0 to obtain the outer confidence feature map.

[0091] In this embodiment, the process of calculating the linear confidence value is: subtract the linearity of the contour pixel points from the maximum linearity to obtain the linearity difference, and use the ratio of the linearity difference to the maximum linearity as the linear confidence value.

[0092] The present invention highlights the inner and outer linear features of the matching closed contour by calculating the linear confidence values of the inner and outer contour pixel points, replacing the original pixel values, and setting other pixel points to 0.

[0093] The present invention uses the ratio of the difference between the maximum linearity and the linearity of the contour pixel points to the maximum linearity as the linear confidence value, which reflects the reliability of the linear features of the contour pixel points. The higher the linear confidence value, the more prominent and expected the linear features of the contour at the position of the pixel point.

[0094] As Figure 3 shown, the three-channel convolutional neural network includes: a first convolutional block, a second convolutional block, a third convolutional block, an adder A1, an adder A2, a first residual neural network, a second residual neural network, a Concat layer, a CNN network, and a fully connected layer;

[0095] The input end of the first convolutional block is used to input the inner confidence feature map; the input end of the second convolutional block is used to input the matching contour binary map; the input end of the third convolutional block is used to input the outer confidence feature map;

[0096] The first input end of adder A1 is connected to the output end of the first convolutional block, and its second input end is connected to the output end of the second convolutional block; the first input end of adder A2 is connected to the output end of the third convolutional block, and its second input end is connected to the output end of the second convolutional block; the output end of adder A1 is connected to the input end of the first residual neural network; the output end of adder A2 is connected to the input end of the second residual neural network; the input ends of the Concat layer are respectively connected to the output ends of the first residual neural network and the second residual neural network, and its output end is connected to the input end of the CNN network; the input end of the fully connected layer is connected to the output end of the CNN network, and its output end serves as the output end of the three-channel convolutional neural network.

[0097] In this embodiment, both the first residual neural network and the second residual neural network include: a first convolutional layer, a first ReLU layer, a second convolutional layer, an adder A3, and a second ReLU layer, as Figure 4 shown.

[0098] In this embodiment, the first convolutional block, the second convolutional block, and the third convolutional block all include: a convolutional layer, an activation function layer, and a normalization layer, and the convolutional layers in the first convolutional block, the second convolutional block, and the third convolutional block all use 3×3 convolutional kernels.

[0099] The present invention integrates multi-source features by respectively processing and fusing the inner confidence feature map, the matching contour binary map, and the outer confidence feature map through convolutional blocks, significantly improving the accuracy of target recognition. At the same time, a residual neural network is introduced to effectively overcome the problem of gradient disappearance in network training. Then, the two-way features are concatenated through a Concat layer, and through the deep processing of the CNN and the fully connected layer, high-precision recognition of unmanned aerial vehicles in the air environment is achieved.

[0100] The three images input by the three-channel convolutional neural network are from the same contour map.

[0101] The specific convolutional layer of the three-channel convolutional neural network of the present invention has significantly fewer layers compared to the existing YOLO neural network for target recognition, with significantly fewer parameters and lower computational overhead.

[0102] In this embodiment, when the classification of the matched closed contour is an unmanned aerial vehicle, the corresponding group of matched closed contours is extracted, and the area ratio is calculated for each matched closed contour. When any area ratio is greater than the area ratio threshold, there is a risk of collision, and an alarm is processed. Alternatively, the area change speed of the group of matched closed contours can also be calculated. When the area change speed is greater than the area change speed threshold, there is a risk of collision, and an alarm is processed. In this embodiment, the area is represented by the number of pixel points occupied by the region. The area change speed threshold and the area ratio threshold are specifically set according to requirements or experiments.

[0103] In this embodiment, in the group of matched closed contours, the number of pixel points occupied by the latest matched closed contour region is subtracted from the number of pixel points occupied by the starting matched closed contour region to obtain the area difference, and the area difference is divided by the interval time to obtain the area change speed.

[0104] The present invention extracts the contours of the environmental image and obtains the closed contours on the contour map, realizing the separation of the target from the background. Then, the linearity of the inner contour pixel points and the outer contour pixel points of each closed contour is obtained to reflect the edge features of the closed contour.

[0105] Describing the closed contour solely relying on the linearity of the inner or outer contour pixel points is prone to interference from environmental factors such as light changes and background noise. By calculating the ratio of the two, the influence of these interference factors can be offset to a certain extent. Because when the light or background changes, the linearity of the inner and outer contour pixel points will be affected simultaneously, but the ratio between them is relatively stable. At the same time, for environmental images taken at different times, the size of the unmanned aerial vehicle in the environmental image is different. Using the ratio of the two is not affected by the size, and the contour linear ratio vector will remain stable.

[0106] By comparing the similarity of the contour linear ratio vectors of the closed contours on the contour maps at different times, the present invention can determine which closed contours belong to the imaging results of the same unmanned aerial vehicle (UAV) at different times. Thus, the corresponding closed contours belonging to the same area are grouped together to extract the contours suspected to be UAVs on each contour map, further excavating the UAV contours and eliminating the interference of other contours.

[0107] The present invention then performs binarization processing on the matching closed contours on the contour map to obtain a matching contour binary map, making the UAV contours prominent. Then, according to the linearity of the inner contour pixel points and the outer contour pixel points of the matching closed contours, a linear confidence value is obtained, which can further quantify the characteristics of the UAV contours. Since the UAV contours are relatively regular, the linear confidence value can well reflect the characteristics of its contours.

[0108] The present invention uses a three-channel convolutional neural network to process the matching contour binary map, the inner confidence feature map, and the outer confidence feature map to obtain the classification of the matching closed contours, realizing the integration of the three aspects of contour features of the suspected UAV, achieving accurate classification of the matching closed contours, and improving the UAV recognition accuracy.

[0109] The matching contour binary map provides the contour information of the target, while the inner and outer confidence feature maps respectively supplement from the perspective of the internal and external linear features, enriching the input feature dimension of the network. Compared with the single input feature of the lightweight neural network, this multi-channel input method enables the network to learn more comprehensive and in-depth feature representations, improving the UAV recognition accuracy, and thus improving the prediction accuracy of UAV air traffic collisions.

[0110] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for predicting UAV air traffic collision, characterized in that: The following steps are involved: The camera carried by the UAV is used to collect the environment image in the forward direction, and the contour is extracted to obtain a contour map, and the closed contours on the contour map are marked; Obtain linearity for inner contour pixels and outer contour pixels on each closed contour; According to the ratio of the inner contour pixel points and the outer contour pixel points of each closed contour in terms of linearity, a contour linear ratio vector is obtained; Calculate the similarity of each closed contour on the contour graph at each moment on the contour linear ratio vector, and find multiple groups of matching closed contours; Binarization processing is performed on the matching closed contour on the contour map to obtain a matching contour binary map; According to the linearity of the inner contour pixel points and the outer contour pixel points of the matching closed contour of the contour map, a linear confidence value is obtained, and an inner confidence feature map and an outer confidence feature map are constructed; A three-channel convolutional neural network is used to process the matching contour binary map, the inner confidence feature map and the outer confidence feature map to obtain the classification of the matching closed contour; When the matched closed contours are classified as drones, it is determined whether there is a collision risk based on the set of matched closed contours.

2. The method for predicting UAV air traffic collision according to claim 1, characterized in that: The process of obtaining linearity includes: Extracting contour pixel points of the inner edge and contour pixel points of the outer edge of the closed contour to obtain an inner closed contour and an outer closed contour; Taking each contour pixel point in the inner closed contour or the outer closed contour as the center, multiple contour pixel points in the neighborhood are taken to obtain a contour segment; In a contour segment, subtract the i+1th contour pixel from the ith contour pixel to obtain the ith vector, where i is a positive integer; Calculate the angle between adjacent vectors; The variance of each angle of a contour is taken as the linearity of the contour pixel at the center.

3. The method for predicting UAV air traffic collision according to claim 2, characterized in that: The formula for calculating the angle is: , where θ i is the i-th angle, v i is the i-th vector, v i+1 is the i+1th vector, arccos is the inverse cosine, i is a positive integer, and || is the modulo operation.

4. The method for predicting UAV air traffic collision according to claim 1, characterized in that: The process of obtaining the contour linear ratio vector: Arrange the linearity of the inner contour pixel points on the closed contour from large to small to obtain the inner linearity sequence; Arrange the linearity of the outer contour pixel points on the closed contour from large to small to obtain an outer linearity sequence; The inner linearity sequence is divided into N segments, and the inner linearity of each segment is averaged to obtain the inner mean vector, where N is a positive integer; Divide the outer linearity sequence into N segments, take the average of the outer linearity of each segment, and obtain the outer mean vector; Divide the outer mean vector by the inner mean vector bitwise to obtain the contour linear ratio vector.

5. The method for predicting UAV air traffic collision according to claim 1, characterized in that: The process of finding multiple sets of matching closed contours includes: Randomly select a contour image from the contour group images at multiple consecutive moments, and calculate the similarity between the contour linear ratio vector of the nth closed contour on the contour image and the contour linear ratio vector of each closed contour on each of the remaining contour images, where the initial value of n is 1; Take the maximum similarity among all similarities corresponding to the remaining contour image; When the maximum similarities on all remaining contour images are greater than the similarity threshold, the corresponding closed contours on each contour image are grouped into a group of matching closed contours; Add 1 to n and repeat the matching process until all closed contours on any selected contour map are traversed, and multiple groups of matching closed contours are obtained.

6. The method for predicting UAV air traffic collision according to claim 1, characterized in that: The process of obtaining the matching contour binary image includes: setting the pixel values ​​of the pixel points belonging to the matching closed contour in the contour image to 1, and setting the pixel values ​​of other pixel points to 0, to obtain the matching contour binary image.

7. The method for predicting UAV air traffic collision according to claim 1, characterized in that: The process of constructing the inner confidence feature map and the outer confidence feature map includes: According to the linearity of the inner contour pixel point of the matching closed contour of the contour map, the linear confidence value of the inner contour pixel point is calculated, the linear confidence value is used to replace the original pixel value, and the pixel values ​​of other pixels on the contour map are set to 0 to obtain the inner confidence feature map; According to the linearity of the outer contour pixel points of the matching closed contour of the contour map, the linear confidence value of the outer contour pixel points is calculated; the linear confidence value replaces the original pixel value, and the pixel values ​​of other pixels on the contour map are set to 0 to obtain the outer confidence feature map.

8. The method for predicting UAV air traffic collision according to claim 7, characterized in that: The process of calculating the linear confidence value is as follows: the linearity difference is obtained by subtracting the linearity of the contour pixel point from the maximum linearity, and the ratio of the linearity difference to the maximum linearity is taken as the linear confidence value.

9. The method for predicting UAV air traffic collision according to claim 1, characterized in that: The three-channel convolutional neural network includes: a first convolution block, a second convolution block, a third convolution block, an adder A1, an adder A2, a first residual neural network, a second residual neural network, a Concat layer, a CNN network and a fully connected layer; The input end of the first convolution block is used to input the inner confidence feature map; the input end of the second convolution block is used to input the matching contour binary map; the input end of the third convolution block is used to input the outer confidence feature map; The first input end of adder A1 is connected to the output end of the first convolution block, and the second input end thereof is connected to the output end of the second convolution block; the first input end of adder A2 is connected to the output end of the third convolution block, and the second input end thereof is connected to the output end of the second convolution block; the output end of adder A1 is connected to the input end of the first residual neural network; the output end of adder A2 is connected to the input end of the second residual neural network; the input end of the Concat layer is respectively connected to the output end of the first residual neural network and the output end of the second residual neural network, and the output end thereof is connected to the input end of the CNN network; the input end of the fully connected layer is connected to the output end of the CNN network, and the output end thereof serves as the output end of the three-channel convolutional neural network.

10. The method for predicting UAV air traffic collision according to claim 1, characterized in that: When the matched closed contour is classified as a drone, the corresponding group of matched closed contours is extracted, and the area ratio of each matched closed contour is calculated. When any area ratio is greater than the area ratio threshold, there is a collision risk.

Images