Low altitude flight safety homing system

Abstract

The application relates to the field of aviation safety, in particular to a low-altitude flight safety coordination system, which comprises a flight airspace demarcation module, an image intelligent recognition module, a position determination module, a collision risk judgment module, a route planning module and a management database. The system constructs an identification model through a convolutional neural network, acquires a flight airspace image, recognizes mobile objects and fixed obstacles in the flight airspace, marks the positions of the mobile objects and the fixed obstacles in the flight airspace image, predicts the motion trajectory of the mobile objects in the flight airspace image, judges whether the aircraft has a collision risk, determines the collision risk position of the aircraft, outputs a safe flight path segment by using a heuristic search algorithm, covers the whole process from environment perception to risk response, and greatly reduces the possibility of collision accidents of the aircraft.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of aviation safety, in particular to a low-altitude flight safety coordination system. BACKGROUND

[0002] With the continuous development of aviation technology, the field of low-altitude flight is showing a flourishing scene. Low-altitude aircraft, including small civilian aircraft, helicopters, and a large number of unmanned aerial vehicles, are widely used in many fields, such as aerial photography, agricultural plant protection, logistics distribution, etc. However, the low-altitude flight environment is complex and changeable, and there are many uncertain factors, such as fixed obstacles in the city, such as high-rise buildings, power lines, and mobile objects such as birds and other aircraft, which bring great safety challenges to low-altitude flight.

[0003] At present, the existing low-altitude flight safety technology has obvious limitations. For example, the existing Chinese patent with application number 202411944390.8 discloses a low-altitude aircraft real-time monitoring and conflict warning system. This scheme collects multi-source data of low-altitude aircraft to obtain real-time flight data, transmits the real-time flight data to a low-altitude aircraft monitoring and warning platform for data fusion processing, and outputs real-time monitoring data of low-altitude aircraft. When it is determined that there is a conflict possibility of low-altitude aircraft, the flight conflict point is predicted, and conflict warning data is generated. The flight route of the low-altitude aircraft is dynamically controlled to resolve the conflict control of the low-altitude aircraft at the flight conflict point, which effectively realizes the flight monitoring of the low-altitude aircraft and ensures the flight safety of the low-altitude aircraft.

[0004] The existing Chinese patent with application number 202411827339.9 discloses a low-altitude aircraft collision avoidance control method, device, equipment and storage medium. This scheme obtains the first position information of the target aircraft at each planned flight trajectory point, the second position information of the air traffic control aircraft, and the third position information of the non-air traffic control aircraft, and performs collision detection and warning at each planned flight trajectory point in the planned flight trajectory. At the same time, by estimating the predicted position of each evading aircraft, the planned flight trajectory of the target aircraft is adjusted, which improves the collision avoidance effectiveness, flight safety and flight trajectory planning rationality of the low-altitude aircraft.

[0005] The above scheme has the following disadvantages: 1. The existing technology only focuses on the conflict between aircraft, and does not have corresponding detection and response strategies for dynamic obstacles in the environment. In the low-altitude flight scene, the randomness of dynamic obstacles is high, and the system does not design a special monitoring, warning and obstacle avoidance process for them, which may cause the aircraft to fail to avoid such obstacles in time.

[0006] II. The prior art mainly optimizes the solution based on preset distance threshold and constraint conditions, lacks flexibility and intelligence, and may not quickly find an effective path in complex situations, requiring targeted search for a feasible path to improve search efficiency. SUMMARY

[0007] To overcome the shortcomings in the background art, embodiments of the present application provide a low-altitude flight safety coordination system that can effectively solve the problems involved in the above background art.

[0008] The object of the present application can be achieved by the following technical solutions: The present application provides a low-altitude flight safety coordination system, comprising: a flight airspace delineation module for obtaining flight airspace data and delineating the boundary range of the flight airspace.

[0009] An image intelligent recognition module is used to construct a recognition model based on a convolutional neural network, obtain flight airspace images, and recognize moving objects and fixed obstacles within the flight airspace.

[0010] A position determination module is used to mark the positions of moving objects and fixed obstacles in the flight airspace images and predict the motion trajectories of the moving objects in the flight airspace images.

[0011] A collision risk judgment module is used to determine whether fixed obstacles pose a collision risk and to determine whether moving objects pose a collision risk in association with timestamp information.

[0012] A route planning module is used to determine the flight path segments affected by the aircraft and to output safe flight path segments through a heuristic search algorithm.

[0013] A management database is used to store aircraft flight plans, flight airspace data, image sample data, aircraft flight path data, fixed obstacle boundary line segment set data, moving object motion trajectory line segment collection data, and safe flight path segment data.

[0014] Preferably, the specific analysis method of the flight airspace delineation module is to obtain the latitude and longitude coordinates of the flight start point and end point of the aircraft, the flight altitude, as flight airspace data.

[0015] The vertical height range and horizontal boundary range of the flight airspace corresponding to the flight airspace data are obtained to determine the boundary range of the flight airspace.

[0016] Preferably, the specific method of constructing a recognition model based on a convolutional neural network is to collect image samples containing various scenes and various objects, construct a recognition model based on a convolutional neural network, divide the image samples into a training set and a test set, and input the training set into the recognition model for training until the loss function is stable.

[0017] inputting the test set into the recognition model until model parameters of the recognition model meet preset requirements, and training is completed to obtain the trained recognition model.

[0018] Preferably, the specific analysis method of the position determination module is: S1. inputting the flight airspace image into the trained recognition model to identify the moving object and the fixed obstacle in the flight airspace image respectively.

[0019] S2. marking the positions of the moving object and the fixed obstacle in the flight airspace image in the form of a bounding box.

[0020] S3. obtaining the motion information of the moving object based on the change of the position marking of the moving object, and predicting the motion trajectory of the moving object in the flight airspace image.

[0021] Preferably, the specific method of step S2 is: S21. for the fixed obstacle, obtaining the edge pixel position of the fixed obstacle in the flight airspace image, taking the minimum circumscribed rectangle corresponding to the edge pixel position as the bounding box, and marking the position of the fixed obstacle in the flight airspace image.

[0022] S22. obtaining each frame image of the moving object in the flight airspace, extracting feature points in adjacent frame images, matching the feature points between adjacent frame images, determining the coordinate position of the moving object in each frame image according to the coordinate position of the matched feature points in the flight airspace image, and completing the position marking of the moving object.

[0023] Preferably, step S22 comprises: S221. obtaining each frame image recording the motion process of the moving object in the flight airspace, taking adjacent frames as a group, extracting feature points in adjacent frame images, taking the feature points as the center, selecting a neighborhood window of a set size, dividing a plurality of pixel point pairs from the neighborhood window, comparing the gray scale values of each pixel point pair, recording 1 if the former is greater than the latter, otherwise recording 0, combining the comparison results to form a binary string, i.e. the descriptor of each feature point.

[0024] S222. calculating the distance between the descriptor of each feature point in one frame image and the descriptor of each feature point in another frame image in each group of adjacent frames, regarding the feature point pair with a distance less than a set threshold as a matching point, i.e. the matched feature point.

[0025] S223. determining the coordinate position of the moving object in each frame image according to the coordinate position of the matched feature point in the flight airspace image, taking the minimum circumscribed rectangle of the moving object as the bounding box, and marking the position of the moving object in each frame image to complete the position marking of the moving object.

[0026] Preferably, the step S3 comprises: S31. obtaining a displacement vector of the moving object in the adjacent frame image according to the center coordinate difference of the position marker of the moving object, determining a moving direction of the moving object according to the displacement vector, and obtaining a moving state of the moving object based on the displacement vector of the moving object in the adjacent frame image and the interval time length of the image frames, so as to take the moving state as the moving information of the moving object.

[0027] S32. setting a prediction time interval, calculating the coordinate positions of the moving object in the flight airspace image at each future time according to the current position and the moving information of the moving object in the flight airspace image, and constructing a future position coordinate point set of the moving object.

[0028] S33. connecting the future position coordinate points in the future position coordinate point set of the moving object in time sequence to obtain a moving track of the moving object in the flight airspace image.

[0029] Preferably, the specific operation method for judging whether the fixed obstacle has a collision risk comprises: regarding four edges of the minimum circumscribed rectangle of the fixed obstacle as line segments, recording the coordinate of the two endpoints of the four line segments to construct a boundary line segment set of the fixed obstacle.

[0030] The endpoint coordinate range of the flight path line segment is compared with the overall coordinate range determined by the boundary line segment set of the fixed obstacle, the part without intersection is excluded, and for the part with intersection, a geometric intersection detection algorithm is used to judge whether there is an intersecting line segment. If there is an intersecting line segment, it is determined that there is a spatial position conflict between the flight path of the aircraft and the fixed obstacle, that is, the aircraft will collide with the fixed obstacle under the current flight path.

[0031] Preferably, the specific operation method for judging whether the moving object has a collision risk comprises: obtaining the flight path of the aircraft, recording the flight path with digital coordinate points, connecting adjacent digital coordinate points in sequence to obtain a converted flight path line segment set, and dividing the moving track of the moving object in the flight airspace image into a moving track line segment set according to the connection line of adjacent coordinate points.

[0032] The endpoint coordinate range of the moving track line segment is compared with the endpoint coordinate range of the flight path line segment, the line segment pairs that cannot intersect are excluded, and a cross-standing experiment is used to judge whether there is an intersecting line segment. If it is detected that there is an intersecting line segment, it is determined that there is a potential conflict area in the spatial position between the moving object and the aircraft.

[0033] The time stamp of the moving object reaching the potential conflict area is determined based on the future position coordinate point set of the moving object and the prediction time interval, and the time stamp of the flight path of the aircraft reaching the potential conflict area is determined according to the flight plan and the flight speed of the aircraft.

[0034] The time difference between the time stamp of the mobile object reaching the potential conflict area and the time stamp of the aircraft flight path reaching the potential conflict area is obtained, and if the time difference is within a preset safety time threshold, it is determined that the aircraft will collide with the mobile object under the current flight path.

[0035] Preferably, the specific operation method of the route planning module is: A1. Centering on the aircraft collision risk position, the conflict influence range is determined in combination with the current flight direction and speed of the aircraft, and the segment of the current flight path of the aircraft in the conflict range is identified as the affected flight path segment.

[0036] A2. The search starting point is set as the starting position of the affected flight path segment, and the end point is the position farthest from the affected flight path segment in the normal airway part.

[0037] A3. The heuristic function of the algorithm is set, and the initialized heuristic search algorithm is used for searching within the range of the affected flight path segment, and the nodes are expanded according to the cost of each search node evaluated by the heuristic function.

[0038] A4. The searched path is verified, if the path verification is passed, the path is output as a new safe flight path segment, if not, return to A3 to continue searching until a safe flight path segment that passes the path verification is found.

[0039] Compared with the prior art, the present application has the following beneficial effects: (1) The present application constructs an identification model based on a convolutional neural network, obtains flight airspace images, identifies mobile objects and fixed obstacles in the flight airspace, and the convolutional neural network has high accuracy in image recognition, which can more accurately detect these potential dangerous elements, and helps to better plan the flight route and prepare for various situations in advance.

[0040] (2) The present application marks the positions of the mobile objects and the fixed obstacles in the flight airspace images, predicts the motion trajectory of the mobile objects in the flight airspace images, and further judges whether the aircraft has a collision risk, so that the dynamic trajectory of the mobile object is considered in the range, making the flight route more reasonable and safe, and avoiding the mobile object that may be at the same position as the aircraft at a certain moment in the future, improving the efficiency and safety of the entire flight process.

[0041] (3) The present application determines the aircraft collision risk position and outputs a safe flight path segment using a heuristic search algorithm, which can quickly search for a safe flight path segment in a complex airspace environment when facing a collision risk, without searching all possible paths in the entire airspace, but according to certain heuristic information to find feasible paths, greatly improving the search efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the description of the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0043] Figure 1 The system module connection diagram of the present application.

[0044] Figure 2 The method flow chart of the position determination module. Figure 1

[0045] Figure 3 The method flow chart of the route planning module. Figure 1 DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort belong to the scope of protection of the present application.

[0047] Please refer to Figure 1 As shown in the figure, a low-altitude flight safety coordination system includes a flight airspace delineation module, an image intelligent recognition module, a position determination module, a collision risk judgment module, a route planning module, and a management database.

[0048] The management database is connected with the flight airspace delineation module, the image intelligent recognition module, the position determination module, the collision risk judgment module, and the route planning module. The collision risk judgment module is connected with the position determination module and the route planning module. The image intelligent recognition module is connected with the flight airspace delineation module and the position determination module.

[0049] The flight airspace delineation module is used to obtain flight airspace data and delineate the boundary range of the flight airspace.

[0050] The specific analysis method of the flight airspace delineation module is to obtain the latitude and longitude coordinates of the flight starting point and ending point of the aircraft, the flight height, and use them as flight airspace data.

[0051] ​​The vertical height range and the horizontal boundary range of the flight airspace corresponding to the flight airspace data are acquired, the boundary range of the flight airspace is determined, the movable range of the aircraft in the vertical height and the horizontal direction is determined, the accurate spatial definition can determine the position of the aircraft in the defined airspace when obstacle avoidance is required, and the relative positions of the surrounding obstacles can be quickly determined.

[0052] The image intelligent recognition module is configured to construct an identification model based on a convolutional neural network, acquire a flight airspace image, and identify moving objects and fixed obstacles in the flight airspace.

[0053] The specific method of constructing the identification model based on the convolutional neural network includes collecting image samples containing various scenes and various objects, constructing an identification model based on a convolutional neural network, dividing the image samples into a training set and a test set, and inputting the training set into the identification model for training until the loss function is stable.

[0054] It should be noted that the convolutional neural network includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The input layer receives image data. The convolutional layer performs convolution operation on the image by sliding different size convolution kernels to extract local features in the image. The pooling layer down-samples the feature map to reduce the amount of data and calculation while retaining the main features and preventing overfitting. The fully connected layer expands the feature map processed by convolution and pooling into a one-dimensional vector, and then performs feature fusion and classification decision through multiple fully connected layers. The output layer is set according to the specific identification task.

[0055] It should be noted that the specific operation process until the loss function is stable includes: in each training iteration, a batch of image data in the training set is input into the identification model, the model makes a prediction according to the current parameters to obtain a prediction result, the loss value between the prediction result and the true label is calculated by a loss function, the gradient of the loss function with respect to the model parameters is calculated based on the calculated loss value, the model parameters are adjusted according to the direction and size of the gradient, and as the training proceeds, the value of the loss function changes constantly. When the change amplitude of the loss function value in continuous multiple training iterations meets the requirements, for example, the difference between the loss values of adjacent two iterations is less than a pre-set threshold, or the loss value fluctuates within a certain range and no longer decreases obviously, it can be considered that the loss function has reached a stable state.

[0056] The test set is input into the identification model until the model parameters of the identification model meet the pre-set requirements, and the training is completed to obtain the trained identification model.

[0057] The position determination module is configured to mark the positions of the moving objects and the fixed obstacles in the flight airspace image, and predict the motion trajectory of the moving objects in the flight airspace image.

[0058] Referring to Figure 2 As shown, the specific analysis method of the position determination module is: S1. inputting the flight airspace image into the trained recognition model to identify the moving object and the fixed obstacle in the flight airspace image respectively; the object with moving characteristics and the relatively static fixed obstacle can be accurately distinguished according to the feature difference of the object, so as to avoid collision accidents.

[0059] S2. marking the position of the moving object and the fixed obstacle in the form of a bounding box on the flight airspace image; by marking the position of the moving object and the fixed obstacle in the form of a bounding box on the same image, the spatial relationship between them can be intuitively presented.

[0060] S3. obtaining the motion information of the moving object based on the change of the position marking of the moving object, and predicting the motion trajectory of the moving object in the flight airspace image; flight decisions can be made in advance according to the predicted motion trajectory, and the safety and efficiency of flight can be improved.

[0061] The specific method of step S2 is: S21. for the fixed obstacle, obtaining the edge pixel position of the fixed obstacle in the flight airspace image, taking the minimum circumscribed rectangle corresponding to the edge pixel position as the bounding box, and marking the position of the fixed obstacle in the flight airspace image; the minimum circumscribed rectangle can closely fit the contour of the fixed obstacle, and the position area of the fixed obstacle in the flight airspace image is accurately defined from a macro perspective.

[0062] It should be noted that the specific method of marking the position of the fixed obstacle is: obtaining the bounding box of the fixed obstacle, determining the four vertex coordinates of the bounding box, corresponding the vertex coordinates to the pixel coordinate system of the flight airspace image, and circumscribing the position of the fixed obstacle in the flight airspace image to complete the position marking of the fixed obstacle.

[0063] S22. obtaining each frame image of the moving object in the flight airspace, extracting feature points in adjacent frame images, matching the feature points between adjacent frame images, determining the coordinate position of the moving object in each frame image according to the coordinate position of the matched feature points in the flight airspace image, and completing the position marking of the moving object; the visual feature information of the object itself is directly used for positioning, and the coordinate position of the moving object in each frame image can be accurately determined.

[0064] It should be noted that the specific method of extracting feature points in the adjacent frame image is: converting the frames of images into gray scale images, using SIFT algorithm to construct Gaussian difference pyramid, selecting a set number of images as the first layer images of the first group of Gaussian pyramid, starting from the first layer image, performing Gaussian blur operation of different scales to obtain other layer images of the group, wherein the first layer image of each group is obtained by downsampling (reducing the image size by half) the third layer image of the previous group, subtracting each group of adjacent two layer images pixel by pixel to obtain the corresponding Gaussian difference image, and combining the Gaussian difference images to form a Gaussian difference pyramid.

[0065] For each group and layer image in the Gaussian difference pyramid, compare each pixel point with the 26 neighboring pixel points (including 8 neighboring pixel points of the same layer, 9 neighboring pixel points of the previous layer and 9 neighboring pixel points of the next layer), if the value of a certain pixel point is greater than or less than the value of all neighboring pixel points, then the pixel point is a local extreme point, and the local extreme points with a contrast less than a set threshold are filtered out to obtain each feature point.

[0066] It should be noted that the specific method of matching feature points is: traversing each feature point in the first frame image, calculating the distance between the feature descriptor of the feature point and the feature descriptor of each feature point in the second frame image through Euclidean distance, and selecting the feature point with the smallest distance as the matching point, thereby traversing all feature points in the first frame image, and completing the matching process of all feature points between the first frame image and the second frame image.

[0067] The step S22 includes: S221. Obtain each frame image recording the movement process of the moving object in the flight airspace, take adjacent frames as a group, extract feature points in the adjacent frame images, select a neighborhood window of a set size as the center of the feature points, divide a plurality of pixel point pairs from the neighborhood window, compare the gray scale values of each pixel point pair, if the former is greater than the latter, record the comparison result as 1, otherwise record the comparison result as 0, combine the comparison results to form a binary string, i.e. the descriptor of each feature point; in a complex flight airspace scene, the feature points of different moving objects may have similar coordinate positions, but the descriptors can be more accurately identified and distinguished.

[0068] S222. Calculate the distance between the descriptors of each feature point in one frame image of each group of adjacent frames and the descriptors of each feature point in another frame image, and regard the feature points with a distance less than a set threshold as matching points, i.e. the matched feature points; in actual images, there may be many similar but not truly corresponding feature points, and through the threshold limit, the accuracy and reliability of matching can be improved.

[0069] It should be noted that the descriptors of two feature points are 、 wherein denotes the index of the vector element of the feature point descriptor, , the Euclidean distance .

[0070] S223. According to the coordinate position of the matching point in the flight airspace image, the coordinate position of the moving object in each frame image is determined, the minimum circumscribed rectangle of the moving object is taken as the bounding box, and the position of the moving object in each frame image is circled to complete the position marking of the moving object; a unified position marking method is provided for the analysis of the entire flight airspace image. Whether in the analysis of a single frame image or in the sequence analysis of multiple frame images, such a unified marking method facilitates data integration.

[0071] The step S3 includes: S31. According to the center coordinate difference of the position marking of the moving object, the displacement vector of the moving object in adjacent frame images is obtained, the motion direction of the moving object is determined according to the displacement vector, and the motion state of the moving object is obtained based on the displacement vector of the moving object in adjacent frame images and the image frame interval duration, which is taken as the motion information of the moving object; the displacement vector, the motion direction and the motion state are integrated into the motion information of the moving object, which can comprehensively describe the motion of the moving object in the flight airspace.

[0072] It should be noted that the center coordinates of the marking are taken to represent the position of the moving object in the image. In the first frame image, the center coordinates of the position marking of the moving object are , and in the frame image, the center coordinates are , the displacement vector is used to describe the change of the position of the object between two adjacent frame images. By calculating the difference between the center coordinates of adjacent frames, the displacement components of the moving object in the x-axis and y-axis directions are obtained, and the displacement vector can be represented as , the motion direction is represented by the included angle between the displacement vector and the coordinate axis, , and the image frame interval duration refers to the time interval between the collection of two adjacent frame images, denoted as .

[0073] It should be noted that according to the definition of vector, the vector from point to point can be obtained by subtracting the starting point coordinates from the ending point coordinates, so the x-axis component of the displacement vector is , the x-axis component is , i.e. ​, the displacement vector and the x-axis , the tangent value of which is equal to the ratio of the y-axis component of the displacement vector to the x-axis component, that is .

[0074] S32. Set a prediction time interval, calculate the coordinate positions of the mobile object in the flight airspace image at each future time according to the current position and motion information of the mobile object in the flight airspace image, and form a future position coordinate point set of the mobile object; the expected position distribution of the mobile object in the future period of time can be completely depicted, and the current position and the predicted position of other mobile objects are compared and analyzed, so as to realize collision warning.

[0075] It should be noted that the motion state of the mobile object includes uniform linear motion and uniform acceleration, uniform deceleration.

[0076] The moving speed of the mobile object in each adjacent frame image of the mobile object is obtained by dividing the displacement of the adjacent frame image of the mobile object by the time length of the image frame interval. If the moving speed of the mobile object in each adjacent frame image of the mobile object is equal, it is judged that the mobile object belongs to uniform linear motion. If the moving speed of the mobile object in each adjacent frame image of the mobile object has no change rule, the average value is taken as the moving speed of the mobile object.

[0077] It should be noted that the current position coordinate of the mobile object in the flight airspace image is , the moving speed is , and the angle between the speed direction and the x-axis is . Since the mobile object moves at a uniform speed in a straight line, the displacement of the mobile object in the x-axis direction at each future time is , and the displacement in the y-axis direction is , , the number of the th future time, , the coordinate position of the mobile object in the flight airspace image at each future time is , thereby forming a future position coordinate point set of the mobile object.

[0078] If the moving speed of the mobile object in each adjacent frame image of the mobile object increases or decreases uniformly with time, it is judged that the mobile object belongs to uniform acceleration or uniform deceleration.

[0079] It should be noted that the initial moving speed is set as , the acceleration is , the angle between the acceleration direction and the x-axis is , the displacement of the mobile object in the x-axis direction at each future time is , and the displacement in the y-axis direction is , the coordinate positions of the moving object in the flight airspace image at each future moment are , thereby forming a future position coordinate point set of the moving object.

[0080] S33. Connecting the future position coordinate points in the future position coordinate point set of the moving object in time sequence to obtain a motion trajectory of the moving object in the flight airspace image.

[0081] The collision risk judgment module is configured to judge whether the fixed obstacle has a collision risk, and is associated with the timestamp information to judge whether the moving object has a collision risk.

[0082] The specific operation method for judging whether the fixed obstacle has a collision risk is: regarding four edges of the minimum circumscribed rectangle of the fixed obstacle as line segments, recording the coordinates of two endpoints of the four line segments to form a boundary line segment set of the fixed obstacle; the boundary information of the fixed obstacle in space can be clearly determined, the coordinates of two endpoints of the four line segments are recorded to form a boundary line segment set of the fixed obstacle, so that the boundary of the fixed obstacle has a clear definition in the coordinate system, which provides basic data for subsequent judgment of the relationship with the flight path of the aircraft.

[0083] It should be noted that, in a specific embodiment, the coordinates of the top-left corner vertex of the minimum circumscribed rectangle of the fixed obstacle are , the coordinates of the bottom-right corner vertex are , the coordinates of two endpoints of four edges are determined and line segments are formed: the upper edge (horizontal direction): the coordinates of two endpoints of the line segment are , , the line segment is recorded as ; the lower edge (horizontal direction): the coordinates of two endpoints of the line segment are , , the line segment is recorded as ; the left edge (vertical direction): the coordinates of two endpoints of the line segment are , , the line segment is recorded as ; the right edge (vertical direction): the coordinates of two endpoints of the line segment are , , the line segment is recorded as .

[0084] The endpoint coordinates of the above four line segments are arranged in the form of a set to obtain a boundary line segment set of the fixed obstacle .

[0085] The end point coordinate range of the flight path segment is compared with the overall coordinate range determined by the fixed obstacle boundary segment set, the part without intersection is excluded, for the part with intersection, a geometric intersection detection algorithm is used to determine whether there is an intersection segment, if there is an intersection segment, it is determined that the aircraft flight path and the fixed obstacle exist spatial position conflict, that is, the aircraft will collide with the fixed obstacle under the current flight path; for the part with intersection, the potential area with possible collision risk is determined, and this hierarchical processing method improves the accuracy and efficiency of judging the collision risk from the overall coordinate range comparison to the local intersection detection.

[0086] It should be noted that the specific steps of the geometric intersection detection algorithm are: for two line segments, respectively denoted as line segment AB and line segment CD, wherein are the two end point coordinates of the line segment AB, are the two end point coordinates of the line segment CD, respectively calculate the vectors , , , , , , calculate the cross product , , , .

[0087] If and are of opposite signs (i.e. ) and and are of opposite signs (i.e. ), because and are of opposite signs, it means that points C and D are on the two sides of line segment AB, and are of opposite signs, which means that points A and B are on the two sides of line segment CD, satisfying the intersection condition, and line segment AB and line segment CD intersect.

[0088] The above operation is repeated for each pair of line segments to determine whether there is an intersection line segment in the entire line segment set by traversing each pair of line segments to determine whether they satisfy the intersection condition.

[0089] The specific operation method for determining whether the moving object exists collision risk is: obtaining the flight path of the aircraft, recording it with digital coordinate points, connecting adjacent digital coordinate points in turn to obtain the transformed flight path segment set, and dividing the movement trajectory of the moving object in the flight airspace image into a movement trajectory segment set according to the connection of adjacent coordinate points; accurate flight path recording helps to plan the route and avoid collision risk between aircrafts.

[0090] The end point coordinate ranges of the moving trajectory line segments and the flight path line segments are compared, impossible intersecting line segment pairs are excluded, a cross-standing experiment is used to determine whether there is an intersecting line segment, if it is detected that there is an intersecting line segment, it is determined that there is a potential conflict region in the spatial position of the moving object and the aircraft; accurately determining whether there is a potential conflict region in the spatial position of the moving object and the aircraft can early warn of possible collision danger, which is crucial for ensuring the safety of the aircraft, the moving object (such as a drone, a bird, etc.) and the like.

[0091] It should be noted that the specific method for determining whether the remaining line segment pairs intersect by using the cross-standing experiment is: the moving trajectory line segment set of the moving object is determined, assuming that the set is Each line segment is determined by two end point coordinates, that is, The flight path line segment set of the aircraft is determined, which is set as Each line segment is also determined by two end point coordinates, that is, .

[0092] For each pair of line segments , the vectors , , , , are calculated, and the cross products , , , are calculated.

[0093] If and are of opposite signs (i.e. ) and and are of opposite signs (i.e. ), then the line segments and intersect, if and , then the four points are collinear, and it is necessary to further determine whether the line segments overlap.

[0094] If and , then the point is on the line segment ; if and , then the point is on the line segment ; if and , then the point is on the line segment ; if and then the point is considered to be the intersection of the line segments .

[0095] Based on the future position coordinate point set of the moving object and the prediction time interval, the timestamp of the moving object reaching the potential conflict region is determined, and the timestamp of the aircraft flight path reaching the potential conflict region is determined according to the flight plan and flight speed of the aircraft.

[0096] The time difference between the timestamp of the moving object reaching the potential conflict region and the timestamp of the aircraft flight path reaching the potential conflict region is obtained, and if the time difference is within a pre-set safety time threshold, it is determined that the aircraft will collide with the moving object under the current flight path.

[0097] A route planning module is configured to determine the affected flight path segment of the aircraft, and output a safe flight path segment through a heuristic search algorithm.

[0098] Referring to Figure 3 , the specific operation method of the route planning module is as follows: A1. Taking the aircraft collision risk position as the center, combining the current flight direction and speed of the aircraft to determine the conflict influence range, and identifying the segment in the current flight path of the aircraft in the conflict range as the affected flight path segment; it can accurately plan how to avoid potential collision risks, instead of making large-scale and possibly unnecessary adjustments to the entire flight path.

[0099] A2. Set the search starting point as the starting position of the affected flight path segment, and the end point as the position closest to the affected flight path segment in the normal airway part; it helps to concentrate search resources and avoid searching in irrelevant airspace or flight path part, thereby improving search efficiency and finding a feasible solution faster.

[0100] A3. Set the heuristic function of the algorithm, and use the initialized heuristic search algorithm to search within the range of the affected flight path segment, and expand nodes according to the cost of each search node evaluated by the heuristic function; it makes the search result better adapt to the complex and variable actual flight situation, and finds the flight path adjustment scheme most suitable for the current situation.

[0101] It should be noted that in one specific embodiment, the current position of the aircraft is , the flight direction is , the speed is m / s, and the conflict influence range is set to a radius of m around the collision risk position .Within a circular area of ​​meters, the aircraft's current flight path consists of a series of coordinate points. Traverse the coordinate points and determine the affected flight path segment from point [point]. Time ,from Begin by moving in the opposite direction of the current flight path, and within the normal flight path (a predefined safe zone), find the closest point to the destination. Define an open list to store nodes to be expanded and a closed list to store expanded nodes. Add the search starting point to the open list, and calculate the actual cost from the starting point to the current node. A value of 0 indicates the estimated cost from the current node to the destination. for ,according to Select from the open list The node with the smallest value is selected as the current expansion node.

[0102] A4. Verify the searched path. If the path verification is successful, output the path as a new safe flight path segment. If it fails, return to A3 to continue searching until a safe flight path segment that has passed the path verification is found.

[0103] It should be noted that the specific method for path verification is as follows: Define the boundary of the safe zone in the flight environment. For each node in the searched path, draw a ray from the node in a certain direction and count the number of intersections between the ray and the boundary of the safe zone. If the number of intersections is odd, the point is inside the polygon, i.e., in the safe zone. If the number is even, it is outside. At the same time, for each line segment connecting adjacent nodes in the path, use the geometric intersection detection algorithm to determine whether each line segment intersects with a fixed obstacle. If they intersect, it is determined that there is a conflict in the path.

[0104] The management database stores aircraft flight plans, airspace data, image sample data, aircraft flight path data, fixed obstacle boundary line segment set data, moving object trajectory line segment set data, and safe flight path segment data.

[0105] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention, which are still covered within the protection scope of the present invention.

Claims

1. A low altitude flight safety co-pilot system, characterized by, The system specifically comprises the following modules: The flight airspace delineation module is configured to obtain flight airspace data and delineate the boundary range of the flight airspace. The image intelligent recognition module is configured to construct an identification model based on a convolutional neural network, obtain flight airspace images, and identify moving objects and fixed obstacles in the flight airspace. The position determination module is configured to mark the positions of the moving objects and the fixed obstacles in the flight airspace images and predict the motion trajectories of the moving objects in the flight airspace images. The collision risk judgment module is configured to determine whether the fixed obstacles have collision risks, associate timestamp information, obtain the flight path of the aircraft, record the flight path in digital coordinate points, sequentially connect adjacent digital coordinate points to obtain a set of converted flight path line segments, and divide the motion trajectories of the moving objects in the flight airspace images into a set of motion trajectory line segments according to the lines connecting adjacent coordinate points. The collision risk judgment module is configured to compare the endpoint coordinate ranges of the motion trajectory line segments and the flight path line segments, exclude impossible intersecting line segment pairs, determine whether there is an intersecting line segment by using a cross-standing experiment, and if an intersecting line segment is detected, determine that there is a potential conflict region in the spatial positions of the moving objects and the aircraft. The collision risk judgment module is configured to determine the timestamp at which the moving objects arrive at the potential conflict region based on the future position coordinate point set of the moving objects and the prediction time interval, determine the timestamp at which the flight path of the aircraft arrives at the potential conflict region based on the flight plan and the flight speed of the aircraft, and obtain the time difference between the timestamps at which the moving objects and the flight path of the aircraft arrive at the potential conflict region. The collision risk judgment module is configured to determine that the aircraft will collide with the moving objects under the current flight path if the time difference is within a pre-set safety time threshold. The route planning module is configured to determine the affected flight path segment of the aircraft and output a safe flight path segment by using a heuristic search algorithm. The management database is configured to store the flight plan of the aircraft, flight airspace data, image sample data, aircraft flight path data, fixed obstacle boundary line segment set data, moving object motion trajectory line segment set data, and safe flight path segment data. The specific analysis method of the flight airspace delineation module is as follows: Obtain the latitude and longitude coordinates and the flight height of the flight start point and the flight end point of the aircraft as the flight airspace data. Obtain the vertical height range and the horizontal boundary range of the flight airspace corresponding to the flight airspace data as the boundary range of the flight airspace.

2. A low altitude flight safety co-pilot system according to claim 1, characterized in that, The specific method for constructing the identification model based on the convolutional neural network is as follows: Collect image samples containing various scenes and various objects, construct an identification model based on a convolutional neural network, divide the image samples into a training set and a test set, input the training set into the identification model for training until the loss function is stable, input the test set into the identification model until the model parameters of the identification model meet the pre-set requirements, and complete the training to obtain the trained identification model.

3. A low altitude flight safety co-pilot system according to claim 2, wherein, The specific analysis method of the position determination module is as follows: S1. input the flight airspace images into the trained identification model to identify the moving objects and the fixed obstacles in the flight airspace images, respectively; S2. Mark the positions of the moving object and the fixed obstacle respectively on the flight airspace image in the form of a bounding box; S3. Obtain the motion information of the moving object based on the change of the position marking of the moving object, and predict the motion trajectory of the moving object in the flight airspace image.

4. A low altitude flight safety co-pilot system according to claim 3, characterized in that, The specific method of step S2 is: S21. For the fixed obstacle, obtain the edge pixel position of the fixed obstacle in the flight airspace image, and take the minimum circumscribed rectangle corresponding to the edge pixel position as the bounding box to mark the position of the fixed obstacle in the flight airspace image; S22. Obtain each frame image of the moving object in the flight airspace, extract feature points in adjacent frame images, match the feature points between adjacent frame images, determine the coordinate position of the moving object in each frame image according to the coordinate position of the matched feature points in the flight airspace image, and complete the position marking of the moving object.

5. A low altitude flight safety co-pilot system according to claim 4, characterized in that, The step S22 includes: S221. Obtain each frame image recording the motion process of the moving object in the flight airspace, take adjacent frames as a group, extract feature points in adjacent frame images, select a neighborhood window of a certain size as the center of the feature points, divide a plurality of pixel point pairs from the neighborhood window, compare the gray values of each pixel point pair, if the former is greater than the latter, record the comparison result as 1, otherwise record the comparison result as 0, combine the comparison results to form a binary string, that is, the descriptor of each feature point; S222. Calculate the distance between the descriptors of each feature point in one frame image and the descriptors of each feature point in another frame image in each group of adjacent frames, and regard the feature points with a distance less than a certain threshold as matching points, that is, the matched feature points; S223. Determine the coordinate position of the moving object in each frame image according to the coordinate position of the matched feature points in the flight airspace image, take the minimum circumscribed rectangle of the moving object as the bounding box, and enclose the position of the moving object in each frame image to complete the position marking of the moving object.

6. A low altitude flight safety co-pilot system according to claim 3, wherein, The step S3 includes: S31. Obtain the displacement vector of the moving object in adjacent frame images according to the center coordinate difference of the position marking of the moving object, determine the motion direction of the moving object according to the displacement vector, and obtain the motion state of the moving object based on the displacement vector of the moving object in adjacent frame images and the interval time length of the image frames, so as to serve as the motion information of the moving object; S32. Set a prediction time interval, calculate the coordinate position of the moving object in the flight airspace image at each future time according to the current position of the moving object in the flight airspace image and the motion information, and form a future position coordinate point set of the moving object; S33. Connect the future position coordinate points in the future position coordinate point set of the moving object in time sequence to obtain the motion trajectory of the moving object in the flight airspace image.

7. A low altitude flight safety co-pilot system according to claim 6, wherein, The specific operation method for judging whether the fixed obstacle has a collision risk is: Treat the four edges of the minimum circumscribed rectangle of the fixed obstacle as line segments, record the coordinates of the two endpoints of the four line segments, and form a boundary line segment set of the fixed obstacle; The end point coordinate range of the flight path segment is compared with the overall coordinate range determined by the fixed obstacle boundary segment set, the part without intersection is excluded, for the part with intersection, a geometric intersection detection algorithm is used to determine whether there is an intersection segment, if there is an intersection segment, it is determined that the aircraft flight path and the fixed obstacle exist spatial position conflict, that is, the aircraft will collide with the fixed obstacle under the current flight path.

8. A low altitude flight safety co-pilot system according to claim 1, wherein, The specific operation method of the route planning module is: A1. Taking the aircraft collision risk position as the center, combining the current flight direction and speed of the aircraft to determine the conflict influence range, identifying the segment in the current flight path of the aircraft in the conflict influence range as the affected flight path segment; A2. Setting the search starting point as the starting position of the affected flight path segment and the end point as the position farthest from the affected flight path segment in the normal airway part; A3. Setting the heuristic function of the algorithm, using the initialized heuristic search algorithm to search within the range of the affected flight path segment, and expanding the nodes according to the cost of each search node evaluated by the heuristic function; A4. Verifying the searched path, if the path passes the verification, outputting the path as a new safe flight path segment, if not, returning to A3 to continue searching until a safe flight path segment that passes the path verification is found.

Images