A method for identifying unmanned aerial vehicle routes

Abstract

The present invention discloses a method for identifying unmanned aerial vehicle (UAV) routes, and belongs to the technical field of UAV route identification. The method and technical solution of the method include the following steps: S1, route mapping; S2, probability model training; S3, probability estimation and processing; S4, route main direction calculation; S5, extraction of Rect area probability data along the main direction; S6, route extraction. The method associates route information with an image, and can guide the UAV to fly along the route through image recognition. The results at each moment are independent of each other, without the influence of cumulative errors, thereby improving the accuracy of UAV route identification.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) route identification, and in particular to a UAV route identification method. Background Art

[0002] Drone navigation and positioning technology plays a vital role in the flight safety of drones. The most commonly used method is to use satellite positioning systems such as GPS and Beidou to provide aircraft location. However, in some scenarios with complex terrain, weather and electromagnetic field environments, satellite positioning will be interfered with and fail.

[0003] For example, the patent publication number CN109211241A discloses an autonomous positioning method for UAVs based on visual SLAM, which consists of a feature extraction and matching motion solution part, an image and inertial measurement unit imu fusion part, and a 3D point depth estimation part; in the motion solution part, a strategy of combining the feature point method and the direct method is adopted to extract point features and line features after selecting key frames, and then minimize the error to complete the calculation of the relative pose; in the image and inertial measurement unit imu fusion part, the error is minimized to fuse them; the last step is to estimate the 3D point depth part. Based on the matching of the selected feature points, a triangulation method is used to solve the 3D position of the point, that is, to obtain the depth value of the point.

[0004] For example, patent publication number CN119845257A discloses a visual-inertial autonomous positioning method and system for fixed-wing UAVs that integrates satellite positioning. The method integrates satellite positioning, visual sensor and IMU data to obtain position and posture for fixed-wing UAVs. However, in the case of satellite denial, long-distance flight will cause cumulative errors, causing the positioning results to gradually diverge, affecting the accurate identification of the UAV route.

[0005] In response to the above problems, there is an urgent need for innovative designs based on the original drone route identification methods. Summary of the Invention

[0006] The purpose of the present invention is to provide a method for identifying drone routes to solve the problem raised in the above background technology that in the case of satellite denial, long-distance flight will cause cumulative errors, causing the positioning results to gradually diverge, affecting the accurate identification of the drone route.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for identifying a drone route, the technical solution comprising the following steps:

[0008] S1. Route Mapping

[0009] Extract the aircraft's route trajectory point set , n is the number of trajectory points.

[0010] All images taken during the flight and the corresponding pose information are retrieved, and the trajectory points are projected into each image.

[0011] S2. Probabilistic model training

[0012] Construct a dataset consisting of an original image set and a probability atlas. For an original image M, create an image M' of the same size. Use the original image set as input and the probability image set as output. Use a neural network for training to obtain a probability model. Take unet as an example for model training.

[0013] S3. Probability estimation and processing

[0014] Given an input image M0 to be predicted, use the above probability model to make predictions and output a probability map M1.

[0015] Perform Gaussian filtering on the probability map M1 to remove obvious outliers.

[0016] M1 is binarized, pixels greater than or equal to the threshold are set to 1, and pixels less than the threshold are set to 0, to obtain image M2.

[0017] Then perform expansion and corrosion operations on M2 to obtain M3.

[0018] S4. Calculation of main direction of route

[0019] Extract the contour of M3, traverse and find the contour with the largest area; calculate the minimum enclosing rectangle and direction of the largest contour.

[0020] S5. Extract Rect area probability data along the main direction

[0021] Calculate the corresponding key point set before and after the extraction of the Rect area and .

[0022] Calculate the transformation matrix T1 and the inverse transformation matrix T2.

[0023] Use T1 to transform the image M3 and fill the boundary area with the value V to obtain the image M4, which is the Rect area probability data map, which is a matrix with width w and height h respectively.

[0024] S6. Extract routes

[0025] For image M4, extract the left and right edge points of the binary image from top to bottom and calculate the corresponding route point coordinates.

[0026] Preferably, in S1, for the image M and the kth trajectory point ,calculate The projection point on M The coordinates of:

[0027]

[0028] where K is the camera parameter matrix, is the camera rotation matrix at the image capture moment, is the camera position at the image capture moment.

[0029] Preferably, the width W and height H of the image are obtained.

[0030] If 0 <= u < W and 0 <= v < H, the point is within the current image and is a valid point; otherwise, the point is discarded .

[0031] After projecting all the trajectory points, the set of valid points is the set of projection points of the flight path on the image M .

[0032] Each image has a corresponding set of projection points.

[0033] Preferably, in S2, the calculation method of each pixel value of the image M' is as follows: <00,00099>

[0035] where u and v are the (u + 1)-th column and (v + 1)-th row of the image respectively, is the standard deviation, and M' is the probability map corresponding to the original image M.

[0036] Preferably, in S4, the minimum bounding rectangle Rect = minAreaRect of the contour is obtained, and minAreaRect is the function for calculating the minimum bounding rectangle.

[0037] where Rect contains three pieces of information about the rectangle: the center point (x, y), the width and height (w, h), and the angle θ.

[0038] Preferably, in S5, the point set is constructed as follows:

[0039]

[0040] <00001!6>[[ID=)7]]

[0041] where is the rotation matrix, .

[0042] Preferably, for the construction of the point set, it is as follows:

[0043]

[0044]

[0045] Preferably, the transformation matrix T1 and the inverse transformation matrix T2 are calculated:

[0046] T1=getPerspectiveTransform(src_pts,dst_ptst)

[0047] T2=getPerspectiveTransform(dst_pts,src_pts)

[0048] Among them, getPerspectiveTransform is a transformation matrix calculation function.

[0049] Preferably, in S6, the route point coordinates are calculated as follows:

[0050] (1) For row j, traverse from left to right and use q i Represents the value of the i-th position point, left and right represent the candidate left and right line point positions respectively.

[0051] (2) If i = 0 and q0 = 1, then i is the left edge point and update left = i.

[0052] (3) If 0 <i<w-1:

[0053] a. If q i Equal to q i-1 : Normal, skip.

[0054] b. If q i Not equal to q i-1 .

[0055] a) If q i-1 =V or q i =V, skip.

[0056] b) If q i-1 =0, mark the i-th point as the left line point, and update left=i.

[0057] c) If q i =0, mark the i-th point as the right edge point, update right=i, and add (left, right) to the candidate edge point pair set.

[0058] (4) If i = w - 1 and pi = 1, then the i-th point is the right edge point, update right = i, and add (left, right) to the candidate edge point pair set.

[0059] (5) Traverse the set of candidate point pairs, calculate the length L = right-left, and find the candidate point pair (left, right) corresponding to the maximum length.

[0060] a. The coordinates of the route point corresponding to this row are (j, (right+left) / 2).

[0061] b. The position mapped to image M0 is:

[0062]

[0063]

[0064] The route extracted from image M0 is represented as a set of points , m is the number of route points.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] (1) By associating route information with images, the UAV can be guided to fly along the route through image recognition.

[0067] (2) The results at each moment are independent of each other and there is no cumulative error. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 This is a schematic diagram of the route identification technical solution of the present invention. DETAILED DESCRIPTION

[0069] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0070] Example 1: The technical solution includes the following steps:

[0071] S1. Route Mapping

[0072] Extract the aircraft's route trajectory point set , n is the number of trajectory points.

[0073] All images taken during the flight and the corresponding pose information are retrieved, and the trajectory points are projected into each image.

[0074] S2. Probabilistic model training

[0075] Construct a dataset consisting of an original image set and a probability atlas. For an original image M, create an image M' of the same size. Use the original image set as input and the probability image set as output. Use a neural network for training to obtain a probability model. Take unet as an example for model training.

[0076] S3. Probability estimation and processing

[0077] Given an input image M0 to be predicted, use the above probability model to make predictions and output a probability map M1.

[0078] Perform Gaussian filtering on the probability map M1 to remove obvious outliers.

[0079] M1 is binarized, pixels greater than or equal to the threshold are set to 1, and pixels less than the threshold are set to 0, to obtain image M2.

[0080] Then perform expansion and corrosion operations on M2 to obtain M3.

[0081] S4. Calculation of main direction of route

[0082] Extract the contour of M3, traverse and find the contour with the largest area; calculate the minimum enclosing rectangle and direction of the largest contour.

[0083] S5. Extract Rect area probability data along the main direction

[0084] Calculate the corresponding key point set before and after the extraction of the Rect area and .

[0085] Calculate the transformation matrix T1 and the inverse transformation matrix T2.

[0086] Use T1 to transform the image M3 and fill the boundary area with the value V to obtain the image M4, which is the Rect area probability data map, which is a matrix with width w and height h respectively.

[0087] S6. Extract routes

[0088] For image M4, extract the left and right edge points of the binary image from top to bottom and calculate the corresponding route point coordinates.

[0089] In S1, for image M and k-th trajectory point ,calculate Projection point on M Coordinates:

[0090]

[0091] Where K is the camera parameter matrix, is the camera rotation matrix at the image capture moment, is the camera position at the image capture moment.

[0092] Obtain the width W and height H of the said image.

[0093] If 0 <= u < W and 0 <= v < H, the point is within the current image and is a valid point; otherwise, discard the point .

[0094] After projecting all the trajectory points, the set of valid points is the set of projected points of the flight path on the image M .

[0095] Each image has a corresponding set of projected points.

[0096] In S2, the calculation method of each pixel value of the image M' is as follows:

[0097]

[0098] where u and v are the (u + 1)-th column and (v + 1)-th row of the image respectively, is the standard deviation, and M' is the probability map corresponding to the original image M.

[0099] In S4, obtain the minimum bounding rectangle Rect = minAreaRect of the contour, where minAreaRect is the function for calculating the minimum bounding rectangle.

[0100] where Rect contains three pieces of information about the rectangle: the center point (x, y), the width and height (w, h), and the angle θ.

[0101] In S5, construct a point set, and there is:

[0102]

[0103]

[0104] where is the rotation matrix, .

[0105] Construct a point set, and there is:

[0106]

[0107]

[0108] Preferably, calculate the transformation matrix T1 and the inverse transformation matrix T2:

[0109] T1=getPerspectiveTransform(src_pts,dst_ptst)

[0110] T2=getPerspectiveTransform(dst_pts,src_pts)

[0111] Among them, getPerspectiveTransform is a transformation matrix calculation function.

[0112] In S6, the coordinates of the route points are calculated as follows:

[0113] (1) For row j, traverse from left to right and use q i Represents the value of the i-th position point, left and right represent the candidate left and right line point positions respectively.

[0114] (2) If i = 0 and q0 = 1, then i is the left edge point and update left = i.

[0115] (3) If 0 <i<w-1:

[0116] a. If q i Equal to q i-1 : Normal, skip.

[0117] b. If q i Not equal to q i-1 .

[0118] a) If q i-1 =V or q i =V, skip.

[0119] b) If q i-1 =0, mark the i-th point as the left line point, and update left=i.

[0120] c) If q i =0, mark the i-th point as the right edge point, update right=i, and add (left, right) to the candidate edge point pair set.

[0121] (4) If i = w - 1 and pi = 1, then the i-th point is the right edge point, update right = i, and add (left, right) to the candidate edge point pair set.

[0122] (5) Traverse the set of candidate point pairs, calculate the length L = right-left, and find the candidate point pair (left, right) corresponding to the maximum length.

[0123] a. The coordinates of the route point corresponding to this row are (j, (right+left) / 2).

[0124] b. The position mapped to image M0 is:

[0125]

[0126]

[0127] The route extracted from image M0 is represented as a set of points , m is the number of route points

[0128] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "connected" and "connection" should be understood in a broad sense. For example, they can refer to fixed connection, detachable connection, or integral connection; mechanical connection, electrical connection; direct connection, or indirect connection through an intermediary. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0129] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying a drone route, characterized in that: The method includes: S1. Route mapping Extract the aircraft's route trajectory point set , n is the number of trajectory points; Take out all the images and corresponding pose information captured during the flight, and project the trajectory points into each image; S2. Probability model training Construct a data set, including the original image set and the probability map set. For an original image M, create an image M' of the same size. Using the original image set as the input and the probability image set as the output, train with a neural network to obtain a probability model. The neural network is a unet network; S3. Probability estimation and processing Given an input image M0 to be predicted, use the above probability model for prediction and output a probability map M1; Perform Gaussian filtering on the probability map M1 to remove obvious outliers; Binarize M1, set pixels greater than or equal to the threshold to 1, and pixels less than the threshold to 0 to obtain an image M2; Then perform dilation and erosion operations on M2 to obtain M3; S4. Calculation of the main direction of the route Extract the contour of M3, traverse and find the contour with the largest area contour; calculate the minimum bounding rectangle Rect and direction of the largest contour; S5. Extract probability data of the Rect area along the main direction Calculate the corresponding key point set before and after the extraction of the Rect area and ; Calculate the transformation matrix T1 and the inverse transformation matrix T2; Use T1 to transform the image M3 and fill the boundary area with the value V to obtain an image M4, which is the probability data map of the Rect area, and is a matrix with width and height w and h respectively; S6. Extract the route For the image M4, extract the left and right edge points of the binary map from top to bottom and calculate the corresponding route point coordinates.

2. The method for identifying a drone route according to claim 1, wherein: In S1, for the image M and the kth trajectory point ,calculate Projection point on M Coordinates: ; where K is the camera parameter matrix, is the camera rotation matrix at the moment the image was taken, is the camera position at the moment the image was taken.

3. The method for identifying a drone route according to claim 2, wherein: Obtain the width W and height H of the image; If 0 <= u < W and 0 <= v < H, the point is within the current image and is a valid point; Otherwise, discard the point ; After projecting all trajectory points, the set of valid points is the set of projection points of the route on the image M. ; Each image has a corresponding set of projection points.

4. The method for identifying a drone route according to claim 3, wherein: In S2, the calculation method of each pixel value of the image M' is as follows: ; where u and v are the u+1th column and v+1th row of the image, respectively. is the standard deviation, and M' is the probability map corresponding to the original image M.

5. The method for identifying a drone route according to claim 1, wherein: In S4, obtain the minimum bounding rectangle Rect = minAreaRect of the contour, and minAreaRect is the function for calculating the minimum bounding rectangle; Where Rect contains three pieces of information about the rectangle: the center point (x, y), width and height (w, h), and angle θ.

6. The method for identifying a drone route according to claim 5, wherein: In the S5, the construction Point set, there are: ； ； in is the rotation matrix, .

7. A method for identifying a drone route according to claim 6, characterized in that: Build Point set, there are: ； 。 8. A method for identifying a drone route according to claim 7, characterized in that: Calculate the transformation matrix T1 and the inverse transformation matrix T2: T1 = getPerspectiveTransform(src_pts, dst_ptst) T2 = getPerspectiveTransform(dst_pts, src_pts) Where getPerspectiveTransform is the function for calculating the transformation matrix.

9. The method for identifying a drone route according to claim 1, wherein: In S6, the calculation method of the route point coordinates is as follows: (1) For row j, traverse from left to right and use q i Represents the value of the i-th position point, left and right represent the candidate left and right line point positions respectively; (2) If i = 0 and q0 = 1, then i is the left edge point, update left = i; (3) If 0 < i < w - 1: a. If q i Equal to q i-1 : Normal point, skip; b. If q i Not equal to q i-1 : a) If q i-1 =V or q i =V, skip; b) If q i-1 =0, mark the i-th point as the left line point, and update left=i; c) If q i =0, mark the i-th point as the right edge point, update right=i, and add (left, right) to the candidate edge point pair set; (4) If i = w - 1 and pi = 1, then the i-th point is the right edge point, update right = i, and add (left, right) to the set of candidate edge point pairs; (5) Traverse the set of candidate point pairs, calculate the length L = right-left, and find the candidate point pair (left, right) corresponding to the maximum length. a. The coordinates of the route point corresponding to this row are (j, (right+left) / 2); b. The position mapped to image M0 is: ; ; The route extracted from image M0 is represented as a set of points , m is the number of route points.

Images