Optical flow positioning method of unmanned aerial vehicle and unmanned aerial vehicle

Abstract

The invention discloses an optical flow positioning method of an unmanned aerial vehicle and the unmanned aerial vehicle, and relates to the technical field of unmanned aerial vehicle positioning, and the method comprises the steps: collecting a continuous image sequence in the flight process of the unmanned aerial vehicle in real time, and constructing a Gaussian pyramid for the collected images; and then illumination compensation is carried out between adjacent frames, speed information of the unmanned aerial vehicle is calculated, feedback compensation is carried out on the speed information of the unmanned aerial vehicle, and accurate positioning of the unmanned aerial vehicle is realized. By inserting the progressive frame between the adjacent frames with the overlarge gray difference value, gradual brightness transition can be realized between the adjacent frames with the overlarge gray difference value, the appearance change of a target can be smoothed, a tracker is helped to adapt to illumination change, visual and algorithm difficulties caused by sudden brightness change are reduced, and the tracking accuracy is improved. And meanwhile, the progressive frame is inserted between the adjacent frames with overlarge gray difference values to provide more references for feature point matching, so that the wrong matching rate of the feature points is reduced, and the accuracy and robustness of unmanned aerial vehicle positioning are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) positioning, and in particular to an optical flow positioning method for an unmanned aerial vehicle (UAV) and the UAV. Background Art

[0002] Optical flow positioning is a common and low-cost method for autonomous positioning of UAVs. In the absence of GPS signals, an optical flow sensor is installed on the bottom of the UAV to collect image data in real time, and the displacement of two frames of images is calculated through the optical flow algorithm to achieve autonomous positioning. In recent years, with the increasing development of science and technology, UAVs have been widely used in military and civilian fields, such as air warning and low-altitude photography in outdoor environments, and intelligence reconnaissance, search and rescue in indoor environments. However, whether the UAV can perceive its own position and speed information in real time is crucial for its autonomous flight.

[0003] In the process of implementing high-precision optical flow positioning of drones, it is inevitable that there will be scenes with large changes in lighting during the flight of drones. The lighting changes will cause great changes in the grayscale values ​​of the image, resulting in an increase in the error matching rate of feature points, affecting the flight accuracy and positioning of the drone. Therefore, improvement is urgently needed. Summary of the invention

[0004] In view of the deficiencies in the prior art, the purpose of the present invention is to provide an optical flow positioning method for a UAV and a UAV, aiming to solve the above technical problems.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, an optical flow positioning method for a drone includes:

[0007] Collect continuous image sequences of the drone in real time during flight, and crop the collected images to construct a Gaussian pyramid;

[0008] Subtract the overall grayscale values ​​of adjacent frame images. If the difference is greater than the set grayscale difference threshold, then perform illumination compensation between adjacent frames, otherwise proceed to the next step;

[0009] Compare the number of feature points in the current frame with a preset feature point number threshold. If the number of feature points in the current frame is less than the preset number threshold, update the information and number of feature points, otherwise proceed to the next step.

[0010] Match and track the feature points of adjacent frames, and filter the matching results of feature points of adjacent frames to remove incorrectly matched feature points;

[0011] Divide the displacement distance of the feature point by the sampling time of the feature point of the adjacent frame to obtain the feature point velocity, and calculate the median of the feature point velocity;

[0012] Feedback compensation is performed on the UAV speed information to achieve accurate positioning of the UAV.

[0013] Preferably, a continuous image sequence is collected in real time during the flight of the drone, and the collected images are cropped to construct a Gaussian pyramid, specifically:

[0014] Obtain the current frame image I0 in the continuous image sequence and set it as the bottom image of the Gaussian pyramid;

[0015] Taking a point every other pixel in each direction on the current frame image I0, so as to halve the size of the current frame image I0, can be achieved by the following formula:

[0016] I k+1 (x,y)=I k (2x,2y)

[0017] Where x and y are the coordinates of the current frame image after downsampling;

[0018] Convolve the current frame image with the Gaussian kernel to obtain the blurred image, which is achieved by the following formula:

[0019] I k+1 =I k *G(x,y)

[0020] Among them I k+1 is the image of the k+1th layer, I k is the image of the kth layer, G(x,y) is the Gaussian kernel function, σ is the standard deviation;

[0021] Repeat the above image size halving and convolution operations to obtain a set of N-layer images {I0, I1, ..., I N}, where I0 is the current frame image, I N is the image of the Nth layer;

[0022] The number of layers of the current frame image is compared with the preset layer threshold. If the number of layers of the current frame image reaches the preset layer threshold, the image size halving operation is stopped; if the number of layers of the current frame image is less than the preset layer threshold, the size halving and convolution operations are continued until the number of layers of the current frame image is equal to the preset layer threshold.

[0023] Preferably, the overall grayscale values ​​of adjacent frame images are subtracted. If the difference is greater than a set grayscale difference threshold, illumination compensation is performed between adjacent frames. Otherwise, the next step is performed, which is specifically:

[0024] Obtain the grayscale values ​​of all pixels in the current frame image and the grayscale values ​​of all pixels in the next frame image;

[0025] The grayscale values ​​of all pixels in the current frame image are summed to obtain the overall grayscale value of the current frame image, and then the grayscale values ​​of all pixels in the next frame image are summed to obtain the overall grayscale value of the next frame image;

[0026] Subtract the overall grayscale value of the previous frame image from the overall grayscale value of the next frame image to obtain the grayscale difference, which is specifically achieved through the following formula:

[0027]

[0028] Where ΔG is the grayscale difference, G t (i) and G t+1 (i) is the grayscale value of the i-th pixel in the current frame and the next frame, respectively, and N represents the total number of pixels in the image;

[0029] The grayscale difference is compared with the set grayscale difference threshold. If the grayscale difference is greater than or equal to the grayscale difference threshold, illumination compensation is performed on the current frame image and the next frame image; if the grayscale difference is less than the grayscale difference threshold, no processing is performed.

[0030] Preferably, when the grayscale difference is greater than or equal to the grayscale difference threshold, illumination compensation is performed on the current frame image and the next frame image, specifically:

[0031] Get the gray value G of each pixel in the current frame t And the gray value G of each pixel in the next frame t+1 ;

[0032] Calculate the grayscale value of each pixel that makes up the interpolated frame This is achieved specifically through the following formula:

[0033]

[0034] Where k = 1, 2, 3, ... m-1, and k is a positive integer, α k is the difference coefficient and satisfies the condition that it is greater than 0 and less than 1, G t is the gray value of each pixel in the current frame, G t+1 is the gray value of each pixel in the next frame;

[0035] The generated insertion frame is inserted between the current frame and the next frame to achieve illumination compensation.

[0036] Preferably, the number of feature points in the current frame is compared with a preset feature point number threshold. If the number of feature points in the current frame is less than the preset number threshold, the information and number of feature points are updated. Otherwise, the next step is entered, which is specifically:

[0037] Determine the feature points in the current frame and filter the feature points;

[0038] The total number of filtered feature points in the current frame is compared with the preset feature point number threshold. If the total number of feature points in the current frame is less than the preset feature point number threshold, the number of feature points in the current frame is updated; if the total number of feature points in the current frame is greater than or equal to the preset feature point number threshold, the number of feature points in the current frame is maintained.

[0039] Preferably, the feature points in the current frame are determined and the feature points are screened, specifically:

[0040] Gaussian blur preprocessing is performed on the current frame, which is achieved through the following formula:

[0041] P b =G(P,σ)

[0042] Where P represents the current frame, G represents the Gaussian blur operation, and σ is the standard deviation;

[0043] Divide the current frame after blur processing into S regions;

[0044] Calculate the horizontal and vertical gradients of the pixels in S regions using the following formula:

[0045] D x =S x *P b

[0046] D y =S y *P b

[0047] Where S x and S y Respectively represent the Sobel operator of the pixel in the horizontal and vertical directions, D x and D y They are the gradient components in the horizontal direction and the gradient components in the vertical direction respectively;

[0048] The gradient magnitude M and direction θ of the pixel are calculated using the following formula:

[0049]

[0050] Harrys corner detection is used for each pixel in the current frame, which is implemented by the following formula:

[0051]

[0052] Where D x and D yThey are the gradient components in the horizontal direction and the gradient components in the vertical direction respectively;

[0053] Calculate Harrys response function R, which is implemented by the following formula:

[0054] R = det(H) - k (trace(H)) 2

[0055] Where det is the determinant of the matrix H, trace is the trace of the matrix H, and k is a constant;

[0056] Select the pixel with the largest R value in each area as the feature point;

[0057] The maximum R value in each area is compared with the preset R value. If the maximum R value in each area is less than the preset R value, the feature point is deleted; if the maximum R value in each area is greater than or equal to the preset R value, the feature point is retained.

[0058] Preferably, the feature points of adjacent frames are matched and tracked, and the results of the feature point matching of adjacent frames are screened to remove the feature points that are incorrectly matched, specifically:

[0059] Select any feature point in the current frame as the feature base point;

[0060] Calculate the Euclidean distance between the feature base point and the feature vectors of all feature points in the next frame to obtain a set of Euclidean distance values;

[0061] Select the smallest Euclidean distance value in the Euclidean distance value set, and obtain the feature vector corresponding to the smallest Euclidean distance value;

[0062] The head of the feature vector is the feature base point in the current frame, and the tail of the feature vector is the feature point corresponding to the feature base point in the next frame.

[0063] Preferably, the displacement distance of the feature point is divided by the feature point sampling time of the adjacent frames to obtain the feature point velocity, and the median of the feature point velocity is calculated, which is achieved by the following formula:

[0064]

[0065] Where (x i ,y i ) is the coordinate of the feature point in the current frame, (x i+1 ,y i+1 ) is the coordinate of the feature point in the next frame, and Δt is the sampling time of the feature point in the adjacent frame;

[0066]

[0067] in represents the median velocity of feature points, and n represents the number of feature points.

[0068] Preferably, feedback compensation is performed on the drone speed information to achieve accurate positioning of the drone, specifically:

[0069] Get the altitude of the drone above the ground;

[0070] Calculate the speed of the drone relative to the opposite side using the following formula:

[0071]

[0072] where v d is the moving speed of the UAV relative to the opposite side, and h is the height of the UAV above the ground;

[0073] The speed of the drone measured by the inertial navigation system is obtained, and the final output speed of the drone is obtained through feedback compensation, which is achieved through the following formula:

[0074] v f =βv d +(1-β)v g

[0075] where v f is the final output speed of the drone, v g is the speed of the UAV measured by the inertial navigation system, and β is the coefficient of the inertial navigation system;

[0076] The final output velocity of the UAV is subjected to Euler integration to achieve precise positioning of the UAV.

[0077] In a second aspect, a drone comprises: a memory, a processor, an optical flow sensor, an altitude sensor and an attitude sensor, wherein the memory, the processor, the optical flow sensor, the altitude sensor and the attitude sensor are communicatively connected with each other, the memory stores computer instructions, and the processor executes the above-mentioned optical flow positioning method for the drone by executing the computer instructions.

[0078] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0079] By inserting progressive frames between adjacent frames with excessively large grayscale differences, a gradual transition of brightness can be achieved between adjacent frames with excessively large grayscale differences. This can smooth changes in the target’s appearance and help the tracker adapt to changes in lighting, thereby reducing the visual and algorithmic difficulties caused by sudden changes in brightness. At the same time, inserting progressive frames between adjacent frames with excessively large grayscale differences provides more references for feature point matching, reduces the feature point error matching rate, and thus improves the accuracy and robustness of UAV positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0081] Figure 1 A flow chart of an optical flow positioning method for a UAV is shown. DETAILED DESCRIPTION

[0082] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0083] Reference Figure 1 The optical flow positioning method of a UAV and an embodiment of the UAV of the present invention are further described.

[0084] An optical flow positioning method for a drone, comprising:

[0085] Step 1: collect continuous image sequences during the flight of the drone in real time, and crop the collected images to construct a Gaussian pyramid;

[0086] Step 2: Subtract the overall grayscale values ​​of adjacent frame images. If the difference is greater than the set grayscale difference threshold, then perform illumination compensation between adjacent frames, otherwise proceed to the next step;

[0087] Step 3, compare the number of feature points in the current frame with a preset feature point number threshold. If the number of feature points in the current frame is less than the preset number threshold, update the information and number of feature points, otherwise proceed to the next step;

[0088] Step 4, matching and tracking the feature points of adjacent frames, and filtering the matching results of the feature points of adjacent frames to remove the feature points that are incorrectly matched;

[0089] Step 5, dividing the displacement distance of the feature point by the feature point sampling time of the adjacent frames to obtain the feature point velocity, and calculating the median of the feature point velocity;

[0090] Step 6: Feedback and compensate the UAV speed information to achieve accurate positioning of the UAV.

[0091] The continuous image sequence of the UAV during flight is collected in real time, and the collected images are cropped to construct a Gaussian pyramid, specifically:

[0092] Step 11, obtaining the current frame image I0 in the continuous image sequence, and setting it as the bottom image of the Gaussian pyramid;

[0093] Step 12, taking a point every other pixel in each direction on the current frame image I0, so as to halve the size of the current frame image I0, can be achieved by the following formula:

[0094] I k+1 (x,y)=I k (2x,2y)

[0095] Where x and y are the coordinates of the current frame image after downsampling;

[0096] Step 13, convolve the current frame image with the Gaussian kernel to obtain a blurred image, which is achieved by the following formula:

[0097] I k+1 =I k *G(x,y)

[0098] Among them I k+1 is the image of the k+1th layer, I k is the image of the kth layer, G(x,y) is the Gaussian kernel function, σ is the standard deviation;

[0099] Step 14, repeat the above image size halving and convolution operations to obtain a set of N-layer images {I0, I1, ..., I N}, where I0 is the current frame image, I N is the image of the Nth layer;

[0100] Step 15, compare the number of layers of the current frame image with the preset layer threshold. If the number of layers of the current frame image reaches the preset layer threshold, stop halving the image size; if the number of layers of the current frame image is less than the preset layer threshold, continue to halve the size and perform convolution operations until the number of layers of the current frame image is equal to the preset layer threshold.

[0101] It should be noted that by constructing a Gaussian pyramid, the pyramid structure is used to make the upper pyramid with a lower resolution represent multiple pixels of the lower pyramid, so a small displacement in the high-level image can represent a large displacement information on the bottom image. Through the image pyramid method, the algorithm is iterated upward, so that a pixel information in the top layer can represent a lot of information in the bottom layer, thereby satisfying the constraints for optical flow calculation. When the image size is small, the window will appear relatively large, and the optical flow can track faster targets. In the original image stage, the optical flow window will appear relatively small. Starting from the highest layer of the image pyramid with the least details to the lower layers of the pyramid with richer details, a process from coarse to fine is carried out, and the optical flow obtained is more accurate.

[0102] The overall grayscale values ​​of adjacent frame images are subtracted. If the difference is greater than the set grayscale difference threshold, illumination compensation is performed between adjacent frames. Otherwise, the next step is performed, which is:

[0103] Step 21, obtaining the grayscale values ​​of all pixels in the current frame image and the grayscale values ​​of all pixels in the next frame image;

[0104] Step 22, summing the grayscale values ​​of all pixels in the current frame image to obtain the overall grayscale value of the current frame image, and then summing the grayscale values ​​of all pixels in the next frame image to obtain the overall grayscale value of the next frame image;

[0105] Step 23, subtract the overall grayscale value of the previous frame image from the overall grayscale value of the next frame image to obtain the grayscale difference, which is specifically achieved by the following formula:

[0106]

[0107] Where ΔG is the grayscale difference, G t (i) and G t+1 (i) is the grayscale value of the i-th pixel in the current frame and the next frame, respectively, and N represents the total number of pixels in the image;

[0108] Step 24, compare the grayscale difference with the set grayscale difference threshold. If the grayscale difference is greater than or equal to the grayscale difference threshold, perform illumination compensation on the current frame image and the next frame image; if the grayscale difference is less than the grayscale difference threshold, no processing is performed.

[0109] When the grayscale difference is greater than or equal to the grayscale difference threshold, illumination compensation is performed on the current frame image and the next frame image, specifically:

[0110] Step 241, obtain the gray value G of each pixel in the current frame t And the gray value G of each pixel in the next frame t+1 ;

[0111] Step 242, calculate the gray value of each pixel constituting the inserted frame This is achieved specifically through the following formula:

[0112]

[0113] Where k = 1, 2, 3, ... m-1, and k is a positive integer, α k is the difference coefficient and satisfies the condition that it is greater than 0 and less than 1, G t is the gray value of each pixel in the current frame, G t+1 is the gray value of each pixel in the next frame;

[0114] Step 243, inserting the generated insertion frame between the current frame and the next frame to achieve illumination compensation.

[0115] It should be noted that by inserting progressive frames between adjacent frames with excessively large grayscale differences, a gradual transition of brightness can be achieved between adjacent frames with excessively large grayscale differences, which can smooth changes in the target's appearance and help the tracker adapt to changes in lighting, thereby reducing the visual and algorithmic difficulties caused by sudden changes in brightness. At the same time, inserting progressive frames between adjacent frames with excessively large grayscale differences provides more references for feature point matching, reduces the feature point error matching rate, and thus improves the accuracy and robustness of UAV positioning.

[0116] Compare the number of feature points in the current frame with the preset feature point number threshold. If the number of feature points in the current frame is less than the preset number threshold, update the information and number of feature points. Otherwise, proceed to the next step, which is:

[0117] Step 31, determining the feature points in the current frame and screening the feature points;

[0118] Step 32, compare the total number of filtered feature points in the current frame with the preset feature point number threshold. If the total number of feature points in the current frame is less than the preset feature point number threshold, update the number of feature points in the current frame; if the total number of feature points in the current frame is greater than or equal to the preset feature point number threshold, maintain the number of feature points in the current frame.

[0119] Determine the feature points in the current frame and filter them, specifically:

[0120] Step 311, perform Gaussian blur preprocessing on the current frame, which is achieved by the following formula:

[0121] P b =G(P,σ)

[0122] Where P represents the current frame, G represents the Gaussian blur operation, and σ is the standard deviation;

[0123] Step 312, dividing the current frame after blurring into S regions;

[0124] Step 313, calculating the gradients of the pixels in the S regions in the horizontal and vertical directions, is implemented by the following formula:

[0125] D x =S x *P b

[0126] D y =S y *P b

[0127] Where S x and S y Respectively represent the Sobel operator of the pixel in the horizontal and vertical directions, D x and D y They are the gradient components in the horizontal direction and the gradient components in the vertical direction respectively;

[0128] Step 314, calculate the gradient magnitude M and direction θ of the pixel point using the following formula:

[0129]

[0130] Step 315, Harrys corner detection is performed on each pixel in the current frame, which is specifically implemented by the following formula:

[0131]

[0132] Where D x and D y They are the gradient components in the horizontal direction and the gradient components in the vertical direction respectively;

[0133] Step 316, calculate Harrys response function R, which is specifically implemented by the following formula:

[0134] R = det(H) - k (trace(H)) 2

[0135] Where det is the determinant of the matrix H, trace is the trace of the matrix H, and k is a constant;

[0136] Step 317, selecting the pixel with the largest R value in each area as a feature point;

[0137] Step 318, compare the maximum R value in each area with the preset R value. If the maximum R value in each area is less than the preset R value, delete the feature point; if the maximum R value in each area is greater than or equal to the preset R value, retain the feature point.

[0138] Match and track the feature points of adjacent frames, and filter the matching results of feature points of adjacent frames to remove incorrectly matched feature points. Specifically:

[0139] Step 41, selecting any feature point in the current frame as a feature base point;

[0140] Step 42, calculating the Euclidean distance between the feature base point and the feature vectors of all feature points in the next frame to obtain a set of Euclidean distance values;

[0141] Step 43, selecting the smallest Euclidean distance value in the Euclidean distance value set, and obtaining a feature vector corresponding to the smallest Euclidean distance value;

[0142] In step 44, the head end of the feature vector is the feature base point in the current frame, and the tail end of the feature vector is the feature point corresponding to the feature base point in the next frame.

[0143] Divide the displacement distance of the feature point by the sampling time of the feature point of the adjacent frame to obtain the feature point velocity, and calculate the median of the feature point velocity, which is achieved by the following formula:

[0144]

[0145] Where (x i ,y i ) is the coordinate of the feature point in the current frame, (x i+1 ,y i+1 ) is the coordinate of the feature point in the next frame, and Δt is the sampling time of the feature point in the adjacent frame;

[0146]

[0147] in represents the median velocity of feature points, and n represents the number of feature points.

[0148] Feedback compensation is performed on the UAV speed information to achieve accurate positioning of the UAV, specifically:

[0149] Step 61, obtaining the altitude of the drone above the ground;

[0150] Step 62, calculate the moving speed of the drone relative to the opposite side, which is achieved by the following formula:

[0151]

[0152] where v d is the moving speed of the UAV relative to the opposite side, and h is the height of the UAV above the ground;

[0153] Step 63, obtaining the speed of the UAV measured by the inertial navigation system, and obtaining the final output speed of the UAV through feedback compensation, which is achieved by the following formula:

[0154] v f =βv d +(1-β)v g

[0155] where v f is the final output speed of the drone, v g is the speed of the UAV measured by the inertial navigation system, and β is the coefficient of the inertial navigation system;

[0156] Step 64, performing Euler integration on the final output velocity of the UAV to achieve accurate positioning of the UAV.

[0157] A drone comprises a memory, a processor, an optical flow sensor, an altitude sensor and an attitude sensor. The memory, the processor, the optical flow sensor, the altitude sensor and the attitude sensor are communicatively connected with each other. Computer instructions are stored in the memory. The processor executes the computer instructions to execute the optical flow positioning method of the drone.

[0158] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0159] The memory is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, the optical flow positioning method of the drone in the above embodiment is realized.

[0160] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, a power grid, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0161] The one or more modules are stored in the memory, and when executed by the processor, the optical flow positioning method for the drone in the embodiment shown in the accompanying drawings is executed.

[0162] The specific details of the above-mentioned computer device can be understood by referring to the corresponding related descriptions and effects in the embodiments shown in the accompanying drawings, and will not be repeated here.

[0163] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0164] The above description of the embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An optical flow positioning method for a drone, characterized in that: include: Collect continuous image sequences of the drone in real time during flight, and crop the collected images to construct a Gaussian pyramid; Subtract the overall grayscale values ​​of adjacent frame images. If the difference is greater than the set grayscale difference threshold, then perform illumination compensation between adjacent frames, otherwise proceed to the next step; Compare the number of feature points in the current frame with a preset feature point number threshold. If the number of feature points in the current frame is less than the preset number threshold, update the information and number of feature points, otherwise proceed to the next step. Match and track the feature points of adjacent frames, and filter the matching results of feature points of adjacent frames to remove incorrectly matched feature points; Divide the displacement distance of the feature point by the sampling time of the feature point of the adjacent frame to obtain the feature point velocity, and calculate the median of the feature point velocity; Feedback compensation is performed on the UAV speed information to achieve accurate positioning of the UAV.

2. The optical flow positioning method for a UAV according to claim 1, characterized in that: The continuous image sequence of the UAV during flight is collected in real time, and the collected images are cropped to construct a Gaussian pyramid, specifically: Obtain the current frame image I0 in the continuous image sequence and set it as the bottom image of the Gaussian pyramid; Taking a point every other pixel in each direction on the current frame image I0, so as to halve the size of the current frame image I0, can be achieved by the following formula: I k+1 (x,y)=I k (2x,2y) Where x and y are the coordinates of the current frame image after downsampling; Convolve the current frame image with the Gaussian kernel to obtain the blurred image, which is achieved by the following formula: I k+1 =I k *G(x,y) Among them I k+1 is the image of the k+1th layer, I k is the image of the kth layer, G(x,y) is the Gaussian kernel function, σ is the standard deviation; Repeat the above image size halving and convolution operations to obtain a set of N-layer images {I0, I1, ..., I N }, where I0 is the current frame image, I N is the image of the Nth layer; The number of layers of the current frame image is compared with the preset layer threshold. If the number of layers of the current frame image reaches the preset layer threshold, the image size halving operation is stopped; if the number of layers of the current frame image is less than the preset layer threshold, the size halving and convolution operations are continued until the number of layers of the current frame image is equal to the preset layer threshold.

3. The optical flow positioning method for a UAV according to claim 2 is characterized in that: The overall grayscale values ​​of adjacent frame images are subtracted. If the difference is greater than the set grayscale difference threshold, illumination compensation is performed between adjacent frames. Otherwise, the next step is performed, which is: Obtain the grayscale values ​​of all pixels in the current frame image and the grayscale values ​​of all pixels in the next frame image; The grayscale values ​​of all pixels in the current frame image are summed to obtain the overall grayscale value of the current frame image, and then the grayscale values ​​of all pixels in the next frame image are summed to obtain the overall grayscale value of the next frame image; Subtract the overall grayscale value of the previous frame image from the overall grayscale value of the next frame image to obtain the grayscale difference, which is specifically achieved through the following formula: Where ΔG is the grayscale difference, G t (i) and G t+1 (i) is the grayscale value of the i-th pixel in the current frame and the next frame, respectively, and N represents the total number of pixels in the image; The grayscale difference is compared with the set grayscale difference threshold. If the grayscale difference is greater than or equal to the grayscale difference threshold, illumination compensation is performed on the current frame image and the next frame image; if the grayscale difference is less than the grayscale difference threshold, no processing is performed.

4. The optical flow positioning method for a UAV according to claim 3 is characterized in that: When the grayscale difference is greater than or equal to the grayscale difference threshold, illumination compensation is performed on the current frame image and the next frame image, specifically: Get the gray value G of each pixel in the current frame t And the gray value G of each pixel in the next frame t+1 ; Calculate the grayscale value of each pixel that makes up the interpolated frame This is achieved specifically through the following formula: Where k = 1, 2, 3, ... m-1, and k is a positive integer, α k is the difference coefficient and satisfies the condition that it is greater than 0 and less than 1, G t is the gray value of each pixel in the current frame, G t+1 is the gray value of each pixel in the next frame; The generated insertion frame is inserted between the current frame and the next frame to achieve illumination compensation.

5. The optical flow positioning method for a drone according to claim 4, characterized in that: Compare the number of feature points in the current frame with the preset feature point number threshold. If the number of feature points in the current frame is less than the preset number threshold, update the information and number of feature points. Otherwise, proceed to the next step, which is: Determine the feature points in the current frame and filter the feature points; The total number of filtered feature points in the current frame is compared with the preset feature point number threshold. If the total number of feature points in the current frame is less than the preset feature point number threshold, the number of feature points in the current frame is updated; if the total number of feature points in the current frame is greater than or equal to the preset feature point number threshold, the number of feature points in the current frame is maintained.

6. The optical flow positioning method for a drone according to claim 5, characterized in that: Determine the feature points in the current frame and filter them, specifically: Gaussian blur preprocessing is performed on the current frame, which is achieved through the following formula: P.S b G(P,σ) Where P represents the current frame, G represents the Gaussian blur operation, and σ is the standard deviation; Divide the current frame after blur processing into S regions; Calculate the horizontal and vertical gradients of the pixels in S regions using the following formula: D x =S x *P b D y =S y *P b Where S x and S y Respectively represent the Sobel operator of the pixel in the horizontal and vertical directions, D x and D y They are the gradient components in the horizontal direction and the gradient components in the vertical direction respectively; The gradient magnitude M and direction θ of the pixel are calculated using the following formula: Harrys corner detection is used for each pixel in the current frame, which is implemented by the following formula: Where D x and D y They are the gradient components in the horizontal direction and the gradient components in the vertical direction respectively; Calculate Harrys response function R, which is implemented by the following formula: R=det(H)-k·(trace(H)) 2 Where det is the determinant of the matrix H, trace is the trace of the matrix H, and k is a constant; Select the pixel with the largest R value in each area as the feature point; The maximum R value in each area is compared with the preset R value. If the maximum R value in each area is less than the preset R value, the feature point is deleted; if the maximum R value in each area is greater than or equal to the preset R value, the feature point is retained.

7. The optical flow positioning method for a UAV according to claim 6, characterized in that: Match and track the feature points of adjacent frames, and filter the matching results of feature points of adjacent frames to remove incorrectly matched feature points. Specifically: Select any feature point in the current frame as the feature base point; Calculate the Euclidean distance between the feature base point and the feature vectors of all feature points in the next frame to obtain a set of Euclidean distance values; Select the smallest Euclidean distance value in the Euclidean distance value set, and obtain the feature vector corresponding to the smallest Euclidean distance value; The head of the feature vector is the feature base point in the current frame, and the tail of the feature vector is the feature point corresponding to the feature base point in the next frame.

8. The optical flow positioning method for a UAV according to claim 7, characterized in that: Divide the displacement distance of the feature point by the sampling time of the feature point of the adjacent frame to obtain the feature point velocity, and calculate the median of the feature point velocity, which is achieved by the following formula: Where (x i ,y i ) is the coordinate of the feature point in the current frame, (x i+1 ,y i+1 ) is the coordinate of the feature point in the next frame, and Δt is the sampling time of the feature point in the adjacent frame; in represents the median velocity of feature points, and n represents the number of feature points.

9. The optical flow positioning method for a UAV according to claim 8, characterized in that: Feedback compensation is performed on the UAV speed information to achieve accurate positioning of the UAV, specifically: Get the altitude of the drone above the ground; Calculate the speed of the drone relative to the opposite side using the following formula: where v d is the moving speed of the UAV relative to the opposite side, and h is the height of the UAV above the ground; The speed of the drone measured by the inertial navigation system is obtained, and the final output speed of the drone is obtained through feedback compensation, which is achieved through the following formula: v f =βv d +(1-β)v g where v f is the final output speed of the drone, v g is the speed of the UAV measured by the inertial navigation system, and β is the coefficient of the inertial navigation system; The final output velocity of the UAV is subjected to Euler integration to achieve precise positioning of the UAV.

10. A drone, characterized in that: include: A memory, a processor, an optical flow sensor, an altitude sensor and an attitude sensor, wherein the memory, the processor, the optical flow sensor, the altitude sensor and the attitude sensor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the optical flow positioning method for a drone according to any one of claims 1 to 9 by executing the computer instructions.

Images