Indoor positioning device and method for ornithopter, terminal, medium and ornithopter

Abstract

The invention provides a flapping-wing air vehicle indoor positioning device and method, a terminal, a medium and a flapping-wing air vehicle. The indoor positioning device comprises an optical flow sensor provided with an event camera, a TOF sensor, a six-axis inertial sensor and a control module. The event camera collects optical flow image information, vibration and real motion of a machine body are separated through space-time filtering and vibration separation processing, and accurate optical flow information is obtained through self-adaptive smoothing and confidence coefficient weighting processing. The acceleration information acquired by the six-axis inertial sensor is subjected to CHAME filtering processing, and the vibration noise of the machine body is lowered, so that the real acceleration information is obtained. And finally, performing Kalman filtering fusion on the height information, the optical flow information and the real acceleration information which are acquired by the TOF sensor after scale normalization to obtain three-axis position information of the ornithopter. According to the invention, more accurate indoor position sensing and positioning of the ornithopter are realized.

Description

Technical Field

[0001] The present application relates to the field of UAV visual positioning technology, and in particular to a flapping-wing aircraft indoor positioning device, method, terminal, medium and flapping-wing aircraft. Background Art

[0002] Currently, common quadcopters use optical flow sensors and time-of-flight sensors to achieve fixed-point flight indoors, integrating data from a six-axis inertial measurement unit (IMU) to achieve displacement sensing. During use, a frame camera and laser sensor are mounted downward on the bottom of the aircraft. The frame camera captures ground images at a high frame rate (typically 100Hz to 200Hz), performing a simple continuous optical flow estimation. Displacement information is then derived from the sensor's focal length and the altitude measured by the time-of-flight sensor, combined with the IMU data. The time-of-flight sensor obtains altitude information by measuring the time difference between the transmitted and reflected light beams.

[0003] Unlike quadcopters, flapping-wing aircraft, due to their unique biomimetic flight mode, pose a greater challenge to indoor positioning than multi-rotors. First, flapping-wing aircraft experience high-frequency vibrations. The movements of flapping-wing aircraft cause severe vibrations, making the frame cameras used in traditional optical flow systems prone to failure. The acceleration measured by the six-axis inertial sensor can reach 6–7 times the acceleration of gravity, resulting in significant errors in the fused estimated velocity and position information. Second, flapping-wing aircraft have stringent weight requirements, making it difficult to carry high-power or heavy optical flow modules. Most importantly, optical flow positioning modules are typically mounted on the tail of a flapping-wing aircraft. Due to the unique aerodynamic pattern of the flapping wings, the tail produces small oscillations of 1–2 Hz during flapping, significantly affecting optical flow sensor data acquisition. Due to these factors, existing positioning methods for flapping-wing aircraft suffer from significant errors. Summary of the Invention

[0004] In view of the above-mentioned shortcomings of the prior art, the purpose of this application is to provide a flapping-wing aircraft indoor positioning device, method, terminal, medium and flapping-wing aircraft, which are used to solve the problem of large errors in flapping-wing aircraft using existing positioning methods.

[0005] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides an indoor positioning device for a flapping-wing aircraft, which is provided on the flapping-wing aircraft and includes: an optical flow sensor, a TOF sensor, a six-axis inertial sensor and a control module; wherein the optical flow sensor adopts an event camera; the event camera is used to collect optical flow image information; the TOF sensor is used to collect flight altitude information; the six-axis inertial sensor is used to collect flight acceleration information and flight angular velocity information; the control module is used to perform spatiotemporal filtering and vibration separation processing on the optical flow image information based on the flight angular velocity information to obtain real optical flow information; perform adaptive smoothing and confidence weighting processing on the real optical flow information to obtain final optical flow information; perform CHAME filtering on the acceleration information based on the flight angular velocity information to obtain real acceleration information; perform Kalman filtering fusion on the scale-normalized altitude information, the final optical flow information and the real acceleration information to obtain the three-axis position information of the flapping-wing aircraft.

[0006] In some embodiments of the first aspect of the present application, based on the flight angular velocity information, the optical flow image information is subjected to spatiotemporal filtering and vibration separation processing to obtain real optical flow information, including: calculating the vibration main frequency information based on the angular velocity information; performing time window filtering on the optical flow image information according to the vibration main frequency information to obtain filtered optical flow image data; performing optical flow estimation on the filtered optical flow image data to obtain original optical flow information; based on the angular velocity information, performing frequency domain analysis and motion compensation processing on the original optical flow information to obtain vibration information; and separating the real optical flow information from the original optical flow information based on the vibration information.

[0007] In some embodiments of the first aspect of the present application, based on the angular velocity information, the original optical flow information is subjected to frequency domain analysis and motion compensation processing to obtain vibration information, including: performing a pixel-by-pixel short-time Fourier transform on the original optical flow information, and filtering the Fourier transformed optical flow information according to a set bandwidth to obtain sparse representation information of the vibration noise optical flow; based on the angular velocity information, calculating the theoretical optical flow information; and aligning the sparse representation information of the vibration noise optical flow with the theoretical optical flow information to obtain vibration information.

[0008] In some embodiments of the first aspect of the present application, the real optical flow information is adaptively smoothed and confidence-weighted to obtain the final optical flow information, including: calculating the confidence of the vibration area based on the vibration information; and performing confidence-based smoothing on the real optical flow information based on the confidence of the vibration area to obtain the final optical flow information.

[0009] In some embodiments of the first aspect of the present application, based on the flight angular velocity information, the acceleration information is CHAME filtered to obtain real acceleration information, including: calculating the vibration main frequency information based on the angular velocity information; using fast Fourier transform to extract the fundamental frequency information and harmonic information from the vibration main frequency information; based on the harmonic information, the acceleration information is sequentially subjected to pre-low-pass filtering, multi-stage band-stop filtering, adaptive sliding average filtering, and motion acceleration and drift compensation to obtain real acceleration information.

[0010] In some embodiments of the first aspect of the present application, the motion acceleration and drift compensation processing includes: performing zero-phase low-pass filtering on the acceleration information after adaptive sliding average filtering to extract the acceleration information after zero-phase low-pass filtering; using high-pass filtering to eliminate the integral drift in the acceleration information after zero-phase low-pass filtering to obtain real acceleration information.

[0011] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides a flapping-wing aircraft indoor positioning method, which is applied to a flapping-wing aircraft indoor positioning device, the device including: an optical flow sensor, a TOF sensor and a six-axis inertial sensor, the method including: receiving optical flow image information from the optical flow sensor, flight altitude information from the TOF sensor, and flight acceleration information and flight angular velocity information from the six-axis inertial sensor; wherein the optical flow sensor adopts an event camera; the event camera is used to collect optical flow image information; based on the flight angular velocity information, the optical flow image information is subjected to spatiotemporal filtering and vibration separation processing to obtain real optical flow information; the real optical flow information is subjected to adaptive smoothing and confidence weighting processing to obtain final optical flow information; based on the flight angular velocity information, the acceleration information is subjected to CHAME filtering processing to obtain real acceleration information; the scale-normalized altitude information, the final optical flow information and the real acceleration information are subjected to Kalman filtering fusion to obtain the three-axis position information of the flapping-wing aircraft.

[0012] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the indoor positioning method of the flapping-wing aircraft when the computer program is executed by a processor.

[0013] To achieve the above-mentioned purpose and other related purposes, the fourth aspect of the present application provides an electronic terminal, including a memory, a processor and a computer program stored in the memory; the processor executes the computer program to implement the indoor positioning method of the flapping-wing aircraft.

[0014] To achieve the above-mentioned purpose and other related purposes, a fifth aspect of the present application provides a flapping-wing aircraft, on which the flapping-wing aircraft indoor positioning device as described above is provided.

[0015] As described above, the flapping-wing aircraft indoor positioning device, method, terminal, medium, and flapping-wing aircraft of the present application have the following beneficial effects:

[0016] This application achieves more accurate indoor position perception and positioning of flapping-wing aircraft. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Shown is a schematic block diagram of an indoor positioning device for a flapping-wing aircraft in one embodiment of the present application.

[0018] Figure 2 Shown is a schematic diagram of the indoor positioning process of a flapping-wing aircraft in one embodiment of the present application.

[0019] Figure 3 Shown is a schematic diagram of the process of obtaining final optical flow information in one embodiment of the present application.

[0020] Figure 4 Shown is a schematic diagram of a process for obtaining real acceleration information in one embodiment of the present application.

[0021] FIG5( a ) is a schematic diagram showing the actual position of the flapping-wing aircraft in the X direction in a specific embodiment of the present application.

[0022] FIG5( b ) is a schematic diagram showing the actual position of the flapping-wing aircraft in the Y direction in a specific embodiment of the present application.

[0023] FIG5( c ) is a schematic diagram showing the actual position of the flapping-wing aircraft in the Z direction in a specific embodiment of the present application.

[0024] Figure 6 Shown is a schematic diagram of the three-axis acceleration of a flapping-wing aircraft in a specific embodiment of the present application.

[0025] Figure 7 Shown is a schematic diagram of vibration information in a specific embodiment of the present application.

[0026] Figure 8 Shown is a flow chart of a method for indoor positioning of a flapping-wing aircraft in one embodiment of the present application.

[0027] Figure 9 Shown is a structural schematic diagram of an electronic terminal in one embodiment of the present application. DETAILED DESCRIPTION

[0028] The following describes the embodiments of the present application through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0029] In the embodiments of this application, terms such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or order of execution, and that terms such as "first" and "second" do not necessarily define differences.

[0030] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" represent examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0031] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.

[0032] Before further explaining the present invention in detail, the nouns and terms involved in the embodiments of the present invention are explained. The nouns and terms involved in the embodiments of the present invention are subject to the following interpretations:

[0033] <1> Flapping-wing aircraft: A flapping-wing aircraft is an aircraft that imitates the flapping of wings of birds or insects to achieve flight.

[0034] <2> Optical flow sensor: An optical flow sensor is a sensor based on optical principles that is used to measure the motion of an object relative to its surroundings.

[0035] <3> Event Camera: An event camera is a new type of visual sensor inspired by biological vision mechanisms.

[0036] <4> TOF sensor: A TOF (Time-of-Flight) sensor is a distance measurement sensor based on the time-of-flight principle. It calculates the distance by emitting a light pulse (usually infrared light) to the surface of the object being measured and then measuring the time it takes for the light pulse to return to the sensor receiver.

[0037] <5> Six-axis inertial sensor: A six-axis inertial sensor is a sensor module that integrates a three-axis accelerometer and a three-axis gyroscope. It can measure the linear acceleration and angular velocity of an object in three-dimensional space, thereby providing attitude, position, and velocity information.

[0038] <6> Short-Time Fourier Transform: The Short-Time Fourier Transform (STFT) is a method for analyzing the frequency characteristics of time-varying signals. It divides the signal into shorter time segments and performs a Fourier transform on the signal within each time segment, thereby observing the frequency distribution of the signal at different time points.

[0039] <7> Inverse Short-Time Fourier Transform (ISTFT): The inverse of the Short-Time Fourier Transform (STFT). The STFT converts a time-domain signal into a time-frequency domain representation, while the ISTFT converts the time-frequency domain representation back into a time-domain signal.

[0040] <8> Optical flow image: Optical flow image refers to the motion vector field of pixel points in an image sequence calculated by the optical flow method, which can reflect the motion information of objects in the image.

[0041] To facilitate understanding of the embodiments of this application, first Figure 1 Detailed description. Figure 1 The following is a schematic block diagram of a flapping-wing aircraft indoor positioning device according to an embodiment of the present invention. The flapping-wing aircraft indoor positioning device according to this embodiment is provided on a flapping-wing aircraft and includes:

[0042] Optical flow sensor 1, TOF sensor 2, six-axis inertial sensor 3 and control module 4;

[0043] The optical flow sensor uses an event camera 11; the event camera 11 is used to collect optical flow image information;

[0044] The TOF sensor 2 is used to collect the flight altitude information of the flapping-wing aircraft; the six-axis inertial sensor 3 is used to collect the flight acceleration information and flight angular velocity information of the flapping-wing aircraft;

[0045] The control module 4 is used to perform spatiotemporal filtering and vibration separation processing on the optical flow image information based on the flight angular velocity information to obtain real optical flow information; perform adaptive smoothing and confidence weighting processing on the real optical flow information to obtain final optical flow information; perform CHAME filtering on the acceleration information based on the flight angular velocity information to obtain real acceleration information; and perform Kalman filtering fusion on the scale-normalized height information, the final optical flow information, and the real acceleration information to obtain the three-axis position information of the flapping-wing aircraft.

[0046] It should be noted that the present invention does not limit the models of optical flow sensors, TOF sensors, and six-axis inertial sensors. Appropriate models of optical flow sensors, TOF sensors, and six-axis inertial sensors can be selected according to actual needs.

[0047] It should also be noted that traditional frame cameras are limited to a fixed frame rate (typically 30-60Hz), which can lead to motion blur in the high-frequency vibration environment of flapping-wing aircraft. Conventional continuous optical flow estimation algorithms are prone to losing key information. The advantage of using an event camera rather than a traditional frame camera as the optical flow sensor in this invention is that the event camera has a temporal resolution of up to μs and only records brightness changes. This avoids the image patterns caused by the global shutter of traditional frame cameras under high-frequency vibration, which can easily cause the optical flow algorithm to fail. This reduces the computing power burden of the control module and improves accuracy.

[0048] In one embodiment, the flapping-wing aircraft indoor positioning device also includes a redundant six-axis inertial measurement unit (IMU), which not only enables precise indoor displacement sensing and positioning, but also improves the reliability of the flapping-wing aircraft. The flapping-wing aircraft indoor positioning device can weigh less than 1g, meeting the requirements of the flapping-wing aircraft's limited payload capacity.

[0049] In one embodiment, the control module may be a single chip microcomputer, or other devices with processing and computing functions, which is not limited in the present invention.

[0050] In one embodiment, based on the flight angular velocity information, the optical flow image information is subjected to spatiotemporal filtering and vibration separation processing to obtain real optical flow information, including: calculating the vibration main frequency information based on the angular velocity information; performing time window filtering on the optical flow image information according to the vibration main frequency information to obtain filtered optical flow image data; performing optical flow estimation on the filtered optical flow image data to obtain original optical flow information; based on the angular velocity information, performing frequency domain analysis and motion compensation processing on the original optical flow information to obtain vibration information; and separating the real optical flow information from the original optical flow information based on the vibration information.

[0051] In one embodiment, based on the angular velocity information, the original optical flow information is subjected to frequency domain analysis and motion compensation processing to obtain vibration information, including: performing pixel-by-pixel short-time Fourier transform on the original optical flow information, and filtering the Fourier transformed optical flow information according to a set bandwidth to obtain sparse representation information of the vibration noise optical flow; based on the angular velocity information, calculating theoretical optical flow information, and aligning the sparse representation information of the vibration noise optical flow with the theoretical optical flow information to obtain vibration information.

[0052] The following will be combined with the attached Figure 2 And attached Figure 3 The method of obtaining the final optical flow information is described in detail:

[0053] According to the following formula 1, the vibration main frequency information is calculated based on the angular velocity information:

[0054]

[0055] Among them, f vib is the main frequency information of vibration, ω z (t) is the angular velocity information, is the Fourier transform.

[0056] Furthermore, according to the vibration main frequency information, the optical flow image information Perform time window filtering to preserve the vibration period T vib Events other than ε filtered .P k Usually indicates an event e k The polarity of the event is a binary value (usually +1 or -1) that indicates the direction of the brightness change of the event. k = +1 indicates that the brightness of the pixel position increases, P k =-1 means that the brightness of the pixel position is reduced.

[0057] Among them, T vib =1 / f vib , α∈(0,1) is the proportion of active vibration period, which can be adjusted according to actual needs.

[0058] Furthermore, optical flow estimation is performed on the filtered optical flow image information to obtain the original optical flow information. It should be noted that optical flow estimation is an important technique in computer vision, used to calculate the motion information of pixels in an image sequence, namely, the direction and speed of movement of each pixel between adjacent frames. The present invention can employ existing optical flow estimation methods, which will not be further described here.

[0059] Furthermore, the original optical flow information is subjected to short-time Fourier transform pixel by pixel according to the following formula 2:

[0060]

[0061] Among them, V(x,y,f,t) is the optical flow information after Fourier transform, represents the short-time Fourier transform, v raw (x, y, t) is the original optical flow information.

[0062] Furthermore, according to the following formula 3, the optical flow information after Fourier transformation is retained (|ff vib |<Δf), which is the sparse representation information of the vibration noise optical flow.

[0063]

[0064] Where Δf is the set bandwidth, V(x,y,f vib ,t) is the sparse representation of the aircraft vibration noise in the frequency domain and is the part that needs to be filtered out. It should be noted that Δf is generally set according to the vibration bandwidth.

[0065] Furthermore, according to the following formula 4, the theoretical optical flow information caused by vibration is calculated based on the angular velocity information:

[0066]

[0067] Among them, ω z is the angular velocity information, (x0, y0) is the rotation center of the flapping-wing aircraft, (x, y) is the pixel coordinate of the original optical flow information, v vib,IMU is the theoretical optical flow information caused by vibration.

[0068] It should be noted that the center of rotation of a flapping-wing aircraft can be estimated using the optical flow field centroid. Optical flow field centroid estimation is a technique for determining the centroid position of a moving target or flow field feature point by analyzing the optical flow field. The present invention can employ existing optical flow field centroid estimation methods, which will not be further described here.

[0069] Furthermore, according to the following formula 5, the sparse representation information of the vibration noise optical flow is aligned with the theoretical optical flow information to obtain the vibration information:

[0070]

[0071] Among them, v vib is the vibration information, β1∈(0,1), is the inverse short-time Fourier transform.

[0072] It should be noted that β1 is the fusion weight of theoretical optical flow information and sparse representation information, which can be set within [0, 1] according to actual needs, and the present invention does not limit this.

[0073] Furthermore, according to the following formula 6, the real optical flow information is separated from the original optical flow information based on the vibration information:

[0074] v motion =v raw -v vib ; (Formula 6)

[0075] Among them, v motion is the real optical flow information, v raw is the original optical flow information, v vib Vibration information.

[0076] In one embodiment, the real optical flow information is adaptively smoothed and confidence-weighted to obtain the final optical flow information, including: calculating the confidence level of the vibration region based on the vibration information; and performing confidence-based smoothing on the real optical flow information based on the confidence level of the vibration region to obtain the final optical flow information. It should be noted that the adaptive smoothing and confidence-weighted processing is used to suppress residual noise and thus improve the robustness of the real optical flow information.

[0077] The following will be combined with the attached Figure 3 The process of adaptive smoothing and confidence weighting is explained:

[0078] According to the following formula 7, based on the vibration information v vib (x,y), calculate the confidence c(x,y) of the vibration area:

[0079]

[0080] Among them, σ is a scale parameter, which can be set according to actual needs.

[0081] Furthermore, according to the following formula 8, based on the confidence c(x,y) of the vibration area, the real optical flow information v motion Perform confidence-based smoothing to obtain the final optical flow information v final:

[0082]

[0083] Among them, λ1 is the weight of the smoothing term, which can be set according to actual needs, and v is the formal variable in the process of optimizing the optical flow field. is the spatial gradient of the target optical flow field v.

[0084] It should be noted that flapping-wing aircraft have problems with wingbeat vibration and body elastic vibration, which are not conducive to continuous estimation of optical flow. The present invention can obtain accurate optical flow estimation through spatiotemporal filtering and vibration separation processing, adaptive smoothing and confidence weighted processing.

[0085] In one embodiment, based on the flight angular velocity information, the acceleration information is subjected to CHAME filtering processing to obtain true acceleration information, including: calculating the vibration main frequency information based on the angular velocity information; extracting the fundamental frequency information and harmonic information from the vibration main frequency information using fast Fourier transform; and based on the harmonic information, sequentially performing pre-low-pass filtering processing, multi-stage band-stop filtering processing, adaptive sliding average filtering processing, and motion acceleration and drift compensation processing on the acceleration information to obtain true acceleration information.

[0086] It should be noted that the method for calculating the vibration main frequency information in this embodiment is the same as that in the above embodiment, and will not be repeated here.

[0087] In one embodiment, the motion acceleration and drift compensation processing includes: performing zero-phase low-pass filtering on the acceleration information after adaptive sliding average filtering to extract the acceleration information after zero-phase low-pass filtering; using high-pass filtering to eliminate the integral drift in the acceleration information after zero-phase low-pass filtering to obtain real acceleration information.

[0088] The following will be combined with the attached Figure 4 The specific process of CHAME filtering is described as follows:

[0089] According to the following formula 9, the vibration main frequency information is converted from the time domain to the frequency domain:

[0090]

[0091] Among them, x[n] is the vibration main frequency information in the time domain, n is the length of the vibration main frequency information in the time domain, and X(k) is the vibration main frequency information in the frequency domain.

[0092] Furthermore, the maximum peak value in the amplitude spectrum corresponding to the main frequency information of the vibration in the frequency domain is found, and the corresponding frequency is the fundamental frequency f0. k =kf0, k=2,3,…,K. It should be noted that f0∈[20,30]Hz, fK ∈[100,500]Hz.

[0093] Furthermore, based on the harmonic information, the acceleration information is pre-low-pass filtered according to the following formula 10 to obtain the acceleration information after pre-low-pass filtering:

[0094]

[0095] Among them, the cutoff frequency of the low-pass filter is f c =2f k It should be noted that the pre-low-pass filter is used to suppress high-frequency noise to avoid the aliasing effect of the band-stop filter.

[0096] Furthermore, according to the following formula 11, the acceleration information after the pre-low-pass filtering is subjected to multi-stage band-stop filtering to obtain the acceleration information after the multi-stage band-stop filtering:

[0097]

[0098] Among them, f s is the sampling rate of the six-axis inertial sensor (usually ≥1kHz), β2∈(0.9,0.99), β2 is used to control the stopband width. When β2→1, the stopband becomes narrower. Each filter stage targets a harmonic frequency, and the cascaded filter suppresses the fundamental frequency and each harmonic.

[0099] Furthermore, according to the following formula 12, the acceleration information after the multi-stage band-stop filtering is subjected to adaptive sliding average filtering to obtain the acceleration information after the adaptive sliding average filtering a sma [n]:

[0100]

[0101] Among them, a notch [n] is the acceleration information after multi-stage band-stop filtering, λ is the attenuation rate, which is used to control the weight distribution, and w is the window length. The window length can be determined according to the following equation:

[0102] w[n]=clip(w base +γ·|a raw [n]|,w min ,w max );

[0103] Among them, w base is the basic window length, γ is the amplitude sensitivity coefficient, which can be set according to the needs, and clip(·) limits the window to [w min ,w max ], a raw [n] is the acceleration information.

[0104] Furthermore, the acceleration information a after adaptive sliding average processing is sma Perform zero-phase low-pass filtering to obtain the acceleration information a after zero-phase low-pass filtering motion :

[0105] a motion =filt(H lp (z),a sma );

[0106] Among them, H lp (z) is the cutoff frequency f c =10 Hz low-pass filter, filt(·) represents zero-phase filtering. It should be understood that bidirectional filtering can eliminate time delay.

[0107] Furthermore, high-pass filtering is used to eliminate the integral drift in the preliminary acceleration information to obtain the real acceleration information:

[0108] a final =a motion -HPF(v drift );

[0109] Among them, a final is the real acceleration information, a motion is the acceleration information after zero-phase low-pass filtering, and HPF(·) is a high-pass filter with a cutoff frequency of 0.1 Hz. drift It can be determined according to the following equation:

[0110]

[0111] Δt=1 / f s , f s Sampling rate of the six-axis inertial sensor.

[0112] It should be noted that since flapping-wing aircraft generate a 20-30 Hz wingbeat main frequency component and several hundred Hz high-order harmonic components of the body's elastic vibration during flight, the traditional IMU vibration reduction solution, without any filtering, will measure acceleration values of up to 6-7 gravitational accelerations, introducing significant body vibration noise. This noise will mask the aircraft's true attitude and motion acceleration, affecting the estimation of attitude, velocity, and position. The present invention introduces CHAME (Cascade Harmonic-Adaptive Motion Extraction) filtering. CHAME filtering obtains the main frequency and harmonic frequencies of the noise through frequency domain analysis. Based on this, a cascaded band-stop filter accurately filters out the 20-30 Hz wingbeat frequency and its high-order harmonics. Then, through adaptive sliding average filtering, the window length is dynamically adjusted according to the instantaneous amplitude of the vibration (the stronger the vibration, the larger the window), balancing noise suppression and signal real-time performance. Finally, zero-phase filtering is used to extract pure low-frequency motion acceleration, and drift compensation is used to ensure stability during long-term flight. While ensuring real-time performance, CHAME filtering can suppress vibration noise from 7-8g to below 0.1g, allowing the true motion state of the aircraft to be clearly presented.

[0113] In a specific embodiment, after adopting the flapping-wing aircraft positioning device of the present invention, the flapping-wing aircraft flew indoors for three minutes and forty seconds. The above three-axis position control of the aircraft was obtained through flight log analysis. Figure 5(a) 、 5(b) As shown in Figure 5(c), the error between the actual response of the flapping-wing aircraft in the X, Y, and Z directions and the setpoint of the flight control is within ±0.5m.

[0114] In one embodiment, Figure 6 The three-axis acceleration information measured by the six-axis inertial sensor of the flapping-wing aircraft is shown in Figure 6 It can be seen that the acceleration of the aircraft in the X direction is 3 to 4 g, and the acceleration in the Y and Z directions is 1 to 2 g. Figure 7 It is an evaluation index to measure the vibration of the aircraft. Figure 7 The indicators shown in the figure are at 2m / s 2 The vibration environment is excellent. The CHAME filtering proposed in this patent is very effective in filtering out wingbeat noise and body vibration noise.

[0115] In one embodiment, if Figure 2As shown, after the scale-normalized aircraft height information collected by the TOF sensor is scaled, the scale-normalized flight height information, the final optical flow information, and the real acceleration information are fused by Kalman filtering to obtain the three-axis position information of the flapping-wing aircraft. It should be understood that the three-axis position information is the position information of the aircraft in three-dimensional space. Scale normalization is a data preprocessing method whose purpose is to adjust the scale of the data to a uniform range for better analysis and processing. Kalman filter fusion is a multi-sensor data fusion technology based on the Kalman filter algorithm, which is widely used in target tracking, navigation positioning, signal processing and other fields. This implementation can use the existing Kalman filter fusion method for fusion, which will not be described here.

[0116] Figure 8 FIG. 1 is a flow chart of the indoor positioning method of a flapping-wing aircraft provided in an embodiment of the present application. Figure 8 As shown, the flapping-wing aircraft indoor positioning method is applied to a flapping-wing aircraft indoor positioning device, the device including: an optical flow sensor, a TOF sensor and a six-axis inertial sensor, the method including:

[0117] Step S81: receiving optical flow image information from an optical flow sensor, flight altitude information from a TOF sensor, and flight acceleration information and flight angular velocity information from a six-axis inertial sensor; wherein the optical flow sensor adopts an event camera; the event camera is used to collect optical flow image information;

[0118] Step S82: Based on the flight angular velocity information, the optical flow image information is subjected to spatiotemporal filtering and vibration separation processing to obtain real optical flow information; the real optical flow information is subjected to adaptive smoothing and confidence weighting processing to obtain final optical flow information; based on the flight angular velocity information, the acceleration information is subjected to CHAME filtering processing to obtain real acceleration information; the scale-normalized height information, the final optical flow information and the real acceleration information are subjected to Kalman filtering fusion to obtain the three-axis position information of the flapping-wing aircraft.

[0119] It should be understood that the specific implementation process of each step in this embodiment has been described in detail in the above device embodiment, and for the sake of brevity, it will not be repeated here.

[0120] In one embodiment, based on the flight angular velocity information, the optical flow image information is subjected to spatiotemporal filtering and vibration separation processing to obtain real optical flow information, including: calculating the vibration main frequency information based on the angular velocity information; performing time window filtering on the optical flow image information according to the vibration main frequency information to obtain filtered optical flow image data; performing optical flow estimation on the filtered optical flow image data to obtain original optical flow information; based on the angular velocity information, performing frequency domain analysis and motion compensation processing on the original optical flow information to obtain vibration information; and separating the real optical flow information from the original optical flow information based on the vibration information.

[0121] In one embodiment, based on the angular velocity information, the original optical flow information is subjected to frequency domain analysis and motion compensation processing to obtain vibration information, including: performing a pixel-by-pixel short-time Fourier transform on the original optical flow information, and filtering the Fourier transformed optical flow information according to a set bandwidth to obtain sparse representation information of the vibration noise optical flow; based on the angular velocity information, calculating the theoretical optical flow information; and aligning the sparse representation information of the vibration noise optical flow with the theoretical optical flow information to obtain vibration information.

[0122] In one embodiment, the real optical flow information is adaptively smoothed and confidence-weighted to obtain the final optical flow information, including: calculating the confidence of the vibration area based on the vibration information; and performing confidence-based smoothing on the real optical flow information based on the confidence of the vibration area to obtain the final optical flow information.

[0123] In one embodiment, based on the flight angular velocity information, the acceleration information is subjected to CHAME filtering processing to obtain true acceleration information, including: calculating the vibration main frequency information based on the angular velocity information; extracting the fundamental frequency information and harmonic information from the vibration main frequency information using fast Fourier transform; and based on the harmonic information, sequentially performing pre-low-pass filtering processing, multi-stage band-stop filtering processing, adaptive sliding average filtering processing, and motion acceleration and drift compensation processing on the acceleration information to obtain true acceleration information.

[0124] In one embodiment, the motion acceleration and drift compensation processing includes: performing zero-phase low-pass filtering on the acceleration information after adaptive sliding average filtering to extract the acceleration information after zero-phase low-pass filtering; using high-pass filtering to eliminate the integral drift in the acceleration information after zero-phase low-pass filtering to obtain real acceleration information.

[0125] Figure 9 : is a schematic block diagram of an electronic terminal provided in an embodiment of the present application. Figure 9As shown, the electronic terminal includes: at least one processor 901, a memory 902, at least one network interface 903 and a user interface 905. The various components in the device are coupled together via a bus system 904. It is understood that the bus system 904 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 904 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 9 In the text, various buses are labeled as bus systems.

[0126] The user interface 905 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.

[0127] It will be appreciated that the memory 902 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM) or a programmable read-only memory (PROM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memories.

[0128] The memory 902 in the embodiment of the present invention is used to store various categories of data to support the operation of the electronic terminal 900. Examples of such data include: any executable program for operating on the electronic terminal 900, such as an operating system 9021 and an application 9022; the operating system 9021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 9022 can include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The implementation of the flapping-wing aircraft indoor positioning method provided in the embodiment of the present invention can be included in the application 9022.

[0129] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 901. Processor 9401 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in processor 901 or by software instructions. The above processor 901 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 901 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor 901 may be a microprocessor or any conventional processor. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory. The processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0130] In an exemplary embodiment, the electronic terminal 900 may be one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), and complex programmable logic devices (CPLDs) for executing the aforementioned method.

[0131] According to the method provided in the embodiment of the present application, the present application also provides a computer program product, which includes: a computer program code, which, when executed on a computer, causes the computer to execute Figure 8 The indoor positioning method of a flapping-wing aircraft in the illustrated embodiment.

[0132] According to the method provided in the embodiment of the present application, the present application also provides a computer-readable storage medium, which stores a program code, and when the program code is run on a computer, the computer executes Figure 8 The indoor positioning method of a flapping-wing aircraft in the illustrated embodiment.

[0133] As used in this specification, the terms "component," "module," "system," and the like are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. By way of illustration, both an application running on a computing device and a computing device can be a component. One or more components can reside in a process and / or an execution thread, and a component can be located on a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures stored thereon. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component on a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0134] Those skilled in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0135] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0137] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0138] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0139] In the above embodiments, the functions of each functional unit can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When the computer program instructions (program) are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. Available media may be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., high-density digital video discs (DVDs), or semiconductor media (e.g., solid state disks (SSDs)).

[0140] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media that can store program codes.

[0141] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

[0142] In summary, the present application provides a flapping-wing aircraft indoor positioning device, method, terminal, medium and flapping-wing aircraft. The indoor positioning device includes: an optical flow sensor equipped with an event camera, a TOF sensor, a six-axis inertial sensor and a control module. The event camera collects optical flow image information, and then separates the body vibration and real movement through spatiotemporal filtering and vibration separation processing, and then obtains accurate optical flow information through adaptive smoothing and confidence weighting processing. The acceleration information obtained by the six-axis inertial sensor is subjected to CHAME filtering processing to suppress the body vibration noise to obtain real acceleration information. Finally, the height information, optical flow information and real acceleration information collected by the TOF sensor after scale normalization are Kalman filtered and fused to obtain the three-axis position information of the flapping-wing aircraft. The present application realizes more accurate indoor position perception and positioning of flapping-wing aircraft. Therefore, the present application effectively overcomes the various shortcomings in the existing technology and has high industrial utilization value.

[0143] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. A flapping-wing aircraft indoor positioning device, characterized in that: The invention is provided on a flapping-wing aircraft, and comprises: Optical flow sensor, TOF sensor, six-axis inertial sensor and control module; Wherein, the optical flow sensor adopts an event camera; the event camera is used to collect optical flow image information; The TOF sensor is used to collect flight altitude information; the six-axis inertial sensor is used to collect flight acceleration information and flight angular velocity information; The control module is used to perform spatiotemporal filtering and vibration separation processing on the optical flow image information based on the flight angular velocity information to obtain real optical flow information; perform adaptive smoothing and confidence weighting processing on the real optical flow information to obtain final optical flow information; perform CHAME filtering processing on the acceleration information based on the flight angular velocity information to obtain real acceleration information; and perform Kalman filtering fusion on the scale-normalized height information, the final optical flow information, and the real acceleration information to obtain the three-axis position information of the flapping-wing aircraft.

2. The indoor positioning device for flapping-wing aircraft according to claim 1, characterized in that: Based on the flight angular velocity information, the optical flow image information is subjected to spatiotemporal filtering and vibration separation processing to obtain real optical flow information, including: Calculating vibration main frequency information based on the angular velocity information; performing time window filtering on the optical flow image information according to the vibration main frequency information to obtain filtered optical flow image data; performing optical flow estimation on the filtered optical flow image data to obtain original optical flow information; Based on the angular velocity information, frequency domain analysis and motion compensation processing are performed on the original optical flow information to obtain vibration information; based on the vibration information, real optical flow information is separated from the original optical flow information.

3. The indoor positioning device for flapping-wing aircraft according to claim 2, characterized in that: Based on the angular velocity information, frequency domain analysis and motion compensation processing are performed on the original optical flow information to obtain vibration information, including: Perform pixel-by-pixel short-time Fourier transform on the original optical flow information, and filter the Fourier transformed optical flow information according to the set bandwidth to obtain sparse representation information of the vibration noise optical flow; Based on the angular velocity information, theoretical optical flow information is calculated; and the sparse representation information of the vibration noise optical flow is aligned with the theoretical optical flow information to obtain vibration information.

4. The indoor positioning device for flapping-wing aircraft according to claim 2, characterized in that: Adaptively smoothing and confidence weighting the real optical flow information to obtain the final optical flow information, including: Calculating the confidence level of the vibration region based on the vibration information; Based on the confidence of the vibration area, the real optical flow information is smoothed based on the confidence to obtain the final optical flow information.

5. The indoor positioning device for flapping-wing aircraft according to claim 1, characterized in that: Based on the flight angular velocity information, the acceleration information is subjected to CHAME filtering to obtain true acceleration information, including: Calculating vibration main frequency information based on the angular velocity information; Extracting fundamental frequency information and harmonic wave information from the vibration main frequency information using fast Fourier transform; Based on the harmonic information, the acceleration information is sequentially subjected to pre-low-pass filtering, multi-stage band-stop filtering, adaptive sliding average filtering, and motion acceleration and drift compensation to obtain true acceleration information.

6. The indoor positioning device for flapping-wing aircraft according to claim 5, characterized in that: The motion acceleration and drift compensation process includes: The acceleration information after the adaptive sliding average filtering is subjected to zero-phase low-pass filtering to extract the acceleration information after the zero-phase low-pass filtering; the integral drift in the acceleration information after the zero-phase low-pass filtering is eliminated by high-pass filtering to obtain the real acceleration information.

7. A method for indoor positioning of a flapping-wing aircraft, characterized in that: The invention is applied to an indoor positioning device for a flapping-wing aircraft, the device comprising an optical flow sensor, a TOF sensor and a six-axis inertial sensor, and the method comprising: receiving optical flow image information from an optical flow sensor, flight altitude information from a TOF sensor, and flight acceleration information and flight angular velocity information from a six-axis inertial sensor; wherein the optical flow sensor adopts an event camera; and the event camera is used to collect optical flow image information; Based on the flight angular velocity information, the optical flow image information is subjected to spatiotemporal filtering and vibration separation processing to obtain real optical flow information; the real optical flow information is subjected to adaptive smoothing and confidence weighting processing to obtain final optical flow information; based on the flight angular velocity information, the acceleration information is subjected to CHAME filtering processing to obtain real acceleration information; the scale-normalized height information, the final optical flow information and the real acceleration information are subjected to Kalman filtering fusion to obtain the three-axis position information of the flapping-wing aircraft.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to claim 7 is implemented.

9. An electronic terminal comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the method of claim 7.

10. A flapping-wing aircraft, characterized in that: The flapping-wing aircraft is provided with the flapping-wing aircraft indoor positioning device according to any one of claims 1 to 6.

Images