Indoor omnidirectional positioning method for aerial photography device of aircraft

Abstract

The invention belongs to the technical field of indoor positioning of aircrafts, and particularly relates to an indoor omnidirectional positioning method for an aerial photographing device of an aircraft, which comprises the following steps of: connecting at least one unmanned aerial vehicle wireless self-charging parking platform with at least one rotor wing vertical take-off and landing aircraft, connecting at least one three-axis cradle head, arranging on the vertical take-off and landing aircraft, and connecting at least one binocular camera module group; the unmanned aerial vehicle wireless self-charging parking platform is characterized in that the unmanned aerial vehicle wireless self-charging parking platform guides an aircraft to land autonomously and be charged wirelessly, the vertical take-off and landing aircraft has vertical take-off and landing, hovering and stable flying performance, the three-axis holder reaches the shooting angle of vertical parallel omni-directional movement, and the unmanned aerial vehicle wireless self-charging parking platform is arranged on the three-axis holder. According to the omni-directional intelligent positioning method, based on a global positioning system (GNSS), UWB (Ultra Wideband) and a binocular vision positioning method are fused, and the technical effect of intelligent positioning shooting of vertical parallel omni-directional movement is achieved.

Description

Technical Field

[0003] The invention belongs to the technical field of indoor positioning of aircraft, and in particular relates to an indoor omnidirectional positioning method for an aircraft aerial photography device. Background Art

[0005] China Utility Model Patent Authorization Announcement No. CN 221327285 U, a multifunctional smart mirror capable of reflecting front and back facade images, discloses: a mobile base cabinet and a mirror frame base, and a mobile micro-suspended camera parking platform for parking and placing the mobile micro-suspended camera. The above utility model proposes a new technical solution suitable for practical use for the shape, structure or combination of the product. However, there is no specific implementation plan for the indoor omnidirectional positioning method of the aircraft aerial photography device. If the following general method is adopted:

[0006] 1. Combination method of global positioning system (GNSS) and inertial measurement unit (IMU): As we all know, unmanned aerial vehicles, also known as "drones", abbreviated as "UAV", are unmanned aircraft controlled by radio remote control equipment and self-provided program control devices. Aircraft are widely used in military, agriculture, photography, smart home and other fields. For example, aircraft are used for reconnaissance, mapping, mounting shooting devices, and aerial photography.

[0007] Traditional UAV positioning methods are a combination of the Global Positioning System (GNSS) and the Inertial Measurement Unit (IMU). GNSS uses artificial satellites to achieve high-precision radio navigation positioning. It includes pseudo-range single-point positioning, carrier phase positioning and real-time differential positioning. Pseudo-range measurement calculates the position by measuring the distance from the satellite to the receiver, while the navigation message is used to calculate the current position of the satellite and the signal transmission time, so that the GNSS receiver can determine its own position. IMU can detect the movement, acceleration and angle and direction changes of objects in space, which is very useful for tracking object movement, monitoring vibration and impact, and providing navigation information on mobile platforms such as vehicles, aircraft and ships. It works in conjunction with the GNSS system to improve the accuracy of position and direction.

[0008] Drones can use GNSS to obtain high-precision positioning and navigation results in outdoor environments with good observation conditions. Although IMU can also be used for position tracking indoors or in environments with weak GPS signals, it also has many limitations and needs to work in conjunction with other sensors to create detailed location maps. For example, when the satellite signal is blocked by buildings, resulting in an insufficient number of observation satellites received, the drone will be unable to meet requirements in complex environments or affected by obstacles, and may even be completely unsuitable for indoor flights.

[0009] 2. SLAM (Simultaneous Localization and Mapping) is a technology that estimates the position and posture of drones and builds environmental maps by processing sensor data (such as laser radar, camera, etc.). Its core issues include data association, state estimation and map representation, etc., which are self-directed flight, obstacle avoidance and map construction tasks, and improve the intelligence level of drones. However, in environments with unclear structural features, such as corridors, tunnels, basements or high-rise buildings, the performance of laser SLAM technology may be affected, and the high price of laser radar leads to a relatively high cost of adopting laser SLAM technology.

[0010] 3. Computer vision technology in the field of smart home can extract useful information from image or video data, simulate the human visual system to identify indoor people, objects and environmental features, and use panoramic shooting technology to record indoor environment, including being able to capture 360-degree images, people's activities, generate panoramic photos or videos, and display the processed panoramic images or videos on the control panel of the smart home, mobile phone APP or other smart terminals. Users can freely view the indoor environment through operations such as sliding and zooming, providing a full range of visual experience; the system can also provide interactive functions, such as clicking on people or objects to obtain more information or perform related operations, such as adjusting lights, controlling home appliances, etc.

[0011] However, the above-mentioned panoramic shooting technology can only be installed and fixed in a suitable position such as a ceiling, wall or furniture. For scenes with complex environments and many obstacles, such as basements or high-rise buildings in shopping malls, performance venues, offices, classrooms, etc., it is impossible to achieve the effect required by the vertical parallel omnidirectional moving shooting technology.

[0012] In summary, there is a need to improve the defects of GNSS that cannot achieve the required effect when the drone is in a complex environment or affected by obstacles, and is even completely unsuitable for indoor flight; or SLAM lidar may be affected in an environment where the structural features are not obvious; computer vision panoramic shooting technology cannot achieve the required effect of vertical parallel omnidirectional moving shooting technology in various indoor scenes with complex environments and many obstacles, such as basements or high-rise buildings in shopping malls, performance venues, offices, and teaching. Summary of the invention

[0014] In view of the above problems, the purpose of the present invention is to provide an indoor omnidirectional positioning method for an aircraft aerial photography device, so as to solve the problem that in various indoor scenes with complex environments and many obstacles, such as basements or high-rise buildings in shopping malls, performance venues, offices, and classrooms, it is impossible to achieve the effect required by vertical parallel omnidirectional movement shooting technology, and to propose a new technical solution to solve the above technical problems.

[0015] The proposed technical solution: an indoor omnidirectional positioning method for an aircraft aerial photography device, comprising at least one wireless self-charging parking platform for unmanned aerial vehicles, connected to at least one rotor vertical take-off and landing aircraft, connected to at least one three-axis gimbal, installed on the vertical take-off and landing aircraft, connected to at least one binocular camera module group, installed on the three-axis gimbal, and equipped with an omnidirectional intelligent positioning method, characterized in that: the wireless self-charging parking platform for unmanned aerial vehicles guides the aircraft to autonomously land and charge wirelessly, the vertical take-off and landing aircraft has vertical take-off and landing, hovering, and stable flight capabilities, the three-axis gimbal can achieve a vertical parallel omnidirectional moving shooting angle, and the omnidirectional intelligent positioning method is based on the global positioning system (GNSS) fusion UWB (Ultra Wideband) + binocular vision positioning method to achieve the vertical parallel omnidirectional moving intelligent positioning shooting technical effect;

[0016] The wireless self-charging parking platform for unmanned aerial vehicles comprises a lifting platform box body for unmanned aerial vehicles, and a wireless charging platform, a power charger connected to the wireless self-charging parking platform for unmanned aerial vehicles, a wireless charger metal coil transmitting end is installed on a spring pressure sensor, and a current drives the transmitting coil to generate an alternating magnetic field. When the autonomous landing method of image navigation positioning is used to guide the unmanned aerial vehicle to land on the wireless charger metal coil transmitting end, through the wireless charger metal coil receiving end on the landing gear, when the unmanned aerial vehicle and the wireless charger are close to and aligned, the receiving coil will sense the magnetic field generated by the transmitting coil, and the receiving end coil will receive these electromagnetic signals and convert them into current, thereby achieving wireless charging to charge the battery of the aircraft, so as to keep the power supply for long-term use;

[0017] The rotor vertical take-off and landing aircraft is used to generate flight power, mainly relying on the rotor to generate lift. The rotor provides lift thrust through rotation, so that the aircraft can hover in the air and take off and land vertically, with the characteristics of high-speed flight and hovering stability. It includes an aircraft body, four sets of arm spiral blades are connected on the left and right sides of the aircraft body, two sets of landing gears are connected on both sides of the bottom of the aircraft body, and the landing gears are connected with a wireless charger metal coil receiving coil. When the engine is started, the rotor starts to rotate, generating a strong airflow. According to Bernoulli's principle, when the airflow passes through the upper surface of the rotor, the speed increases and the pressure decreases, generating an upward lift; when passing through the lower surface of the rotor, the speed decreases and the pressure increases, generating a downward airflow. By adjusting the rotation speed and blade angle of the rotor, the ascent and descent of the aircraft can be controlled;

[0018] The front of the aircraft body is connected to a three-axis gimbal, and the three-axis gimbal also includes a gimbal fixing bracket connected to a heading axis, which is responsible for horizontal stability and maintains horizontal stability by controlling the left and right rotation of the gimbal, a pitch axis, which maintains vertical stability by controlling the up and down movement of the gimbal, and a roll axis, which is responsible for resisting the lateral tilt of the gimbal and maintains stability by controlling the lateral rotation of the gimbal. Accelerometers MEMS gyroscopes are installed on the heading axis, roll axis and pitch axis as IMU sensors. The accelerometer is used to detect and respond to changes in angular velocity. The MEMS gyroscope is based on the principle of conservation of angular momentum and can achieve angular velocity measurement under conditions of high precision, small size and strong anti-interference. The Coriolis force and other physical effects convert mechanical motion into electrical signals, thereby realizing the measurement of angular velocity; each axis is equipped with a corresponding servo motor and drive circuit. The servo motors are responsible for the rotation in the three directions of front and back, left and right, and up and down. The stable adjustment of the gimbal is realized by accurately controlling the speed and direction of these motors. Combined with the IMU feedback system, it is responsible for collecting the attitude information of the object, including the data in the three directions of roll, pitch and heading, and real-time monitoring and adjustment of the attitude of the gimbal to realize the stability of the attitude control device of the target object. Ensure the clarity and stability of the shooting picture. In a dynamic environment, it realizes the precise control and stability of the object's attitude, thereby providing high-quality shooting effects in various application scenarios;

[0019] The three-axis gimbal also includes a binocular camera module group, which is installed on the three-axis gimbal, usually called a Compact Camera Module (CCM), and is used to capture and record images or videos. Its main components include a binocular camera (Lens), which can focus light onto a camera sensor. Key indicators include focal length, aperture, field of view, etc. The focal length determines the image size, field of view, and depth of field; the aperture controls the amount of light passing through the lens, affecting the brightness and depth of field of the image; the field of view reflects the shooting range of the picture, directly affecting the clarity and color performance of the image; the image sensor (Sensor), which is a CMOS (complementary metal oxide semiconductor), is responsible for converting the light focused by the lens into electrical signals. There are millions to tens of millions of photodiodes on the surface of the sensor. Each photodiode generates an electric charge when exposed to light, converts the light into an electrical signal, and then generates a digital image; the image processor (ISP, Image Signal Processor) Processor, which is used to process the raw image data captured by the camera sensor, including color correction, contrast enhancement, noise reduction, etc., to improve the quality of the final image; and other auxiliary components, flexible printed circuit board (FPC), connecting chip, responsible for electrical signal transmission, autofocus motor (such as VCM, Voice Coil Motor), by controlling the current to adjust the distance from the lens to the sensor to achieve autofocus function, infrared filter (IR Cut Filter), filter out infrared light to ensure accurate image color, flash, provide additional light source, improve shooting effect in low light environment, its working principle is optical imaging: light enters the camera module through the lens and is focused by the lens system to form an optical image of the actual scene; photoelectric conversion: the optical image is projected onto the photosensitive element of the camera module (such as CMOS or CCD sensor), and the photosensitive element converts the light signal into an electrical signal; signal processing: the camera module amplifies and filters the electrical signal to remove noise, adjust color balance, enhance details, etc., to obtain a clear and accurate image;

[0020] The omnidirectional intelligent positioning method includes an intelligent omnidirectional arrangement based on the global positioning system (GNSS) + wireless pulse communication technology UWB (Ultra Wideband) fused with a binocular vision positioning method. When the global positioning system (GNSS) signal is blocked by buildings, resulting in an insufficient number of received observation satellites, wireless transmission is achieved by intelligently selecting the ultra-narrow pulse communication technology UWB that sends and receives pulses below the microsecond level. Due to the extremely short pulse time width, UWB can achieve high-precision distance measurement, thereby performing high-precision positioning. At the same time, a binocular camera device accelerometer MEMS gyroscope is used as an IMU sensor, and the binocular vision positioning method is fused in a tightly coupled manner, and the scale information of the feature points is obtained using the binocular parallax method. The pyramid LK optical flow algorithm is used to track feature points, and the RANSAC (random sampling consistency) algorithm is used to remove mismatched feature points; at the same time, the inertial measurement data is pre-integrated to keep the data synchronization between sensors; a sliding window is constructed to select and marginalize key frames, and the position, attitude, speed, gravity direction and drift of the inertial device of the drone are estimated by graph optimization. Finally, the indoor positioning of the drone in a dynamic environment and the closed-loop detection and re-indoor positioning of the previously reached positions are designed, which effectively improves the high-precision indoor positioning of the drone and achieves the effect required by the vertical parallel omnidirectional mobile shooting technology;

[0021] The GNSS+UWB positioning method:

[0022] (1) Pre-set up UAV fake base equipment points and UAV mobile stations;

[0023] (2) The UAV pseudo-base equipment points surround the UAV mobile station according to a preset spatial distribution;

[0024] (3) The UAV mobile station detects the number of GNSS satellite signals currently received in real time, and compares the number of GNSS satellite signals detected with a preset threshold;

[0025] (4) When the number of GNSS satellite signals detected is greater than a threshold, the UAV mobile station uses a preset GNSS positioning module to receive positioning information sent by the base station and performs real-time coordinate differential analysis;

[0026] (5) When the number of GNSS satellite signals detected is less than a threshold, the UAV mobile station uses a preset UWB positioning module to measure distance via UWB and receive positioning information sent by a preset UAV pseudo-base equipment point to calculate the real-time coordinates of the UAV mobile station;

[0027] (6) The main construction principles of the drone fake base equipment include deception equipment and signal delay technology;

[0028] The deception device can generate a fake Global Positioning System (GPS) signal, adjust the satellite signal delay, forge the GPS signal, and cause the drone to misjudge its own location;

[0029] The signal delay technology described above: The spoofing device adjusts the delay of each branch satellite signal to forge the GPS signal, causing the drone to receive incorrect location information, thereby changing its flight path;

[0030] The binocular vision-inertial fusion positioning method:

[0031] (1) Using binocular vision and IMU as sensors, we can obtain data from sensors with finer precision and inertial sensors, extract ORB features from the binocular images, and determine the matching relationship between the left and right images.

[0032] (2) As shown in Figure 5, the imaging model of the binocular camera, point P in the figure is the position of the feature point of the object, OL and OR are the left camera and the right camera respectively. The coordinates of the feature point P in the camera coordinate system obtained by the binocular parallax method are:

[0033] Xc=b(uL-uO) / d

[0034] Yc=b(vL-vO) / d

[0035] Zc=bf / d

[0036] (Where [Xc, Yc, Zc]T is the scaled camera coordinate of point P; f is the focal length of the camera, uL and vL are the horizontal and vertical coordinates of point PL in the camera coordinate system, b is the baseline, i.e. the actual distance between the binocular centers; d is the parallax, d = uL -uR; (uO, vO) are the coordinates of the optical center of the left camera);

[0037] (3) Use the pyramid LK optical flow method to track features and determine the matching relationship between the left and right images;

[0038] The process of the pyramid LK optical flow algorithm is as follows: (1) establish a pyramid - (2) iterate and calculate the optical flow - (3) convergence condition

[0039] (1) Building a pyramid: First, perform multi-scale pyramid decomposition on the input image through Gaussian blur and downsampling operations to form a pyramid hierarchy. The resolution of each layer of the image is lower than that of the previous layer, but it covers a larger field of view. For example, a 5-layer image pyramid is constructed for the first and second frames of the image. The horizontal and vertical coordinates of the feature points detected in the first frame of the original image are reduced to 1 / 16 of the original coordinates, that is, the coordinates of the corner points at the top layer of the pyramid of the first frame of the original image are obtained. The coordinates of the corner points at the top layer of the pyramid of the first frame of the original image are used as the predicted coordinates of the corner points at the top layer of the pyramid of the second frame of the image.

[0040] (2) Iteratively calculate the optical flow using the Lucas-Kanade optical flow algorithm. Since the image resolution of each layer is low, the motion displacement is also reduced accordingly, which makes it possible to accurately estimate the optical flow at each layer. Then, the optical flow calculated at each layer is passed to the next layer as the initial value for the calculation of the next layer, and then the predicted values ​​of the corner point coordinates of the second frame are matched and corrected.

[0041] (3) Convergence condition. The iteration process will continue until the convergence condition is reached. The convergence condition can be that the change in optical flow is less than a certain threshold, or the change in the cost function is less than a certain threshold. This ensures the stability and accuracy of the optical flow calculation. If the bottom layer of the image pyramid is not reached, the horizontal and vertical coordinates of the feature points in the current pyramid layer of the first frame image and the horizontal and vertical coordinates of the corner points of the current pyramid layer of the second frame image corrected by the LK optical flow method are multiplied by 2 to obtain the horizontal and vertical coordinates of the feature points of the next layer of the pyramid layer of the first frame image and the horizontal and vertical coordinates of the predicted feature points of the next layer of the pyramid of the second frame image. Otherwise, return (2);

[0042] (4) In applications, the pyramid LK optical flow algorithm is often used in video processing, motion detection and tracking, and other fields. In order to optimize the performance of the algorithm, select an appropriate integral window size. A smaller window is suitable for detail preservation and occlusion, and a larger window is suitable for fast motion. Adjust the number of pyramid layers. Increasing the number of pyramid layers can improve the robustness of the algorithm.

[0043] The present invention proposes an indoor omnidirectional positioning method for an aircraft aerial photography device. The method adopts a wireless self-charging parking platform for an unmanned aerial vehicle, is connected to a rotor vertical take-off and landing aircraft, is connected to a three-axis gimbal, is connected to a binocular camera module group, and is equipped with an omnidirectional intelligent positioning method to realize the positioning of the unmanned aerial vehicle in a complex environment. When the satellite signal is affected, the positioning method based on the global positioning system (GNSS) fusion UWB (Ultra Wideband) + binocular vision can overcome scenes with many obstacles in complex environments, such as basements or high-rise buildings in shopping malls, performance venues, offices, and teaching, and other indoor scenes, and achieve the intelligent positioning and shooting technical effect of vertical parallel omnidirectional movement. The method is a low-cost indoor real-time positioning method for unmanned aerial vehicles that can improve production efficiency and is easy to implement. The method can be promoted and applied to other robots that require precise indoor positioning, and effectively solves the high-precision defect of indoor positioning of unmanned aerial vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of an indoor omnidirectional positioning method for an aircraft aerial photography device;

[0046] Figure 2 Schematic diagram of the wireless self-charging docking station for drones;

[0047] Figure 3 It is a schematic diagram of a rotary wing vertical take-off and landing aircraft;

[0048] Figure 4 It is a schematic diagram of a three-axis gimbal binocular camera module group;

[0049] Figure 5 It is a schematic diagram of the imaging model of the binocular camera of the omnidirectional intelligent positioning method;

[0050] Figure 6 It is the flow chart of the pyramid LK optical flow algorithm of the omnidirectional intelligent positioning method;

[0051] In the figure: UAV wireless self-charging parking platform 1, UAV lifting platform box 10, wireless charging platform 11, wireless charger metal coil transmitter 110, spring pressure sensor 111, rotor vertical take-off and landing aircraft 2, aircraft body 20, four sets of arm spiral blades 21, two sets of landing gear 22, wireless charger metal coil receiver 23, three-axis gimbal 3, gimbal fixing bracket 30, heading axis 301, pitch axis 302, roll axis 303, accelerometer MEMS gyroscope 31, servo motor and drive circuit 32, IMU feedback system 33, binocular camera module group 4, binocular camera 40, image sensor 41, image processor 42, omnidirectional intelligent positioning method 5. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0054] Specific embodiments of the proposed technical solution: Figure 1 As shown, an indoor omnidirectional positioning method for an aircraft aerial photography device comprises at least one unmanned aerial vehicle wireless self-charging parking platform 1, connected to at least one rotor vertical take-off and landing aircraft 2, connected to at least one three-axis gimbal 3, installed on the vertical take-off and landing aircraft 2, connected to at least one binocular camera module group 4, installed on the three-axis gimbal 3, and applying an omnidirectional intelligent positioning method 5. The device is characterized in that: the unmanned aerial vehicle wireless self-charging parking platform 1 guides the aircraft to autonomously land and charge wirelessly, the vertical take-off and landing aircraft 2 has vertical take-off and landing, hovering, and stable flight capabilities, the three-axis gimbal 3 achieves a vertical parallel omnidirectional moving shooting angle, and the omnidirectional intelligent positioning method 5 is based on the global positioning system (GNSS) fusion UWB (Ultra Wideband) + binocular vision positioning method to achieve the vertical parallel omnidirectional moving intelligent positioning shooting technical effect;

[0055] like Figure 2 As shown, the UAV wireless self-charging parking platform 1 includes a UAV lifting platform box body 10, and a wireless charger metal coil transmitter 11 arranged on the charging platform, which is connected to the power supply of the UAV wireless self-charging parking platform 1, and the wireless charger metal coil transmitter 110 is installed on the spring pressure sensor 111. The current drives the transmitter coil to generate an alternating magnetic field. When the autonomous landing method of image navigation positioning is used to guide the UAV to land on the wireless charger metal coil transmitter 11, through the wireless charger metal coil receiving coil on the landing gear, when the UAV and the wireless charger are close and aligned, the receiving coil will sense the magnetic field generated by the transmitting coil, and the receiving end coil receives these electromagnetic signals and converts them into current, thereby achieving wireless charging to charge the battery of the aircraft, so as to keep the power supply for long-term use;

[0056] like Figure 3As shown, the rotor vertical take-off and landing aircraft 2 is used to generate flight power, mainly relying on the rotor to generate lift. The rotor provides lift thrust through rotation, so that the aircraft can hover in the air and take off and land vertically, with the characteristics of high-speed flight and hovering stability. It includes an aircraft body 20, four sets of arm spiral blades 21 are connected on the left and right sides of the aircraft body 20, two sets of landing gears 22 are connected on both sides of the bottom of the aircraft body 20, and the landing gear 22 is connected to a wireless charger metal coil receiving coil 23. When the engine is started, the rotor begins to rotate, generating a strong airflow. According to Bernoulli's principle, when the airflow passes through the upper surface of the rotor, the speed increases and the pressure decreases, generating an upward lift; when passing through the lower surface of the rotor, the speed decreases and the pressure increases, generating a downward airflow. By adjusting the rotation speed of the rotor and the blade angle, the ascent and descent of the aircraft can be controlled;

[0057] like Figure 4 As shown, the aircraft body 20 is connected to a three-axis gimbal 3 in the front. The three-axis gimbal 3 also includes a gimbal fixing bracket 30 connected to a heading axis 301, which is responsible for horizontal stability and maintains horizontal stability by controlling the left and right rotation of the gimbal, a pitch axis 302, which maintains vertical stability by controlling the up and down movement of the gimbal, and a roll axis 303, which is responsible for resisting the lateral tilt of the gimbal and maintains stability by controlling the lateral rotation of the gimbal. Accelerometers MEMS gyroscopes 31 are installed on the heading axis 301, the roll axis 302 and the pitch axis 303 as IMU sensors. The accelerometer is used to detect and respond to changes in angular velocity. The MEMS gyroscope is based on the principle of conservation of angular momentum and can achieve high precision, small size and strong anti-interference performance. Angular velocity is measured under certain conditions, and mechanical motion is converted into electrical signals by using physical effects such as the Coriolis force, thereby realizing the measurement of angular velocity; each axis is equipped with a corresponding servo motor and drive circuit 32, and the servo motors are responsible for the rotation in the three directions of front and back, left and right, and up and down, respectively. The stable adjustment of the gimbal is realized by accurately controlling the speed and direction of these motors. In combination with the IMU feedback system 33, it is responsible for collecting the attitude information of the object, including data in the three directions of roll, pitch and heading, and real-time monitoring and adjusting the attitude of the gimbal to realize the device for stabilizing the attitude control of the target object. The device ensures the clarity and stability of the shooting picture, realizes the precise control and stabilization of the attitude of the object in a dynamic environment, and thus provides high-quality shooting effects in various application scenarios;

[0058] like Figure 4As shown, the three-axis gimbal 3 also includes a binocular camera module group 4, which is installed on the three-axis gimbal 3 and is usually called a Compact Camera Module (CCM). It is used to capture and record images or videos. Its main components include a binocular camera (Lens) 40, which can focus light onto a camera sensor. Key indicators include focal length, aperture, field of view, etc. The focal length determines the image size, field of view size and depth of field; the aperture controls the amount of light passing through the lens, affecting the brightness and depth of field of the image; the field of view reflects the shooting range of the picture, directly affecting the clarity and color performance of the image; the image sensor (Sensor) 41 is a CMOS (complementary metal oxide semiconductor), which is responsible for converting the light focused by the lens into electrical signals. There are millions to tens of millions of photodiodes on the surface of the sensor. Each photodiode generates an electric charge when exposed to light, converts the light into an electrical signal, and then generates a digital image; an image processor (ISP, Image Signal Processor) Processor) 42, used to process the raw image data captured by the camera sensor, including color correction, contrast enhancement, noise reduction, etc., to improve the quality of the final image; and other auxiliary components, flexible printed circuit board (FPC), connecting chip, responsible for electrical signal transmission, auto focus motor (such as VCM, Voice Coil Motor), by controlling the current to adjust the distance from the lens to the sensor to achieve the auto focus function, infrared filter (IR Cut Filter), filter out infrared light to ensure the image color accuracy, flash, provide additional light source, improve the shooting effect in low light environment, its working principle is optical imaging: light enters the camera module through the lens, and is focused by the lens system to form an optical image of the actual scene; photoelectric conversion: the optical image is projected onto the photosensitive element (such as CMOS or CCD sensor) of the camera module, and the photosensitive element converts the light signal into an electrical signal; signal processing: the camera module amplifies and filters the electrical signal to remove noise, adjust color balance, enhance details, etc., to obtain a clear and accurate image;

[0059] like Figure 5As shown, the omnidirectional intelligent positioning method 5 includes an intelligent omnidirectional arrangement based on the global positioning system (GNSS) + wireless pulse communication technology UWB (Ultra Wideband) fused with a binocular vision positioning method. When the global positioning system (GNSS) signal is blocked by buildings, resulting in an insufficient number of received observation satellites, the wireless transmission is achieved by intelligently selecting the ultra-narrow pulse communication technology UWB that sends and receives sub-microsecond levels. Due to the extremely short pulse time width, UWB can achieve high-precision distance measurement, thereby performing high-precision positioning. At the same time, a binocular camera device accelerometer MEMS gyroscope is used as an IMU sensor to fuse the binocular vision positioning method, and the fusion is performed in a tightly coupled manner. The scale information of the feature points is obtained using the binocular parallax method. The pyramid LK optical flow algorithm is used to track feature points, and the RANSAC (random sampling consistency) algorithm is used to remove mismatched feature points; at the same time, the inertial measurement data is pre-integrated to keep the data synchronization between sensors; a sliding window is constructed to select and marginalize key frames, and the position, attitude, speed, gravity direction and drift of the inertial device of the drone are estimated by graph optimization. Finally, the indoor positioning of the drone in a dynamic environment and the closed-loop detection and re-indoor positioning of the previously reached positions are designed, which effectively improves the high-precision indoor positioning of the drone and achieves the effect required by the vertical parallel omnidirectional mobile shooting technology;

[0060] The GNSS+UWB positioning method:

[0061] (1) Pre-set up UAV fake base equipment points and UAV mobile stations;

[0062] (2) The UAV pseudo-base equipment points surround the UAV mobile station according to a preset spatial distribution;

[0063] (3) The UAV mobile station detects the number of GNSS satellite signals currently received in real time, and compares the number of GNSS satellite signals detected with a preset threshold;

[0064] (4) When the number of GNSS satellite signals detected is greater than a threshold, the UAV mobile station uses a preset GNSS positioning module to receive positioning information sent by the base station and performs real-time coordinate differential analysis;

[0065] (5) When the number of GNSS satellite signals detected is less than a threshold, the UAV mobile station uses a preset UWB positioning module to measure distance via UWB and receive positioning information sent by a preset UAV pseudo-base equipment point to calculate the real-time coordinates of the UAV mobile station;

[0066] (6) The main construction principles of the drone fake base equipment include deception equipment and signal delay technology;

[0067] The spoofing device can generate a fake Global Positioning System (GPS) signal, adjust the satellite signal delay, forge the GPS signal, and cause the drone to misjudge its own location;

[0068] The signal delay technology described above: The deception device adjusts the delay of each branch satellite signal to forge the GPS signal, causing the drone to receive incorrect location information, thereby changing its flight path;

[0069] The binocular vision-inertial fusion positioning method:

[0070] (1) Using binocular vision and IMU as sensors, we can obtain data from sensors with finer precision and inertial sensors, extract ORB features from the binocular images, and determine the matching relationship between the left and right images.

[0071] (2) As shown in Figure 5, the imaging model of the binocular camera, point P in the figure is the position of the feature point of the object, OL and OR are the left camera and the right camera respectively. The coordinates of the feature point P in the camera coordinate system obtained by the binocular parallax method are:

[0072] Xc=b(uL-uO) / d

[0073] Yc=b(vL-vO) / d

[0074] Zc=bf / d

[0075] (Where [Xc, Yc, Zc]T is the scaled camera coordinate of point P; f is the focal length of the camera, uL and vL are the horizontal and vertical coordinates of point PL in the camera coordinate system, b is the baseline, i.e. the actual distance between the binocular centers; d is the parallax, d = uL -uR; (uO, vO) are the coordinates of the optical center of the left camera);

[0076] (3) Use the pyramid LK optical flow method to track features and determine the matching relationship between the left and right images;

[0077] The process of the pyramid LK optical flow algorithm is as follows: (1) establish a pyramid - (2) iterate and calculate the optical flow - (3) convergence condition

[0078] (1) Building a pyramid: First, perform multi-scale pyramid decomposition on the input image through Gaussian blur and downsampling operations to form a pyramid hierarchy. The resolution of each layer of the image is lower than that of the previous layer, but it covers a larger field of view. For example, a 5-layer image pyramid is constructed for the first and second frames of the image. The horizontal and vertical coordinates of the feature points detected in the first frame of the original image are reduced to 1 / 16 of the original coordinates, that is, the coordinates of the corner points at the top layer of the pyramid of the first frame of the original image are obtained. The coordinates of the corner points at the top layer of the pyramid of the first frame of the original image are used as the predicted coordinates of the corner points at the top layer of the pyramid of the second frame of the image.

[0079] (2) Iteratively calculate the optical flow using the Lucas-Kanade optical flow algorithm. Since the image resolution of each layer is low, the motion displacement is also reduced accordingly, which makes it possible to accurately estimate the optical flow at each layer. Then, the optical flow calculated at each layer is passed to the next layer as the initial value for the calculation of the next layer, and then the predicted values ​​of the corner point coordinates of the second frame are matched and corrected.

[0080] (3) Convergence condition. The iteration process will continue until the convergence condition is reached. The convergence condition can be that the change in optical flow is less than a certain threshold, or the change in the cost function is less than a certain threshold. This ensures the stability and accuracy of the optical flow calculation. If the bottom layer of the image pyramid is not reached, the horizontal and vertical coordinates of the feature points in the current pyramid layer of the first frame image and the horizontal and vertical coordinates of the corner points of the current pyramid layer of the second frame image corrected by the LK optical flow method are multiplied by 2 to obtain the horizontal and vertical coordinates of the feature points of the next layer of the pyramid layer of the first frame image and the horizontal and vertical coordinates of the predicted feature points of the next layer of the pyramid of the second frame image. Otherwise, return (2);

[0081] (4) In applications, the pyramid LK optical flow algorithm is often used in video processing, motion detection and tracking, and other fields. In order to optimize the performance of the algorithm, select an appropriate integral window size. A smaller window is suitable for detail preservation and occlusion, and a larger window is suitable for fast motion. Adjust the number of pyramid layers. Increasing the number of pyramid layers can improve the robustness of the algorithm.

[0082] The present invention proposes an indoor omnidirectional positioning method for an aircraft aerial photography device. The method adopts a wireless self-charging parking platform for an unmanned aerial vehicle, is connected to a rotor vertical take-off and landing aircraft, is connected to a three-axis gimbal, is connected to a binocular camera module group, and is equipped with an omnidirectional intelligent positioning method to realize the positioning of the unmanned aerial vehicle in a complex environment. When the satellite signal is affected, the positioning method based on the global positioning system (GNSS) fusion UWB (Ultra Wideband) + binocular vision can overcome scenes with many obstacles in complex environments, such as basements or high-rise buildings in shopping malls, performance venues, offices, and teaching, and other indoor scenes, and achieve the intelligent positioning and shooting technical effect of vertical parallel omnidirectional movement. The method is a low-cost indoor real-time positioning method for unmanned aerial vehicles that can improve production efficiency and is easy to implement. The method can be promoted and applied to other robots that require precise indoor positioning, and effectively solves the high-precision defect of indoor positioning of unmanned aerial vehicles.

Claims

1. An indoor omnidirectional positioning method for an aircraft aerial photography device, comprising at least one unmanned aerial vehicle wireless self-charging parking platform (1), connected to at least one rotary wing vertical take-off and landing aircraft (2), connected to at least one three-axis gimbal (3), mounted on the vertical take-off and landing aircraft (2), connected to at least one binocular camera module group (4), mounted on the three-axis gimbal (3), and assembled using an omnidirectional intelligent positioning method (5), characterized in that: The unmanned aerial vehicle wireless self-charging parking platform (1) guides the aircraft to autonomously land and wirelessly charge; the vertical take-off and landing aircraft (2) has vertical take-off and landing, hovering, and stable flight capabilities; the three-axis gimbal (3) achieves a vertical, parallel, and omnidirectional moving shooting angle; and the omnidirectional intelligent positioning method (5) is based on a global positioning system (GNSS) + wireless pulse communication technology UWB (Ultra Wideband) fused binocular vision positioning method to achieve a vertical, parallel, and omnidirectional moving intelligent positioning shooting technical effect.

2. According to claim 1, a method for indoor omnidirectional positioning of an aircraft aerial photography device, characterized in that: The unmanned aerial vehicle wireless self-charging parking platform (1) comprises a unmanned aerial vehicle lifting platform box body (10) and a wireless charging platform (11), a power charger connected to the unmanned aerial vehicle wireless self-charging parking platform (1), and a wireless charger metal coil transmitting end (110) is installed on a spring pressure sensor (111), so as to wirelessly charge the battery of the aircraft and maintain the power supply for long-term use.

3. The indoor omnidirectional positioning method of an aircraft aerial photography device according to claim 1, characterized in that: The rotor vertical take-off and landing aircraft (2) comprises an aircraft body (20), four sets of arm spiral blades (21) are connected to the left and right sides of the aircraft body (20), two sets of landing gears (22) are connected to the bottom sides of the aircraft body (20), and a wireless charger metal coil receiving end (23) is connected to the landing gear (22). When the engine is started, the rotor starts to rotate to generate upward and downward lift. The rotation speed of the rotor and the blade angle are adjusted to control the ascent and descent of the aircraft.

4. A method for indoor omnidirectional positioning of an aircraft aerial photography device according to claim 1 or 3, characterized in that: The aircraft body (20) is connected to a three-axis gimbal (3) at the front. The three-axis gimbal (3) also includes a gimbal fixing bracket (30) connected to a heading axis (301), a pitch axis (302), and a roll axis (303). Accelerometers MEMS gyroscopes (31) are installed on the heading axis (301), the roll axis (302), and the pitch axis (303) as IMU sensors. Each axis is equipped with a corresponding servo motor and a drive circuit (32). Combined with an IMU feedback system (33), the device collects the attitude information of the object, including data in the three directions of roll, pitch, and heading, monitors and adjusts the attitude of the gimbal in real time, and realizes the attitude stabilization control of the target object. The device ensures the clarity and stability of the captured image. In a dynamic environment, the device realizes the precise control and stabilization of the attitude of the object, thereby providing high-quality shooting effects in various application scenarios.

5. The indoor omnidirectional positioning method of an aircraft aerial photography device according to claim 1 or 4, characterized in that: The three-axis gimbal (3) further comprises a binocular camera module group (4), which is installed on the three-axis gimbal (3) and comprises a binocular camera (40), an image sensor (41), an image processor (42), and other auxiliary components, a flexible printed circuit board (FPC), an autofocus motor, and an infrared filter (IR Cut Filter). Light enters the camera module through the lens and is focused by the lens system to form an optical image of the actual scene; photoelectric conversion: the optical image is projected onto a photosensitive element (such as a CMOS or CCD sensor) of the camera module, and the photosensitive element converts the optical signal into an electrical signal; signal processing: the camera module amplifies and filters the electrical signal to remove noise, adjust color balance, enhance details, etc., so as to obtain a clear and accurate image.

6. The indoor omnidirectional positioning method of an aircraft aerial photography device according to claim 1, characterized in that: The omnidirectional intelligent positioning method (5) includes a binocular vision positioning method based on the global positioning system (GNSS) + UWB (Ultra Wideband). When the global positioning system (GNSS) signal is blocked by buildings, resulting in an insufficient number of observation satellites received, wireless transmission is achieved by intelligently selecting an ultra-narrow pulse communication technology (UWB) below the microsecond level for sending and receiving. The accelerometer MEMS gyroscope (31) installed on the three-axis gimbal (3) is used as an IMU sensor to feed back to the UWB (Ultra Wideband). The binocular vision positioning method is performed in a tightly coupled manner, and the scale information of the feature points is obtained by using the binocular parallax method. The pyramid LK optical flow algorithm is used to obtain the scale information of the feature points. Track feature points and use the RANSAC (random sampling consistency) algorithm to remove mismatched feature points; pre-integrate inertial measurement data to keep data synchronization between sensors; build a sliding window, select and marginalize key frames, and use graph optimization to estimate the position, attitude, speed, gravity direction and drift of inertial devices of the drone. Finally, design the indoor positioning of the drone in a dynamic environment and the functions of closed-loop detection and re-indoor positioning of the locations that have been reached, which effectively improves the high-precision indoor positioning of the drone and achieves the effect required by the vertical, parallel and omnidirectional mobile shooting technology.

7. The indoor omnidirectional positioning method of an aircraft aerial photography device according to claim 1 or 6, characterized in that: The GNSS+UWB positioning method: (1) pre-setting the UAV pseudo-base equipment point and the UAV mobile station; (2) The UAV pseudo-base equipment points surround the UAV mobile station according to a preset spatial distribution; (3) The UAV mobile station detects the number of GNSS satellite signals currently received in real time, and compares the number of GNSS satellite signals detected with a preset threshold; (4) When the number of GNSS satellite signals detected is greater than the threshold, the UAV mobile station uses a preset GNSS positioning module to receive positioning information sent by the base station and performs real-time coordinate differential analysis; (5) When the number of GNSS satellite signals detected is less than the threshold, the UAV mobile station uses the preset UWB positioning module to measure distance via UWB and receive the positioning information sent by the preset UAV pseudo-base equipment point to calculate the real-time coordinates of the UAV mobile station.

8. The indoor omnidirectional positioning method of an aircraft aerial photography device according to claim 1 or 7, characterized in that: The drone fake base equipment mainly includes deception equipment and signal delay technology; the deception equipment can generate a false global positioning system (GPS) signal, and forge the GPS signal by adjusting the delay of the satellite signal, causing the drone to misjudge its own position; the signal delay technology can be used to forge the GPS signal by adjusting the delay of each branch satellite signal, causing the drone to receive incorrect location information, thereby changing its flight path.

9. The indoor omnidirectional positioning method of an aircraft aerial photography device according to claim 1 or 6, characterized in that: The binocular vision-inertial fusion positioning method: (1) uses binocular vision and IMU as sensors to obtain data from sensors and inertial sensors with finer precision, performs ORB feature extraction on the binocular images, and determines the matching relationship between the left and right images; (2) The imaging model of the binocular camera shown in the figure, point P is the position of the object feature point, OL and OR are the left camera and the right camera respectively, and the coordinates of the feature point P in the camera coordinate system obtained by the binocular parallax method are: Xc=b(uL-uO) / d Yc=b(vL-vO) / d Zc=bf / d (wherein, [Xc, Yc, Zc]T is the camera coordinate with scale of point P; f is the focal length of the camera, uL and vL are the horizontal and vertical coordinates of point PL in the camera coordinate system respectively, b is the baseline, that is, the actual distance between the binocular centers; d is the parallax, d=uL-uR; (uO, vO) are the coordinates of the optical center of the left camera).

10. The indoor omnidirectional positioning method of an aircraft aerial photography device according to claim 1 or 6, characterized in that: The pyramid LK optical flow method is used to track features and determine the matching relationship between the left and right images. The process of the pyramid LK optical flow algorithm is as follows: (1) establish a pyramid - (2) iteratively calculate the optical flow - (3) convergence condition; (1) Establish a pyramid: First, perform multi-scale pyramid decomposition on the input image through Gaussian blur and downsampling operations to form a pyramid hierarchy. The resolution of each layer of the image is lower than that of the previous layer, but it covers a larger field of view. For example, a 5-layer image pyramid is constructed for the first frame and the second frame of the image; the horizontal and vertical coordinates of the feature points detected in the first frame of the original image are reduced to 1 / 16 of the original, that is, the coordinates of the corner points at the top layer of the pyramid of the first frame of the original image are obtained; the coordinates of the corner points at the top layer of the pyramid of the first frame of the original image are used as the predicted values ​​of the coordinates of the corner points at the top layer of the pyramid of the second frame of the image; (2) Iteratively calculate the optical flow, use the Lucas-Kanade optical flow algorithm to calculate the optical flow. Since the image resolution of each layer is low, the motion displacement is also reduced accordingly, which makes it possible to estimate the optical flow more accurately at each layer, and then calculate each layer. The optical flow is passed to the next layer as the initial value for the calculation of the next layer, and then the predicted value of the corner point coordinates of the second frame is matched and corrected; (3) Convergence condition. The iterative process will continue until the convergence condition is reached. The convergence condition can be that the change in optical flow is less than a certain threshold, or the change in the cost function is less than a certain threshold. This ensures the stability and accuracy of the optical flow calculation. If the bottom layer of the image pyramid is not reached, the horizontal and vertical coordinates of the feature points in the current pyramid layer of the first frame image and the horizontal and vertical coordinates of the corner points in the current pyramid layer of the second frame image corrected by the LK optical flow method are multiplied by 2 to obtain the horizontal and vertical coordinates of the feature points in the next layer of the pyramid layer of the first frame image and the horizontal and vertical coordinates of the predicted feature points in the next layer of the pyramid layer of the second frame image. Otherwise, return to (2); (4) In applications, the pyramid LK optical flow algorithm is often used in video processing, motion detection and tracking and other fields. In order to optimize the performance of the algorithm, select a suitable integral window size. A smaller window is suitable for detail retention and occlusion, and a larger window is suitable for fast motion. Adjust the number of pyramid layers. Increasing the number of pyramid layers can improve the robustness of the algorithm.

Images