Visible light combined binocular measurement and multi-rotor unmanned aerial vehicle landing guiding method and system

Abstract

The invention discloses a visible light combined binocular measurement and multi-rotor unmanned aerial vehicle landing guiding method and system. The method comprises the steps that four cameras are arranged in a measurement field in a rectangular mode; the four cameras are grouped in pairs to perform binocular system parameter calibration respectively; acquiring multi-view target unmanned aerial vehicle image data by using four cameras; splicing the image data of the four cameras to obtain a full-view target image based on the matching relationship between the feature point in the image data of one camera and the matching point of the feature point in the image data of the other three cameras; resolving state parameters of the target unmanned aerial vehicle relative to the landing center; the target unmanned aerial vehicle platform adjusts flight parameters according to the state parameters fed back in real time, and the landing process is completed. According to the method, any sensor or auxiliary equipment does not need to be installed on an aircraft fuselage, the high-precision full-view target image is obtained only through an optical measurement mode based on the image processing technology, and all-directional landing guidance can be conducted on the target unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle (UAV) landing guidance, and in particular to a method and system for measuring and guiding the landing of a multi-rotor UAV by combining visible light with binocular vision. Background Art

[0002] With the rapid development of autonomous control technology of drones and the rapid increase in the demand for carrying out diversified tasks, the demand for drones to have the ability to perform tasks autonomously is becoming more and more urgent. As we all know, during the mission of an aircraft, the take-off and landing stages have the highest risk factors. Among them, the navigation positioning and flight status measurement values ​​of drones that exceed the limit will cause errors in judgment during the autonomous decision-making process and cause flight accidents.

[0003] In terms of the measurement and guidance of the UAV landing process, the traditional method is mainly to use GPS with multi-sensor integrated measurement systems such as height-measuring radar or altimeter. However, this method has problems such as insufficient positioning accuracy of the UAV, large deviation in flight parameter measurement, and inability to measure attitude angles. In addition, its system architecture is complex, electromagnetic interference is serious, and the guidance process is highly dependent on the detection data of all sensors, and the functions of each module are not redundantly designed. In recent years, laser guidance technology and aircraft landing optical auxiliary technology based on identification light sources also require the installation of measurement sensors or other auxiliary equipment on the aircraft fuselage, which will not only occupy the precious payload of the aircraft, squeeze the installation space of other functional payloads, cause the aircraft fuselage to be too heavy and the range to be reduced, but also make the aerodynamic layout design, electrical design, signal and control and other data flow designs more complicated.

[0004] Affected by these unfavorable factors, there is an urgent need for a reliable, simple, non-contact, high-precision measurement and guidance system that has no electromagnetic interference to drones and other systems and does not rely on navigation systems to make up for the shortcomings of traditional technology. Summary of the invention

[0005] In view of the problems existing in the above-mentioned prior art, the present invention provides a visible light combined binocular measurement and guidance method and system for landing of a multi-rotor UAV, the purpose of which is to simplify the landing guidance process of the UAV through a single visual measurement, and utilize the position design of four cameras and the image processing technology to optimize the matching method between the four images, thereby expanding the observation angle, and measuring and solving the target UAV in all directions at 360°, thereby ensuring that simple optical measurement can also achieve the high-precision effect of multi-sensor joint measurement.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a method for measuring and guiding a multi-rotor UAV landing by using visible light combined with binocular, comprising:

[0008] The system site location was evaluated based on the size of the target UAV, the landing speed, and the camera measurement field of view, and four cameras were placed in a rectangular form in the measurement field;

[0009] The four cameras are divided into two groups to calibrate the binocular system parameters;

[0010] Use four cameras to collect multi-view target drone image data;

[0011] Based on the matching relationship between the feature points in the image data of one of the cameras and the matching points of the feature points in the image data of the other three cameras, the image data of the four cameras are spliced ​​to obtain a full-view target image;

[0012] Calculating the state parameters of the target UAV relative to the landing center based on the full-view target image;

[0013] The target UAV platform adjusts the flight parameters according to the real-time feedback status parameters and completes the landing process.

[0014] As a preferred implementation, the four cameras are arranged in a rectangular form in the measurement field as follows:

[0015] Four cameras are arranged in a rectangular manner at the four corners of the measurement field. The center of the landing area is located at the intersection of the diagonals of the rectangle. The optical axes of the four cameras point to the landing center, and the angle between the optical axis and the line connecting the cameras is 45°.

[0016] As a preferred implementation, the step of acquiring a full-view target image specifically includes:

[0017] For the image data collected at the same time, the image data of one camera is selected as the feature image, and the image data of the other three cameras are recorded as the images to be matched;

[0018] Calculate the grayscale change difference of each pixel in the feature image in the horizontal and vertical directions to extract the feature points;

[0019] Determine the matching points of the feature points in the images to be matched by epipolar positioning in the three images to be matched in turn according to the neighboring relationship of the cameras;

[0020] According to the matching relationship between multiple feature points and matching points, the image data of the four cameras are spliced ​​to obtain a full-view target image.

[0021] As a preferred implementation, the step of calculating the grayscale variation difference of each pixel in the feature image in the horizontal and vertical directions to extract the feature points comprises:

[0022] The horizontal / vertical gradient value of the pixel is obtained based on the median difference of the grayscale values ​​of the adjacent pixels before and after the pixel in the horizontal / vertical direction;

[0023] Calculate the ratio of the smaller value to the larger value in the horizontal / vertical gradient value to obtain the grayscale change difference of the pixel point;

[0024] When the grayscale change difference of a pixel point is less than a preset difference threshold, the pixel point is used as a feature point of the feature image.

[0025] As a preferred implementation, the step of determining the matching point of the feature point in the corresponding image data by epipolar positioning includes:

[0026] (1) Select any feature point in the feature image and record it as feature point A;

[0027] (2) Based on the binocular system calibration parameters between the two cameras that obtain the feature image and the first image to be matched, epipolar line correction is performed to make the optical axes of the two cameras parallel, thereby obtaining the first epipolar line of the feature point A in the first image to be matched;

[0028] (3) determining a window to be matched of feature point A in the first image to be matched according to a gradient change of feature point A on the first epipolar line;

[0029] (4) In the window to be matched, the similarities between the pixels in the eight neighborhoods of each pixel to be matched and the feature point A are calculated in sequence and weighted summed to obtain the total similarity of each pixel to be matched, wherein the pixels in the eight neighborhoods of each pixel to be matched are weighted by a Gaussian function;

[0030] (5) If the total similarity of all the pixels to be matched in the window to be matched is less than the similarity threshold, return to step (2) to determine the matching point of feature point A in the next image to be matched;

[0031] Otherwise, the pixel point to be matched in the to-be-matched window with the greatest similarity to the feature point A is selected as the matching point, and the process returns to step (2) to determine the matching point of the feature point A in the next to-be-matched image;

[0032] (6) When the matching points of feature point A in the three images to be matched are determined, return to step (1) and reselect a feature point for matching until the matching points of all feature points in the feature image in the three images to be matched are determined.

[0033] As a preferred implementation, determining the window to be matched of the feature point in the first image to be matched according to the gradient change of the feature point on the first epipolar line includes:

[0034] Projecting the gradient value of the feature point onto the first epipolar line to obtain a projected gradient value;

[0035] Calculate the average gradient value of each pixel point on the first epipolar line;

[0036] A window to be matched of the feature point in the first image to be matched is determined based on the projection gradient value and the average gradient value.

[0037] As a preferred embodiment, before the target UAV platform adjusts the flight parameters according to the real-time feedback status parameters, it also includes converting the camera coordinate system of the measurement system composed of four cameras to the platform coordinate system of the target UAV to adjust the flight parameters, and converting the body coordinate system to the platform coordinate system of the target UAV to obtain the flight attitude angle of the target UAV.

[0038] In a second aspect, the present invention provides a visible light combined binocular measurement and guidance multi-rotor UAV landing system, comprising:

[0039] The system deployment module is used to evaluate the system deployment position according to the size of the target UAV, the landing speed and the camera measurement field of view, and to deploy four cameras in a rectangular form in the measurement field;

[0040] The parameter calibration module is used to group the four cameras into two groups and perform binocular system parameter calibration respectively;

[0041] An image acquisition module is used to collect target drone image data from multiple perspectives using four cameras;

[0042] A view stitching module, used for stitching the image data of four cameras to obtain a full-view target image based on the matching relationship between the feature points in the image data of one camera and the matching points of the feature points in the image data of the other three cameras;

[0043] A state calculation module, used for calculating the state parameters of the target UAV relative to the landing center based on the full-view target image;

[0044] The flight adjustment module is used to adjust the flight parameters through the target UAV platform according to the real-time feedback status parameters to complete the landing process.

[0045] In a third aspect, the present invention provides an electronic device, comprising:

[0046] A memory for storing executable instructions;

[0047] The processor is used to implement a visible light combined with binocular measurement and guidance method for landing a multi-rotor UAV as described above when running the executable instructions stored in the memory.

[0048] In a fourth aspect, the present invention provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, implement a visible light combined binocular measurement and guidance method for landing a multi-rotor UAV as described above.

[0049] The present invention provides a method and system for measuring and guiding the landing of a multi-rotor UAV by combining visible light with binocular vision, which has the following beneficial effects:

[0050] 1. The present invention simplifies the overall architecture of the UAV landing guidance system, and can complete the measurement and guidance tasks only by using the optical system set up on the ground without increasing the load burden of the UAV.

[0051] 2. The present invention is based on the technology in the field of image processing, matches the four images collected by the camera, and avoids the problem of perspective occlusion existing in the traditional binocular camera in a large field of view through multiple complementary perspectives, and can obtain more comprehensive and accurate information on the position, speed, posture and other status of the target in space, thereby improving the accuracy and reliability of landing guidance. In addition, more edge feature points are extracted during the matching process, rather than being limited to corner points, thereby increasing the number of matching point pairs; the appropriate window to be matched is optimized according to the difference of feature points in different images to be matched, and the neighborhood information of multiple pixel points to be matched in the window to be matched is aggregated, so as to more accurately find the matching points corresponding to the feature points.

[0052] 3. The system of the present invention is portable and low-cost in design, and is easy to disassemble, transport and deploy, with low procurement and operation costs and high cost performance. The system deployment is simple and quick, and only four high-speed cameras need to be deployed at the four corners of the landing area, and other modules and equipment can be placed in a corner of the landing area, without occupying a large amount of space in the landing area and other auxiliary systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a schematic diagram of the composition of a visible light combined binocular measurement and guidance multi-rotor UAV landing device provided by an embodiment of the present invention;

[0054] Figure 2 It is a flow chart of a method for measuring and guiding a multi-rotor UAV landing by combining visible light with binocular vision according to the present invention;

[0055] Figure 3 is a top view of a system measurement perspective in one embodiment of the present invention;

[0056] Figure 4 It is an embodiment of the present invention to establish a NEU coordinate system with the center of the landing area as the origin;

[0057] Figure 5 Schematic diagram of the flight attitude angle of a multi-rotor UAV in one embodiment of the present invention;

[0058] Figure 6 is a schematic diagram of the conversion relationship between the NEU coordinate system and the NED coordinate system in one embodiment of the present invention;

[0059] Figure 7The present invention is a structural block diagram of a visible light combined binocular measurement and guidance multi-rotor UAV landing system. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] First, if Figure 1 As shown, the present invention first provides an optical multi-sensor fixed-wing UAV detection and tracking guidance device, comprising:

[0062] The visible light measurement camera group consists of four high-speed cameras, which are system optical measurement sensors used to collect image data of multi-rotor drones during vertical landing. The high-speed camera is equipped with a variable-focus optical lens that can meet the requirements of clear imaging in different fields of view. It can be remotely controlled, automatically triggered or manually triggered, and has image key frame downloading, slow playback, fast detection, and remote focusing functions.

[0063] The calibration device is used to calibrate the four high-speed cameras, and during the calibration process, the control instructions and calibration data are transmitted through the network and wireless communication. In a specific embodiment, the calibration device is composed of a handheld locator base station and a handheld locator mobile station; the handheld locator base station is used to receive satellite signals; the handheld locator mobile station includes a receiver, and the receiver is used to receive the carrier information sent by the handheld locator base station in real time; a target is hung on the receiver, and the target is used to be a label photographed by the high-speed camera.

[0064] Time system equipment, which is used to receive the time data of Beidou or GPS satellite navigation system and convert it into the network format of year, month, day, hour, minute and second, and provide time information accurate to the whole second for each measurement module of the system through network transmission;

[0065] Wireless communication equipment, which is used for communication and data transmission between the system and the drone platform; the wireless communication module receives the drone intelligence measured by the system through network cable transmission, and the wireless communication module uploads the drone intelligence to the drone platform in the form of radio waves;

[0066] An industrial computer, which is used for controlling each measurement module of the system, displaying the equipment status, processing the collected data and transmitting the information; the industrial computer controls the triggering, synchronization, preview and high-speed collection working mode of the high-speed camera through a network cable or a serial cable, and completes the storage and transmission of the collected data;

[0067] A 10 Gigabit Ethernet switch is used for data exchange between various measurement modules of the system; the high-speed camera, calibration module, time system module, wireless communication module, and industrial computer are respectively connected to the 10 Gigabit Ethernet switch via network cables;

[0068] A portable power supply is used to provide 220VAC AC and 12V, 24V DC power to the system.

[0069] In actual application, after the system deployment is completed, the system is powered on, and communication links between four high-speed cameras and other equipment are established. The system enters standby mode and is ready to guide the multi-rotor drone to land at any time.

[0070] To facilitate understanding of this embodiment, a method for visible light combined with binocular measurement and guidance of multi-rotor UAV landing disclosed in an embodiment of the present invention is described in detail below.

[0071] like Figure 2 As shown, the method includes:

[0072] The system site location was evaluated based on the size of the target UAV, the landing speed, and the camera measurement field of view, and four cameras were placed in a rectangular form in the measurement field;

[0073] The four cameras are divided into two groups to calibrate the binocular system parameters;

[0074] Use four cameras to collect multi-view target drone image data;

[0075] Based on the matching relationship between the feature points in the image data of one of the cameras and the matching points of the feature points in the image data of the other three cameras, the image data of the four cameras are spliced ​​to obtain a full-view target image;

[0076] Calculating the state parameters of the target UAV relative to the landing center based on the full-view target image;

[0077] The target UAV platform adjusts the flight parameters according to the real-time feedback status parameters and completes the landing process.

[0078] The present invention collects multi-view target drone images by designing the positions of four high-speed cameras, and increases the number of matching point pairs by extracting more edge feature points rather than limiting to corner points during the matching process, so that the subsequent four images can have more corresponding points to splice to obtain a full-view image; according to the difference of feature points in different images to be matched, the appropriate window to be matched is optimized, and the neighborhood information of multiple pixels to be matched in the window to be matched is aggregated to further increase the accuracy of the match. Through multiple complementary perspectives, the perspective occlusion problem existing in traditional binocular cameras in a large field of view is avoided, and more comprehensive and accurate information on the position, speed, attitude and other status of the target in space can be obtained, thereby improving the accuracy of landing guidance.

[0079] As a preferred embodiment, Figure 3 As shown, four cameras are arranged in a rectangular form in the measurement field as follows:

[0080] Four cameras are arranged in a rectangular manner at the four corners of the measurement field. The center of the landing area is located at the intersection of the diagonals of the rectangle. The optical axes of the four cameras point to the landing center, and the angle between the optical axis and the line connecting the cameras is 45°.

[0081] Exemplarily, four high-speed cameras are numbered as high-speed camera 1, high-speed camera 2, high-speed camera 3, and high-speed camera 4 according to their IP addresses. The high-speed cameras are sequentially grouped in pairs to form visible light binocular measurement modules, where high-speed cameras 1-2 are the first group, high-speed cameras 2-3 are the second group, high-speed cameras 3-4 are the third group, and high-speed camera 4-1 is the fourth group. The four groups are numbered and are responsible for cutting the landing area in different areas of the field of view. The four groups of visible light binocular measurement modules jointly perform binocular vision measurement on the vertically landing multi-rotor drone simultaneously and synchronously.

[0082] The four cameras are combined in pairs to form four binocular systems. The three-dimensional coordinates of each point in the space are determined by the four binocular systems. The spatial information, flight parameters and landing attitude information of the four degrees of freedom of the target multi-rotor UAV fuselage relative to the landing center can be obtained. Compared with a single binocular system, the measurement accuracy has been steadily improved, and the problem of field of view obstruction can be effectively solved by complementing multiple perspectives.

[0083] In one embodiment, the binocular system parameter calibration process is that the receiver and the target are manually moved within the field of view. The receiver receives the carrier information sent by the handheld locator base station in real time, and sends the timestamp and positioning information to the industrial computer in real time. The industrial computer calculates the dynamic position coordinates of the handheld locator mobile station in real time. At the same time, four high-speed cameras aim at the target in real time to collect target image data, and synchronously send the image data with timestamps to the industrial computer. The industrial computer completes the calibration data processing in combination with the target positioning information to form a system calibration file.

[0084] When the system starts measuring, the B-code timing module of the time system equipment obtains the time information of the Beidou or GPS satellite navigation system, converts its data format into a network format and transmits it to four high-speed cameras; the four high-speed cameras are triggered by the B-code signal to synchronously shoot the multi-rotor UAV target in four degrees of freedom, and upload the target image data with timestamps to the industrial computer through the network cable. The industrial computer matches the image data taken by each high-speed camera through the timestamp information.

[0085] As a preferred implementation, the step of acquiring a full-view target image specifically includes:

[0086] For the image data collected at the same time, the image data of one camera is selected as the feature image, and the image data of the other three cameras are recorded as the images to be matched;

[0087] In this embodiment, a time synchronization device is provided to control four high-speed cameras to simultaneously collect image data of the target drone, and obtain four image data respectively, ensuring accurate performance and real-time synchronization performance during shooting. The calibration device integrates the data information of the four images according to the timestamp. One of them is used as a feature image, and the matching points corresponding to the feature image are found in the remaining three images.

[0088] Calculate the grayscale change difference of each pixel in the feature image in the horizontal and vertical directions to extract the feature points;

[0089] Determine the matching points of the feature points in the images to be matched by epipolar positioning in the three images to be matched in turn according to the neighboring relationship of the cameras;

[0090] According to the matching relationship between multiple feature points and matching points, the image data of the four cameras are spliced ​​to obtain a full-view target image.

[0091] From the above-mentioned four cameras, it can be seen that there must be common image parts in the adjacent image data. In this embodiment, matching is performed in the order of cameras 1, 2, 3, and 4, and the image data of camera 1 is used as the feature image, that is, for feature point A in camera 1, matching point A2 is found in camera 2, and then matching point A3 is found in camera 3 based on matching point A2, and then matching point A4 is found in camera 4 based on matching point A3.

[0092] As a preferred implementation, the step of calculating the grayscale variation difference of each pixel in the feature image in the horizontal and vertical directions to extract the feature points comprises:

[0093] The horizontal / vertical gradient value of the pixel is obtained based on the median difference of the grayscale values ​​of the adjacent pixels before and after the pixel in the horizontal / vertical direction;

[0094] Exemplarily, the horizontal / vertical gradient value is calculated using the following formula:

[0095]

[0096]

[0097] Among them, dx(i,j) represents the horizontal gradient value, dy(i,j) represents the vertical gradient value, and I(i,j) represents the grayscale value of the pixel point (i,j).

[0098] Calculate the ratio of the smaller value to the larger value in the horizontal / vertical gradient value to obtain the grayscale change difference of the pixel point;

[0099] When the grayscale change difference of a pixel point is less than a preset difference threshold, the pixel point is used as a feature point of the feature image.

[0100] In the prior art, corner points in an image are usually extracted as feature points, but in actual applications, there is less corner point data, which is not suitable for high-precision image matching. Therefore, the present invention screens edge points in an image by calculating the grayscale change differences of pixel points in the horizontal and vertical directions. Specifically, the horizontal / vertical gradient value represents the change difference between adjacent pixel points in the horizontal / vertical direction of the pixel point. At the edge position, due to the transition of pixels, there will be a large difference in the change difference between adjacent pixel points in the horizontal / vertical direction. For example, the gradient values ​​on the left and right sides of a vertical edge line are larger, but the upper and lower gradients are not obvious. The present invention can obtain a certain number of obvious edge points by calculating the ratio of the smaller value to the larger value in the horizontal / vertical gradient value, thereby expanding the original data points for image matching and providing a data basis for the accuracy of subsequent full-view image stitching.

[0101] As a preferred implementation, the step of determining the matching point of the feature point in the corresponding image data by epipolar positioning includes:

[0102] (1) Select any feature point in the feature image and record it as feature point A. The three images to be matched are respectively recorded as the first image to be matched, the second image to be matched, and the third image to be matched.

[0103] (2) Based on the binocular system calibration parameters between the two cameras that obtain the feature image and the first image to be matched, epipolar line correction is performed to make the optical axes of the two cameras parallel, thereby obtaining the first epipolar line of the feature point A in the first image to be matched;

[0104] It should be noted that the epipolar line correction may adopt the Fusiello correction method, the Bouguet correction method, etc. in the prior art, and the present invention does not make any requirements for this.

[0105] After epipolar correction, when matching feature points, it is only necessary to search near the same height of the corresponding image, which greatly improves the matching efficiency.

[0106] (3) determining a window to be matched of feature point A in the first image to be matched according to a gradient change of feature point A on the first epipolar line;

[0107] (4) In the window to be matched, the similarities between the pixels in the eight neighborhoods of each pixel to be matched and the feature point A are calculated in sequence and weighted summed to obtain the total similarity of each pixel to be matched, wherein the pixels in the eight neighborhoods of each pixel to be matched are weighted by a Gaussian function;

[0108] The present invention takes into account the correlation between neighborhood points and weights the pixels in the eight neighborhoods of the pixel to be matched, which can avoid interference from similar pixels to be matched in the matching window. The total similarity obtained after aggregating the information of multiple neighborhood points better characterizes the similarity between the point to be matched and the feature point in the surrounding pixel environment.

[0109] (5) If the total similarity of all the pixels to be matched in the window to be matched is less than the similarity threshold, return to step (2) to determine the matching point of feature point A in the next image to be matched;

[0110] When the total similarity of all the pixels to be matched in the matching window is less than the similarity threshold, it means that the feature point is likely to be blocked in the viewing angle of the first image to be matched. The feature point is matched in other images to compensate for the blocked information.

[0111] Otherwise, the pixel point to be matched in the to-be-matched window with the greatest similarity to the feature point A is selected as the matching point, and the process returns to step (2) to determine the matching point of the feature point A in the next to-be-matched image;

[0112] (6) When the matching points of feature point A in the three images to be matched are determined, return to step (1) and reselect a feature point for matching until the matching points of all feature points in the feature image in the three images to be matched are determined.

[0113] As a preferred implementation, determining the window to be matched of the feature point in the first image to be matched according to the gradient change of the feature point on the first epipolar line includes:

[0114] Projecting the gradient value of the feature point onto the first epipolar line to obtain a projected gradient value;

[0115] Calculate the average gradient value of each pixel point on the first epipolar line;

[0116] A window to be matched of the feature point in the first image to be matched is determined based on the projection gradient value and the average gradient value.

[0117] Specifically, when the projected gradient value is larger than the average gradient value, that is, the gradient of the feature point changes rapidly on the first epipolar line, it means that the feature point is more conspicuous in the image to be matched and is more different from the surrounding points. Then, the first window to be matched can be smaller, and the information of the neighborhood points does not contribute much to the similarity.

[0118] Exemplarily, the lateral gradient value of the feature point on the feature image is dx(i, j), the longitudinal gradient value is dy(i, j), and the unit direction vector of the first polar line is a. Then, the feature point is projected onto the first polar line to obtain a projection gradient value Gx(i, j):

[0119] Gx(i,j)=[dx(i,j),dy(i,j)]a

[0120] Since there are certain differences between images from different perspectives, and the difference of each feature point in each image is different, the present invention projects the gradient value of the feature point in the feature image to the corresponding image to be matched, and designs a matching window for each feature point that is adapted to the image to be matched when matching each image to be matched, which can effectively improve the accuracy of feature point matching.

[0121] As a preferred embodiment, before the target UAV platform adjusts the flight parameters according to the real-time feedback status parameters, it also includes converting the camera coordinate system of the measurement system composed of four cameras to the platform coordinate system of the target UAV to adjust the flight parameters, and converting the body coordinate system to the platform coordinate system of the target UAV to obtain the flight attitude angle of the target UAV.

[0122] Specifically, the system of this embodiment uses the North East Sky (NEU) coordinate system as the world coordinate system, takes the landing center O as the coordinate origin, the due north direction as the X axis, the due east direction as the Y axis, and the zenith as the Z axis to establish the North East Sky (NEU) coordinate system of the landing area, such as Figure 4 The system calibration module converts the latitude and longitude values ​​in the WGS-84 coordinate system to the North East coordinate system, and the measured target spatial position and motion parameters are also the values ​​in the North East coordinate system.

[0123] Multi-rotor UAV platforms usually use two coordinate systems: the "body coordinate system" and the "Earth-fixed coordinate system". The body coordinate system uses the center of gravity of the UAV as the coordinate origin, the direction of the nose as the X-axis, the vertical axis of the Z-axis pointing downward, and the direction of the Y-axis is determined by the right-hand rule. In order to keep the UAV platform working in conjunction with other associated systems, the "Earth-fixed coordinate system" must be used as a unified coordinate system. UAV platforms usually use the North-East Earth (NED) coordinate system as the "Earth-fixed coordinate system", with the center of the multi-rotor UAV take-off and landing area as the coordinate origin, the north direction as the X-axis, the east direction as the Y-axis, and the downward direction of the body as the Z-axis to establish the North-East Earth (NED) coordinate system of the landing area. When the UAV takes off and lands, its "body coordinate system" needs to be converted to the "Earth-fixed coordinate system". During the conversion process, the flight attitude angles of the UAV can be obtained: pitch is the pitch angle, roll is the roll angle, and yaw is the yaw angle, such as Figure 5 shown.

[0124] When guiding a multi-rotor drone to land, this system also needs to convert its own North East Universe (NEU) coordinate system to the North East Earth (NED) coordinate system of the drone platform, so as to guide it to land safely with the drone platform as the main body. Keep the X-axis and Y-axis unchanged, and invert the altitude value Z of the drone (Z = -Zˊ). The coordinate conversion process is as follows: Figure 6 shown.

[0125] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0126] Based on the same inventive concept, the embodiment of the present application also provides a system for realizing the above-mentioned visible light combined binocular measurement and guiding the landing of a multi-rotor drone. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiment of the visible light combined binocular measurement and guiding the landing of a multi-rotor drone provided below can be referred to the limitations of the method above, and will not be repeated here.

[0127] like Figure 7 As shown, the present invention also provides a visible light combined binocular measurement and guidance multi-rotor UAV landing system, comprising:

[0128] The system deployment module is used to evaluate the system deployment position according to the size of the target UAV, the landing speed and the camera measurement field of view, and to deploy four cameras in a rectangular form in the measurement field;

[0129] The parameter calibration module is used to group the four cameras into two groups and perform binocular system parameter calibration respectively;

[0130] An image acquisition module is used to collect target drone image data from multiple perspectives using four cameras;

[0131] A view stitching module, used for stitching the image data of four cameras to obtain a full-view target image based on the matching relationship between the feature points in the image data of one camera and the matching points of the feature points in the image data of the other three cameras;

[0132] A state calculation module, used for calculating the state parameters of the target UAV relative to the landing center based on the full-view target image;

[0133] The flight adjustment module is used to adjust the flight parameters through the target UAV platform according to the real-time feedback status parameters to complete the landing process.

[0134] It should be noted that: the visible light combined binocular measurement and guidance multi-rotor UAV landing system provided in this embodiment only uses the division of the above-mentioned functional modules as an example when processing the take-off and landing guidance of the rotor UAV. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules. Each functional module can be composed of a single execution unit, or it can be integrated into a functional module by two or more execution units to realize all the functions of the functional module.

[0135] Those skilled in the art will appreciate that the above modules may be implemented in whole or in part by software, hardware, or a combination thereof. The above modules may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in a computer device in the form of software, so that the processor may call and execute operations corresponding to the above modules.

[0136] An embodiment of the present invention further provides an electronic device, including:

[0137] A memory for storing executable instructions;

[0138] The processor is used to implement a visible light combined binocular measurement and guidance multi-rotor drone landing method as described above when running the executable instructions stored in the memory. The processor can be a central processing unit, other general processors, digital signal processors, application-specific integrated circuits, field programmable gate arrays or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., or a combination of the above chips.

[0139] The specific details of the above-mentioned computer device guiding the landing of the multi-rotor drone can be understood by referring to the relevant descriptions and effects of the previous method, and will not be repeated here.

[0140] Another embodiment of the present invention further provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, implement a visible light combined binocular measurement and guidance method for landing a multi-rotor UAV as described above.

[0141] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disk, or any other computer-readable storage medium that can store program codes.

[0142] The present invention is not limited to the above-mentioned specific implementation modes. Various changes made by ordinary technicians in this field based on the above-mentioned concepts without creative work are all within the protection scope of the present invention.

Claims

1. A visible light combined binocular measurement and guidance method for landing a multi-rotor UAV, characterized in that: The steps include: The system site location was evaluated based on the size of the target UAV, the landing speed, and the camera measurement field of view, and four cameras were placed in a rectangular form in the measurement field; The four cameras are divided into two groups to calibrate the binocular system parameters; Use four cameras to collect multi-view target drone image data; Based on the matching relationship between the feature points in the image data of one of the cameras and the matching points of the feature points in the image data of the other three cameras, the image data of the four cameras are spliced ​​to obtain a full-view target image; Calculating the state parameters of the target UAV relative to the landing center based on the full-view target image; The target UAV platform adjusts the flight parameters according to the real-time feedback status parameters and completes the landing process.

2. The visible light combined binocular measurement and guidance multi-rotor UAV landing method according to claim 1 is characterized in that: The four cameras are arranged in a rectangular form in the measurement field as follows: Four cameras are arranged in a rectangular manner at the four corners of the measurement field. The center of the landing area is located at the intersection of the diagonals of the rectangle. The optical axes of the four cameras point to the landing center, and the angle between the optical axis and the line connecting the cameras is 45°.

3. The visible light combined binocular measurement and guidance method for landing a multi-rotor UAV according to claim 1 is characterized in that: The step of acquiring the full-view target image specifically includes: For the image data collected at the same time, the image data of one camera is selected as the feature image, and the image data of the other three cameras are recorded as the images to be matched; Calculate the grayscale change difference of each pixel in the feature image in the horizontal and vertical directions to extract the feature points; Determine the matching points of the feature points in the images to be matched by epipolar positioning in the three images to be matched in turn according to the neighboring relationship of the cameras; According to the matching relationship between multiple feature points and matching points, the image data of the four cameras are spliced ​​to obtain a full-view target image.

4. The visible light combined binocular measurement and guidance method for landing a multi-rotor UAV according to claim 3 is characterized in that: The step of calculating the grayscale variation difference of each pixel in the feature image in the horizontal and vertical directions to extract the feature point comprises: The horizontal / vertical gradient value of the pixel is obtained based on the median difference of the grayscale values ​​of the adjacent pixels before and after the pixel in the horizontal / vertical direction; Calculate the ratio of the smaller value to the larger value in the horizontal / vertical gradient value to obtain the grayscale change difference of the pixel point; When the grayscale change difference of a pixel point is less than a preset difference threshold, the pixel point is used as a feature point of the feature image.

5. The visible light combined binocular measurement and guidance method for landing a multi-rotor UAV according to claim 4 is characterized in that: The step of determining the matching points of the feature points in the corresponding image data by epipolar positioning comprises: (1) Select any feature point in the feature image and record it as feature point A; (2) Based on the binocular system calibration parameters between the two cameras that obtain the feature image and the first image to be matched, epipolar line correction is performed to make the optical axes of the two cameras parallel, thereby obtaining the first epipolar line of the feature point A in the first image to be matched; (3) determining a window to be matched of feature point A in the first image to be matched according to a gradient change of feature point A on the first epipolar line; (4) In the window to be matched, the similarities between the pixels in the eight neighborhoods of each pixel to be matched and the feature point A are calculated in sequence and weighted summed to obtain the total similarity of each pixel to be matched, wherein the pixels in the eight neighborhoods of each pixel to be matched are weighted by a Gaussian function; (5) If the total similarity of all the pixels to be matched in the window to be matched is less than the similarity threshold, return to step (2) to determine the matching point of feature point A in the next image to be matched; Otherwise, the pixel point to be matched in the to-be-matched window with the greatest similarity to the feature point A is selected as the matching point, and the process returns to step (2) to determine the matching point of the feature point A in the next to-be-matched image; (6) When the matching points of feature point A in the three images to be matched are determined, return to step (1) and reselect a feature point for matching until the matching points of all feature points in the feature image in the three images to be matched are determined.

6. The visible light combined binocular measurement and guidance method for landing a multi-rotor UAV according to claim 5 is characterized in that: Determining the window to be matched of the feature point in the first image to be matched according to the gradient change of the feature point on the first epipolar line comprises: Projecting the gradient value of the feature point onto the first epipolar line to obtain a projected gradient value; Calculate the average gradient value of each pixel point on the first epipolar line; A window to be matched of the feature point in the first image to be matched is determined based on the projection gradient value and the average gradient value.

7. The visible light combined binocular measurement and guidance method for landing a multi-rotor UAV according to claim 1, characterized in that: Before the target UAV platform adjusts the flight parameters according to the real-time feedback state parameters, the camera coordinate system of the measurement system composed of four cameras is converted to the platform coordinate system of the target UAV to adjust the flight parameters, and the body coordinate system is converted to the platform coordinate system of the target UAV to obtain the flight attitude angle of the target UAV.

8. A visible light combined binocular measurement and guidance multi-rotor UAV landing system, characterized in that: include: The system deployment module is used to evaluate the system deployment position according to the size of the target UAV, the landing speed and the camera measurement field of view, and to deploy four cameras in a rectangular form in the measurement field; The parameter calibration module is used to group the four cameras into two groups and perform binocular system parameter calibration respectively; An image acquisition module is used to collect target drone image data from multiple perspectives using four cameras; A view stitching module, used for stitching the image data of four cameras to obtain a full-view target image based on the matching relationship between the feature points in the image data of one camera and the matching points of the feature points in the image data of the other three cameras; A state calculation module, used for calculating the state parameters of the target UAV relative to the landing center based on the full-view target image; The flight adjustment module is used to adjust the flight parameters through the target UAV platform according to the real-time feedback status parameters to complete the landing process.

9. An electronic device, characterized in that: The electronic device comprises: A memory for storing executable instructions; The processor is used to implement the visible light combined binocular measurement and guidance multi-rotor UAV landing method as described in any one of claims 1 to 7 when running the executable instructions stored in the memory.

10. A computer-readable storage medium storing executable instructions, characterized in that: When the executable instructions are executed by the processor, a visible light combined binocular measurement and guidance method for landing a multi-rotor UAV as described in any one of claims 1 to 7 is implemented.

Images