Unmanned aerial vehicle air docking method and device based on multiple two-dimensional codes

Abstract

The invention provides an unmanned aerial vehicle air docking method and device based on multiple two-dimensional codes. The method provided by the invention comprises the steps of determining a blank area of the outer surface of a docking unmanned aerial vehicle; arranging a plurality of two-dimensional codes which have the same size, different sizes and different numbers with the blank area of the target position points in the plurality of target position points of the blank area; generating an angular point coordinate set of each two-dimensional code according to the position of each two-dimensional code; the demand unmanned aerial vehicle obtains a real-time image of the outer surface of the docking unmanned aerial vehicle, and target two-dimensional code information which is completely shot is analyzed from the real-time image; determining relative positions of the demand unmanned aerial vehicle and the docking unmanned aerial vehicle according to the target two-dimensional code information and the angular point coordinate set corresponding to the target two-dimensional code; and controlling the required unmanned aerial vehicle to perform docking flight to the docking unmanned aerial vehicle according to the relative position. According to the unmanned aerial vehicle air docking method and device based on the multiple two-dimensional codes, accurate docking of the unmanned aerial vehicle in the air can be achieved.

Description

Technical Field

[0001] The present application relates to the technical field of drone aerial docking, and in particular to a drone aerial docking method and device based on multiple QR codes. Background Art

[0002] With the rapid development of drone technology, drones are increasingly being used in logistics and distribution, power inspections, emergency rescue, and other fields. As a key technology for enabling collaborative drone swarm operations, automatic charging and replenishment, and rapid payload switching, drone aerial docking technology's reliability and accuracy are core requirements for expanding drone applications. Visual servoing systems, with their advantages such as strong environmental perception, high real-time performance, and high positioning accuracy, have become a key research area for drone aerial docking technology.

[0003] Currently, drone aerial docking technology primarily relies on non-visual sensors such as lidar and ultrasound, or pre-programmed trajectory control. Lidar-based solutions use point cloud data to build an environmental model for docking, but the equipment is expensive and susceptible to interference under complex lighting conditions. Ultrasonic solutions are limited by their range and struggle to meet long-distance docking requirements. Pre-programmed trajectory control lacks adaptability to dynamic environments and cannot cope with attitude deviations or airflow disturbances that occur during docking. Some studies have attempted to introduce visual sensors to assist docking, but these typically only place a single QR code in a single area on the toolbox drone. This results in docking failures when the working drone fails to detect the QR code. The existing technology uses a fixed size for the QR code, which cannot adapt to detection requirements at different distances, resulting in poor detection results at long or close distances.

[0004] Existing drone aerial docking technologies based on non-visual sensors generally have defects such as poor environmental adaptability, high cost or limited range. Traditional visual servoing solutions have a single perception dimension and insufficient algorithm robustness. During the flight docking process, drones need to deal with fluctuations in motion speed and environmental interference at the same time. In complex environments (such as lighting changes and background interference), the false detection rate of existing algorithms is high, which can easily lead to program errors and interruptions. It is difficult to achieve high-precision pose estimation and real-time control in dynamic and complex scenes. In addition, there is a lack of effective false detection identification and processing mechanisms, and the stability of the docking process after QR code false detection cannot be guaranteed. In addition, there is a lack of emergency processing mechanisms when errors occur in QR code recognition. When errors occur during the docking process, manual intervention is required for docking, and the degree of automation and reliability of the entire docking system are low. Summary of the Invention

[0005] In view of this, the present application provides a method and device for drone aerial docking based on multiple QR codes, so as to achieve precise docking of drones in the air.

[0006] Specifically, this application is implemented through the following technical solutions:

[0007] In a first aspect, the present application provides a method for aerial docking of a drone based on multiple QR codes, the method comprising:

[0008] Determine the clear area on the outer surface of the docking drone;

[0009] Arranging a plurality of QR codes in the plurality of target position points in the blank area, which are of the same size as the blank area of the target position points but of different sizes and with different numbers;

[0010] Generate a set of corner point coordinates of each of the two-dimensional codes according to the position of each of the two-dimensional codes;

[0011] The drone is required to obtain a real-time image of the outer surface of the docked drone, and parse the complete target QR code information from the real-time image;

[0012] Determine the relative positions of the desired UAV and the docking UAV based on the target QR code information and the corner point coordinate set corresponding to the target QR code;

[0013] The demand UAV is controlled to fly toward the docking UAV according to the relative position.

[0014] The second aspect of the present application provides a drone aerial docking device based on multiple QR codes, the device comprising a layout module, a calculation module and a control module; wherein,

[0015] The arrangement module is used to determine the blank area on the outer surface of the docking drone;

[0016] The arrangement module is further configured to arrange a plurality of QR codes in the plurality of target locations in the blank area, the QR codes having the same size as the blank area of the target location but having different sizes and numbers;

[0017] The calculation module is used to generate a set of corner point coordinates of each of the two-dimensional codes according to the position of each of the two-dimensional codes;

[0018] The computing module is further configured to require the drone to obtain a real-time image of the outer surface of the docked drone, and parse the real-time image to obtain the complete target QR code information;

[0019] The calculation module is further configured to determine the relative position of the desired UAV and the docking UAV based on the target QR code information and the corner point coordinate set corresponding to the target QR code;

[0020] The control module is used to control the demand UAV to dock with the docking UAV according to the relative position.

[0021] The present application provides a method and device for drone aerial docking based on multiple QR codes. By arranging QR codes of different sizes and numbers at multiple target locations in the blank area of the docking drone's outer surface, each QR code is maximized in its corresponding position, occupying all blank areas and areas with the required curvature at the arrangement location, maximizing the utilization of the blank area on the drone's outer surface. This ensures that the desired drone for docking has a corresponding QR code image with complete image information and appropriate size when flying to any relative position or distance. Furthermore, when determining the relative position of the two drones during docking, the relative speed and distance differences between the two drones are reduced, simplifying the drone aerial docking process. A set of corner point coordinates is generated by combining a three-dimensional coordinate system, allowing the aircraft to obtain real-time images and parse the target QR code information using the desired drone during flight. The relative position between the desired drone and the docking drone is then determined based on the QR code information and the set of corner point coordinates, and docking is controlled based on the relative positions. By designing QR code layouts of different positions, sizes, and shapes, the operating drone can detect at least one QR code at any flight altitude, angle, and attitude, achieving redundant recognition and improving detection robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a flowchart of Example 1 of the drone aerial docking method based on multiple QR codes provided by this application;

[0023] Figure 2 This is a schematic diagram of the structure of a docking drone shown in an exemplary embodiment of the present application;

[0024] Figure 3 This is a schematic diagram showing an exemplary embodiment of the present application in which a QR code is arranged on the top area of a drone;

[0025] Figure 4 This is a structural diagram of Example 1 of the drone aerial docking device based on multiple QR codes provided in this application. DETAILED DESCRIPTION

[0026] Exemplary embodiments are described in detail herein, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numerals in different drawings represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with this application.

[0027] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms "a," "the," and "the" used in this application are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0028] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0029] Specific embodiments are given below to introduce the technical solutions of the present application in detail.

[0030] Figure 1 This is a flow chart of the first embodiment of the drone air docking method based on multiple QR codes provided by this application. Figure 1 The method provided in this embodiment may include:

[0031] S101. Determine a blank area on the outer surface of the docking drone.

[0032] Specifically, a demanding drone refers to the drone that performs docking in a drone-to-drone aerial docking scenario; a docking drone refers to the drone docked by the demanding drone, providing resource replenishment or collaborative support to the demanding drone. During aerial docking, the demanding drone and the docking drone can remain stationary, allowing the demanding drone to dynamically dock with the docking drone, or the docking drone and the demanding drone can dynamically dock with each other.

[0033] Further, Figure 2 This is a schematic diagram of the structure of a docking drone shown in an exemplary embodiment of the present application. Figure 3 This is a schematic diagram of an exemplary embodiment of the present application showing a QR code arranged on the top area of a drone. Please also refer to Figure 2 and Figure 3 , Figure 2 and Figure 3The blank box in the figure represents the original QR code image. To provide physical space for the QR code layout and avoid conflicts with the drone's own components (such as antennas, sensors, rotors, etc.), and to ensure that the QR code can be clearly recognized, it is necessary to determine the blank area on the surface of the docked drone. A comprehensive scan can be performed on the external surface of the external drone to identify flat areas without obstructions and complex textures (such as large blank areas on the top and sides). Taking the top as an example, a location with a wide field of view that is not easily disturbed by the rotor airflow is preferred to ensure that the camera of the required drone can shoot stably.

[0034] Specifically, the method of determining the blank area on the outer surface of the docking drone includes: determining the location of the components on the outer surface of the docking drone; screening the areas without components on the outer surface of the docking drone as candidate blank areas; determining the minimum QR code side length constraint and the QR code recognition curvature constraint; dividing the candidate blank area into non-overlapping candidate blank areas under each view; eliminating unqualified blank areas inside the candidate blank areas under each view according to the QR code recognition curvature constraint; determining the boundary of the eliminated candidate blank area under each view according to the outer contour of the docking drone; eliminating the areas that do not meet the minimum QR code side length constraint from the eliminated candidate blank area under each view according to the boundary, to obtain the blank area under each view, so as to divide the blank area under each view into multiple sub-areas.

[0035] Specifically, the outer surface point cloud data of the docking drone can be obtained through laser three-dimensional scanning or structured light scanning, and the location of the component installation area of the docking drone can be marked. The other areas of the docking drone except the component installation location are used as candidate blank areas, and the candidate blank areas are divided into regions using the geometric dividing lines of the basic surfaces of the docking drone as view dividing lines. For example, the candidate blank areas are divided into top surface, side surface, bottom surface and wing surface through the view dividing line. The QR code on the top surface of the docking drone can only be completely arranged in the blank area of the top surface.

[0036] Furthermore, the curvature constraint for QR code recognition refers to the maximum curvature threshold of the surface of the candidate blank area, which is used to eliminate candidate blank areas with large surface curvatures, ensuring that when the QR code is attached to the candidate blank area, the surface of the candidate blank area is sufficiently flat to meet the visual recognition requirements. The curvature is calculated by calculating the inverse of the rate of change of the normal vector at any point on the surface of the candidate blank area in each view, and the candidate blank area in each view is triangulated. The curvature of each triangle is calculated, and unqualified blank areas with curvature greater than the curvature threshold in the candidate blank area in each view are eliminated; the minimum QR code side length constraint refers to the minimum physical side length of the QR code, which is used to ensure that the position and size of the QR code placed in the blank area meet the rubber and plastic resolution requirements of visual inspection, and to avoid QR code recognition failure due to the QR code being too small.

[0037] Furthermore, for the candidate blank areas after deleting the unqualified blank areas in each view, the inner boundary of the area is determined according to the size of the parts in the candidate blank area, the outer boundary of the area is determined according to the geometric dividing line of the docking drone, the surface geodesic distance between the inner boundary and the outer boundary of the area is calculated, and the surface geodesic distance is compared with the minimum QR code side length. The blank areas where the surface geodesic distance is less than the minimum QR code side length are eliminated, thereby obtaining the blank area on the surface of the docking drone where the QR code can be arranged.

[0038] For example, taking the top surface area as an example, the top of the docking drone is equipped with an antenna and four rotor motor bases. The outer contours of the antenna and each rotor motor base are determined, and the inner boundary is formed by extending the outer contour of each component outward for a fixed distance as the starting line. The value of the fixed distance is set according to actual needs, but it cannot be too large, it can be one centimeter. The four edges of the top surface area are used as the outer boundary. After determining the inner and outer boundaries of the top surface area, the geodesic distance between the inner and outer boundary lines is calculated. The geodesic distance refers to the shortest path length between two points on the curved surface. By comparing the geodesic distance with the minimum QR code side length, the area where the geodesic distance is less than the minimum QR code side length is eliminated, and the area where the geodesic distance is greater than or equal to the minimum QR code side length is retained.

[0039] Furthermore, the average curvature difference at the boundary of adjacent view areas is calculated, and based on this average curvature difference, whether the QR code can be placed across regions is determined. If the average curvature difference is less than a curvature difference threshold, the curvature of the adjacent view areas is considered continuous, and the QR code can be placed across adjacent view areas with continuous curvature. It should be noted that the curvature threshold and curvature difference threshold are set based on actual needs and are not limited in this embodiment.

[0040] S102 , arranging a plurality of QR codes in the plurality of target locations in the blank area, which have the same size as the blank area of the target location but are of different sizes and have different numbers.

[0041] Specifically, the target position point refers to the geometric center point of the blank area on the outer surface of the docking drone, which is used to arrange the QR code. The target position point corresponds to the actual installation position of the QR code on the outer surface of the docking drone. It should be noted that the sizes of QR codes with different numbers may be the same or different. The size of each QR code is determined by the size of the blank area corresponding to the target position point of the QR code. In other words, the QR code should be as close as possible to the size of the blank area of the target position point. In this way, by arranging multiple QR codes, it is ensured that the QR codes are evenly distributed on the surface of the drone to avoid overlap or occlusion, so that the required drone can identify the QR codes at various angles on the docking drone from different perspectives.

[0042] Optionally, the specific implementation steps of arranging multiple QR codes with the same size as the blank area of the target location point, but with different sizes and numbers at the multiple target location points of the blank area include:

[0043] (1) Dividing the blank area into multiple sub-areas according to the locations of the parts to be docked with the UAV;

[0044] Specifically, a blank area refers to the blank area within a view. For each view area of the docking drone, the blank area within that view area needs to be divided into multiple sub-areas. The docking drone's outer surface may be distributed with components such as antennas, cameras, rotors, and heat dissipation vents. The areas occupied by these components need to be excluded, leaving only unobstructed blank areas. Based on the distribution of components on the docking drone's outer surface (such as antennas and rotors), the blank area is divided into multiple first areas according to the principle of uniform distribution. Based on the flatness and continuity of the multiple first areas, a secondary split is performed to split the first area into second areas. Multiple sub-areas are formed from the first areas that have not been split twice and the multiple second areas. Each sub-area obtained after the split is an independent QR code layout area. For example, if the first area has bumps, curves, or discontinuous gaps (such as discontinuous planes at the corners of the fuselage), it needs to be further split into second areas to ensure that the surface within each sub-area is flat and unbroken, meeting the requirements for clear QR code imaging.

[0045] For example, in one possible implementation, the outer surface of a drone generally includes a top and side surfaces, wherein the area of the side surface is a large blank area. At this time, according to the principle of uniform distribution, the side surface is divided into multiple first areas, and each first area is further inspected for flatness and continuity. The first area with defects is divided again to obtain a flat and continuous second area, and the area of the second area is smaller than the first area, thereby obtaining a sub-area consisting of the first area that has not been divided twice and the second area.

[0046] (2) Calculate the center point of each sub-region and use the center point as the target position point;

[0047] Specifically, for regularly shaped sub-regions, their geometric center point is directly calculated and used as the sub-region's center point, serving as the corresponding target location. For irregularly shaped sub-regions, the center of mass is calculated using a weighted average or integral method of vertex coordinates, serving as the center point, serving as the corresponding target location. Arranging the QR code based on the center point ensures symmetrical distribution within the sub-region, reducing image distortion errors caused by installation offsets.

[0048] (3) Draw a maximum inscribed square with the target location as the center, and determine the size of the QR code based on the maximum inscribed square;

[0049] Specifically, with the target location as the center, a maximum inscribed square is drawn within the sub-region of the target location. The maximum inscribed square refers to a square drawn within the sub-region such that at least one vertex of the square is on the sub-region's boundary and does not exceed the sub-region's boundary. By drawing the maximum inscribed square, sub-regions of different shapes can generate QR codes of optimal size, avoiding the situation where the QR codes of some sub-regions are too large or too small due to forced uniformity of size, resulting in a mismatch between the QR code and the sub-region size. It also maximizes the use of blank areas, ensuring that the required drone can clearly capture the QR code in small areas. The shooting angle has been expanded, and in large areas, the QR code can be clearly captured without being very close or having a small relative speed difference.

[0050] (4) Generate a unique code for the QR code according to the target location point, and store the target location point and its corresponding QR code in a resource library.

[0051] Specifically, the unique QR code consists of a location identifier and a sequence number to clearly identify the QR code's installation location and order on the docked drone. The location identifier indicates the drone's location, and the sequence number indicates the order in which the QR codes are placed within the same area, ensuring unique QR codes within the same area. For example, if a QR code is placed on the top of a drone, the code might be "TOP-01," indicating the first QR code on the top.

[0052] Furthermore, the step of arranging a QR code at the boundary of the blank area includes:

[0053] Mark the blank areas of adjacent view areas with continuous curvature as merged blank areas, determine the inner boundary of the merged blank areas based on the position of the parts, determine the outer boundary of the merged blank areas based on the outer contour of the docked drone, calculate the geodesic distance between the inner and outer boundaries of the merged blank areas, retain the merged blank areas with geodesic distances greater than the minimum QR code side length, and divide the merged blank areas into multiple sub-areas while ensuring that the merged blank areas meet the minimum QR code side length constraint. Calculate the center point of each sub-area as the target position point, draw the largest inscribed square with the target position point as the center, and arrange QR codes in each sub-area of the merged blank area. The implementation process of the relevant steps refers to the previous description and will not be repeated here.

[0054] Furthermore, the coordinates of the target location point are determined based on the coordinate values of its corresponding QR code in the three-dimensional coordinate system of the docked drone. The mapping relationship between the target location point and the code of the QR code is formed in a table format and stored in the resource library for easy subsequent identification. In this way, after the required drone parses the QR code, the corresponding target location point coordinates can be directly retrieved from the resource library, eliminating the need for on-site calculations or matching complex features, reducing computational time. Even if the QR codes in different sub-areas are the same size, their spatial positions can be clearly identified through the unique code. The uniqueness of the code and the accuracy of the coordinates ensure positioning accuracy.

[0055] S103: Generate a set of corner point coordinates of each of the two-dimensional codes according to the position of each of the two-dimensional codes.

[0056] The specific implementation steps include:

[0057] (1) Establishing a first three-dimensional coordinate system with the center of mass of the docked UAV as the origin;

[0058] (2) Calculating the coordinate position of each corner point of each of the two-dimensional codes in the first three-dimensional coordinate system according to the position of each of the two-dimensional codes, using the coordinate position as the corner point coordinate corresponding to each corner point, and combining the multiple corner point coordinates into a corresponding corner point coordinate set of the two-dimensional code.

[0059] Specifically, the center of mass of the docked drone refers to the geometric center of the mass distribution of the docked drone. It is used as the coordinate origin to construct the first three-dimensional coordinate system, providing a unified coordinate reference for all objects on the surface of the docked drone (including QR codes, parts, etc.).

[0060] Furthermore, the position of each QR code is determined by its target position point. Combined with the corresponding QR code size, the side length of each QR code can be obtained. With the target position point as the center, the offset of each corner point of the QR code relative to the target position point can be obtained. According to the offset and the coordinate position of the target position point, the three-dimensional coordinates of each corner point in the first three-dimensional coordinate system can be obtained. The corner point coordinate set of the QR code is constructed according to the corner point coordinates corresponding to the four corner points of the QR code, and the corner point coordinate set is bound together with the QR code code and stored in the resource library.

[0061] For example, in one embodiment, the coordinates of the target position point of a certain QR code are (1, 1, 1), and the side length of the QR code is L. At this time, the offsets of the four corner points of the QR code relative to the target position point are (-L / 2, L / 2, 0), (L / 2, L / 2, 0), (L / 2, -L / 2, 0), and (-L / 2, -L / 2, 0). Combined with the coordinates of the target position point, the coordinates of the four corner points relative to the origin of the first three-dimensional coordinate system are obtained (1- Based on these four corner point coordinates, a corner point coordinate set is constructed: {(1-L / 2, 1+L / 2, 1), (1+L / 2, 1+L / 2, 1), (1+L / 2, 1-L / 2, 1), (1-L / 2, 1-L / 2, 1)}. This gives the corner point coordinate set corresponding to the QR code. This corner point coordinate set represents the standard positions of the QR code, ensuring accuracy during subsequent processing of the QR code information.

[0062] S104: The demanding drone obtains a real-time image of the outer surface of the docked drone, and parses the real-time image to obtain the complete target QR code information.

[0063] Specifically, the drone is required to be equipped with a camera for capturing images. This camera can be used to capture real-time images of the docked drone's exterior. This camera is a binocular camera, an imaging device that simulates the principles of human binocular vision. A binocular camera consists of two monocular cameras with identical (or similar) parameters arranged in parallel, maintaining a fixed baseline distance (usually a few centimeters to tens of centimeters) between them. Its core working principle is to use the left and right cameras to capture images of the same object from different perspectives, calculate scene depth information using the principle of parallax, and then obtain the object's three-dimensional spatial coordinates. After obtaining the real-time image, the following also needs to be done:

[0064] (1) performing time domain filtering on the real-time image using a sliding smoothing filter;

[0065] Specifically, sliding smoothing filtering is a time-domain filtering method that can smooth out instantaneous noise, thereby obtaining a more stable QR code recognition signal. By averaging the pixel values of multiple consecutive frames of images in a time series, image noise can be reduced and inter-frame jitter can be eliminated. By setting a sliding window, the pixel values at the same pixel position of each frame of the real-time image within the sliding window are arithmetic averaged to obtain the filtered value of the pixel in the current frame. As new frames are input, the window slides backward, discarding the oldest frame and adding the latest frame, repeating the above averaging process. By performing time-domain filtering on real-time images, random fluctuations in pixel values caused by thermal noise of the camera sensor, sudden changes in ambient light, etc. can be suppressed, making the image smoother in the time dimension. Key features such as the edges of the QR code can be further smoothed in the time domain to avoid feature misjudgment due to single-frame noise.

[0066] (2) A first-order discrete low-pass filter is used to perform frequency domain filtering on the real-time image after time domain filtering to obtain a denoised real-time image.

[0067] Specifically, noise in an image typically corresponds to high-frequency components (such as salt-and-pepper noise and fine texture interference), while valid signals (such as the edges and outlines of a QR code) are mostly low- or medium-frequency components. A first-order discrete low-pass filter achieves denoising by attenuating high-frequency components and retaining low-frequency components.

[0068] The real-time image after time domain filtering can be filtered in the frequency domain using the following formula:

[0069] ;

[0070] in, is the pixel value of the real-time image after time domain filtering;

[0071] is the filter coefficient.

[0072] The filter coefficient determines the filter strength. The smaller the filter coefficient, the smoother the filtered signal, but the worse the real-time performance. The larger the filter coefficient, the weaker the noise suppression effect, but the faster the response speed. Therefore, you can set the corresponding filter coefficient according to the actual situation.

[0073] Through sliding smoothing filtering, jitter and slowly changing noise between real-time image frames are first eliminated in the time dimension. First-order low-pass filtering then suppresses high-frequency random noise in the real-time image after time domain filtering in the spatial frequency domain, forming a complementary effect and improving the quality of the real-time image. In this way, in the denoised image, the boundaries between the black and white modules of the QR code are clearer, and the contour detection algorithm can accurately extract corner points, avoiding misjudgment or omission of corner points due to noise.

[0074] Furthermore, the steps for parsing the complete target QR code information from the real-time image include:

[0075] (1) Locating a plurality of QR code position points from the real-time image, parsing the plurality of QR code position points respectively, and obtaining the parsed QR code information;

[0076] Specifically, the real-time image can be recognized through the image recognition algorithm to locate multiple QR code position points in the real-time image. It can be understood that when the binocular camera on the demand drone shoots the docked drone to obtain a real-time image, there is a surface image of the docked drone under the corresponding shooting perspective in the real-time image, that is, multiple QR codes on the docked drone. By recognizing the real-time image through the image recognition algorithm, the multiple QR code position points in the real-time image can be located, and each QR code position point can be further decoded to obtain the QR code information of the QR code under each QR code position point. The QR code information includes the QR code number and the coordinates of each corner point of the QR code.

[0077] (2) locating the docked drone parts in the real-time image, and determining the coding ranges of the multiple QR code position points based on the corresponding shooting ranges of the docked drone parts and the real-time image;

[0078] Specifically, in combination with the above description, the docked drone parts in the real-time image are identified through the image recognition algorithm to obtain specific information of the docked drone parts, and the area of the docked drone in the real-time image is determined based on the parts information. The shooting range corresponding to the real-time image is further determined based on the intrinsic and extrinsic parameters of the binocular camera. Combined with the docked drone parts and the shooting range, the coding range of multiple QR code position points in the real-time image can be determined.

[0079] For example, in one embodiment, the component of the docking drone in the real-time image is identified as the antenna. Based on the antenna, it can be determined that the area captured in the real-time image is the top area of the docking drone. The QR code layout area near the antenna is further determined based on the shooting range, and the QR code coding range pre-arranged in the area is associated. This can avoid misjudging the QR code outside the shooting range as the target QR code, thereby reducing cross-region false detection.

[0080] (3) Perform the first verification on the successfully parsed QR code information according to the encoding rules;

[0081] Specifically, the encoding rule refers to the rule for setting a corresponding number for each QR code when the QR code is arranged on the docking drone. The rule is unique, ensuring that the encoding of each QR code is unique on the surface of the docking drone. The QR code number and real-time corner point coordinates of each QR code are obtained according to the decoding information. The corner point coordinate set corresponding to each QR code is searched in the resource library according to the QR code number, and the real-time corner point coordinates and the corresponding corner point coordinate set are compared. The QR codes whose real-time corner point coordinates do not match the corresponding corner point coordinate set are eliminated from the QR codes, so as to perform the first verification of the QR code in the real-time image.

[0082] (4) Perform a second verification on the QR code information that has passed the first verification according to the coding range.

[0083] Specifically, the coding range includes the standard QR code coding range of the QR codes in the area. The QR code corresponding to the QR code retained after the first verification is compared with the standard QR code coding of each QR code in the coding range to check whether the coding of the QR code identified in the real-time image is correct, avoid false detection, and eliminate QR codes whose QR code coding is different from the standard QR code coding of each QR code in the coding range. In this way, the QR code in the real-time image is verified for the second time to ensure that the complete and correct target QR code is identified, improve the anti-interference ability and recognition accuracy, and provide reliable data for the relative position calculation of the drone. Furthermore, after parsing and obtaining the complete target QR code information, it also includes:

[0084] (1) Obtaining a set of corner point coordinates corresponding to each QR code according to the target QR code information;

[0085] Specifically, the encoding information of each target QR code can be obtained from the target QR code information, see the above description. After obtaining the encoding information of the target QR code, the corner point coordinate set corresponding to the QR code can be directly searched from the resource library.

[0086] (2) calculating the square difference between the corner point coordinates of each target QR code information within a specified time period and the corner point coordinate set;

[0087] Specifically, the specified time refers to a specified sliding time window. According to the target QR code information, for each frame within the specified sliding time window, the two-dimensional pixel corner coordinates of the QR code are extracted. For each corner point, the square difference between it and the corresponding corner point coordinates in the corner point coordinate set is calculated. The square differences of the four corner points are summed to obtain the square difference of the target QR code. The square difference reflects the degree of deviation between the actual detection position of the QR code corner point and the theoretical position. The smaller the value, the higher the recognition accuracy.

[0088] (3) Determine the accuracy of the target two-dimensional code information based on the relationship between the square difference and the error threshold, and correct the target two-dimensional code information based on the accuracy.

[0089] Specifically, the error threshold is set according to actual needs and is not limited in this embodiment. If the square difference is less than or equal to the error threshold, it indicates that the target QR code information is accurate. If the square difference is greater than the error threshold, it indicates that the target QR code information has errors and needs to be corrected.

[0090] Furthermore, the target QR code information can be corrected through single-frame correction, multi-frame fusion correction and other methods, providing reliable visual input for subsequent relative position solution and docking control. For the specific implementation process of correcting the target QR code information, please refer to the description in the relevant technology, which will not be repeated here.

[0091] S105: Determine the relative positions of the requesting UAV and the docking UAV based on the target QR code information and the corner point coordinate set corresponding to the target QR code.

[0092] Specific implementation steps may include:

[0093] (1) Obtaining the 2D corner coordinates of the target QR code in the image pixel coordinate system according to the target QR code information;

[0094] Specifically, the 2D corner coordinates of the target QR code in the image pixel coordinate system refer to the plane coordinates of the four corner points of the QR code in the real-time image. By constructing a two-dimensional coordinate system with one corner of the real-time image as the origin and constructing an image pixel coordinate system in pixels, the 2D corner coordinates of the target QR code are obtained.

[0095] (2) solving the 2D corner point coordinates and the corner point coordinate set based on the PNP algorithm, and calculating the first relative position between the target two-dimensional code and the camera according to the 2D corner point coordinates and the corner point coordinate set;

[0096] Specifically, calculating the 2D corner point coordinates and the corner point coordinate set based on the PNP algorithm includes:

[0097] Determine the parameters of the camera on the desired drone that will capture the image. Use the PNP algorithm to calculate the rotation matrix and translation vector between the image pixel coordinate system and the first three-dimensional coordinate system. Calculate the first relative position based on the camera parameters, rotation matrix, and translation vector. The rotation matrix and translation vector represent the camera's pose relative to the docked drone, representing the first relative position.

[0098] Specifically, it can be calculated according to the following formula:

[0099] ;

[0100] in, is the distance ratio between the corner point and the camera;

[0101] is the 2D corner coordinate of a corner point;

[0102] is the coordinate of the corner point corresponding to the corner point;

[0103] is a 3*3 rotation matrix;

[0104] is a 3*1 translation vector;

[0105] are the camera parameters.

[0106] (3) calculating a second relative position between the camera shooting the real-time image and the required UAV based on an external parameter matrix of the camera relative to the required UAV;

[0107] Specifically, the extrinsic parameter matrix is used to describe the camera's installation position and posture on the desired drone, that is, the camera's second relative position relative to the desired drone. The pose matrix is constructed based on the rotation matrix and translation vector, and the second relative position is obtained by combining the pose matrix and the extrinsic parameter matrix. The second relative position can be calculated using the following formula:

[0108] ;

[0109] in, is the external parameter matrix;

[0110] is the pose matrix.

[0111] (4) Calculating the relative position of the request UAV and the docking UAV based on the first relative position and the second relative position.

[0112] Specifically, a world coordinate system is constructed with the starting point of the required UAV's flight path as the origin, and the coordinate positions of the required UAV and the docking UAV in the world coordinate system are obtained respectively. Further, the relative positions can be calculated using the following formula:

[0113] ;

[0114] in, The coordinate position of the docked drone in the world coordinate system;

[0115] The coordinate position of the required drone in the world coordinate system;

[0116] is a 3*3 rotation matrix;

[0117] is the position vector in the pose matrix.

[0118] Furthermore, Kalman filtering is performed on the relative position to smooth it.

[0119] S106: Control the demand UAV to fly toward the docking UAV according to the relative position.

[0120] Specifically, the relative position includes translation deviation and attitude deviation. The translation and attitude of the demand UAV are controlled based on the PID algorithm in combination with the translation deviation and attitude deviation, so that the demand UAV can dock with the docking UAV. It should be noted that during the docking flight, the position of the docking UAV can be fixed or the demand UAV and the docking UAV can fly relative to each other. When the demand UAV and the docking UAV fly relative to each other, the translation and attitude of the demand UAV and the docking UAV are controlled based on the PID algorithm in combination with the translation deviation and attitude deviation to ensure successful docking. During the docking process, the relative position is updated in real time, and the docking of the demand UAV and the docking UAV is controlled based on the control amount updated according to the real-time updated relative position. It should be noted that the update frequency of the relative position is set according to actual needs and is not limited in this embodiment.

[0121] Furthermore, during the docking process between the demand UAV and the docking UAV, the following steps are further included:

[0122] (1) counting a first number of a plurality of target QR codes in the real-time image;

[0123] Specifically, the camera of the drone is required to continuously capture images of the docked drone, and all complete QR codes in the image are identified through a QR code detection algorithm. The complete QR codes in the image are determined as target QR codes, and the first number of target QR codes is further determined.

[0124] (2) comparing the corner point coordinates of each target two-dimensional code in the plurality of target two-dimensional codes with the corner point coordinate set, and calculating a second number of target two-dimensional codes whose corner point coordinates are different from the corner point coordinate set;

[0125] Specifically, the corresponding set of corner point coordinates is obtained according to the encoding of the target QR code, the corner point coordinates are projected onto the image plane according to the calculation formula of the first relative position to obtain the standard pixel coordinates, the deviation between the actual pixel coordinates of the corner points and the standard pixel coordinates is calculated, and the second number of target QR codes where the corner points have a non-zero deviation is counted.

[0126] (3) determining a false detection rate of the camera based on a ratio of the second number to the first number;

[0127] Specifically, the false positive rate can be calculated using the following formula:

[0128] ;

[0129] in, is the second quantity;

[0130] The first quantity.

[0131] (4) Adjusting the docking operation progress according to the numerical relationship between the false detection rate and the false detection threshold.

[0132] Specifically, the false detection threshold is set according to actual needs and is not limited in this embodiment. When the false detection rate is greater than the false detection threshold, the connection is stopped.

[0133] Furthermore, when monitoring the QR code recognition results in real time, it is necessary to dynamically adjust the judgment threshold based on the noise level of the current environment and the distribution of the detection data. The dynamic threshold can be expressed by the following formula:

[0134] ;

[0135] in It is an empirical coefficient, determined based on actual test data;

[0136] is the statistical mean of the current detection signal;

[0137] is the standard deviation.

[0138] When the deviation of detected QR code information (such as corner position and QR code size) exceeds a dynamic threshold, it is judged as a false detection or anomaly, triggering the protection mechanism. In addition, if the number of QR codes detected is less than the preset standard, it is also considered an abnormal state.

[0139] The method for aerial docking of drones based on multiple QR codes provided in this embodiment ensures that the required drone can detect at least one QR code at various flight altitudes, angles, and attitudes by arranging QR codes of different sizes and numbers at multiple target position points in the blank area on the outer surface of the docking drone. The multiple redundant design enables the system to maintain stable operation in complex flight environments, overcoming the problem of recognition failure of traditional visual servo systems when there is insufficient light or large changes in flight attitude. A set of corner coordinates of each QR code is generated by combining a three-dimensional coordinate system established with the center of mass of the docking drone as the origin. The real-time image is acquired using the camera carried by the required drone and subjected to dual denoising processing through sliding smoothing filtering and time-domain and frequency-domain. The sliding smoothing filter first eliminates jitter and slow-changing noise between frames of the real-time image in the time dimension. The first-order low-pass filter then suppresses high-frequency random noise in the real-time image after time-domain filtering in the spatial frequency domain. The multi-stage filtering works synergistically to enhance the denoising capability and improve the filtering effect. The 2D corner coordinates of the target QR code are accurately parsed. Then, based on the PNP The algorithm calculates the mapping relationship between the QR code and the three-dimensional corner point coordinate set, thereby achieving high-precision calculation of the relative position of the two drones. This allows the on-demand drone to quickly determine the three-dimensional relative position between the two drones after recognizing the QR code, significantly improving the accuracy and real-time performance of position estimation. Furthermore, a squared error correction mechanism is used to compare the real-time detected QR code corner point coordinates with a pre-stored set of corner point coordinates. By calculating the relationship between the squared error and the error threshold, recognition deviations are dynamically corrected to ensure the accuracy of the QR code information. Furthermore, by counting the number of target QR codes and corner point deviations in the real-time image, the false detection rate is calculated and compared with a dynamic threshold. When the false detection rate exceeds the limit, the docking progress is automatically adjusted or a protection mechanism is triggered, enabling real-time monitoring and response to false detections. The system also monitors the QR code recognition status in real time, employing dynamic threshold adjustment and matching judgment algorithms to ensure that the QR code recognition results meet expectations. If the number of recognized QR codes does not meet the preset standard, the system automatically triggers a protection mechanism to prevent position estimation errors caused by false detections. This false detection prevention algorithm effectively reduces the false detection rate, ensuring stable operation of the system even in complex environments. Through the false detection recognition mechanism, the system can automatically correct errors when they are detected, avoiding program interruptions, improving system reliability and fault tolerance, significantly enhancing the anti-interference capability and positioning robustness of the UAV in dynamic environments, and can also promptly initiate emergency processing when abnormalities are identified, ensuring that the dual-machine docking process is always in a safe and stable state.

[0140] Corresponding to the aforementioned embodiment of a drone air docking method based on multiple QR codes, the present application also provides an embodiment of a drone air docking device based on multiple QR codes.

[0141] Figure 4 This is a structural diagram of the first embodiment of the drone air docking device based on multiple QR codes provided by this application. Figure 4 The device provided in this embodiment includes a placement module 410, a calculation module 420 and a control module 430; wherein,

[0142] The arrangement module 410 is used to determine a blank area on the outer surface of the docking drone;

[0143] The arrangement module 410 is further configured to arrange a plurality of QR codes having the same size as the blank area of the target location point, but having different sizes and numbers, at the plurality of target location points of the blank area;

[0144] The calculation module 420 is configured to generate a set of corner point coordinates of each of the two-dimensional codes according to the position of each of the two-dimensional codes;

[0145] The calculation module 420 is further configured to request the drone to obtain a real-time image of the outer surface of the docked drone, and parse the real-time image to obtain the complete target QR code information;

[0146] The calculation module 420 is further configured to determine the relative position of the requesting UAV and the docking UAV based on the target QR code information and the corner point coordinate set corresponding to the target QR code;

[0147] The control module 430 is configured to control the requesting UAV to fly toward the docking UAV according to the relative position.

[0148] The device of this embodiment can be used to perform Figure 1 The steps, specific implementation principles and implementation processes of the method embodiment shown are similar and will not be repeated here.

[0149] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0150] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0151] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for drone aerial docking based on multiple QR codes, characterized in that: The method comprises: Determine the clear area on the outer surface of the docking drone; Arranging a plurality of QR codes in the plurality of target position points in the blank area, which are of the same size as the blank area of the target position points but of different sizes and with different numbers; Generate a set of corner point coordinates of each of the two-dimensional codes according to the position of each of the two-dimensional codes; The drone is required to obtain a real-time image of the outer surface of the docked drone, and parse the complete target QR code information from the real-time image; Determine the relative positions of the desired UAV and the docking UAV based on the target QR code information and the corner point coordinate set corresponding to the target QR code; The demand UAV is controlled to fly toward the docking UAV according to the relative position.

2. The method according to claim 1, characterized in that The method comprises arranging a plurality of QR codes having the same size as the blank area of the target location point, different sizes and different numbers at the plurality of target location points in the blank area; comprising: Dividing the blank area into multiple sub-areas according to the locations of the parts to be docked with the drone; Calculate the center point of each sub-area and use the center point as the target position point; Draw a maximum inscribed square with the target location point as the center, and determine the size of the QR code according to the maximum inscribed square; A unique code is generated for the two-dimensional code according to the target location point, and the target location point and its corresponding two-dimensional code are stored in a resource library.

3. The method according to claim 1, characterized in that Determining the blank area on the outer surface of the docking drone includes: Determining the locations of the components on the outer surface of the docked drone; Screening areas without components on the unmanned outer surface of the docking unit as candidate blank areas; Determine the minimum QR code side length constraint and QR code recognition curvature constraint; Dividing the candidate blank area into candidate blank areas in different non-overlapping views; Eliminating unqualified blank areas within the candidate blank areas in each view according to the curvature constraint of the QR code recognition; Determine the boundaries of the candidate blank areas after elimination in each view according to the outer contour of the docked drone; After removing the candidate blank areas in each view according to the boundary, areas that do not meet the minimum QR code side length constraint are removed to obtain the blank areas in each view, so as to divide the blank areas in each view into multiple sub-areas.

4. The method according to claim 1, wherein Generating a set of corner point coordinates of each of the two-dimensional codes according to the position of each of the two-dimensional codes comprises: Establishing a first three-dimensional coordinate system with the center of mass of the docked drone as the origin; According to the position of each of the two-dimensional codes, the coordinate position of each corner point of each of the two-dimensional codes in the first three-dimensional coordinate system is calculated, the coordinate position is used as the corner point coordinate corresponding to each corner point, and multiple corner point coordinates are combined into a corresponding corner point coordinate set of the two-dimensional code.

5. The method according to claim 1, wherein The step of parsing the complete target QR code information from the real-time image includes: Locating a plurality of QR code position points from the real-time image, and respectively parsing the plurality of QR code position points to obtain successfully parsed QR code information; Locating the docked drone component in the real-time image, and determining the coding range of the multiple QR code position points according to the shooting range corresponding to the docked drone component and the real-time image; Perform the first verification on the successfully parsed QR code information according to the encoding rules; The two-dimensional code information that has passed the first verification is verified for the second time according to the coding range.

6. The method according to claim 1, characterized in that The method of determining the relative positions of the desired UAV and the docked UAV based on the target QR code information and the corner point coordinate set corresponding to the target QR code comprises: Obtaining the 2D corner coordinates of the target QR code in the image pixel coordinate system according to the target QR code information; Solving the 2D corner point coordinates and the corner point coordinate set based on a PNP algorithm, and calculating a first relative position between the target two-dimensional code and the camera according to the 2D corner point coordinates and the corner point coordinate set; Calculate a second relative position between the camera shooting the real-time image and the required drone according to an extrinsic parameter matrix relative to the required drone; The relative position of the demand UAV and the docking UAV is calculated according to the first relative position and the second relative position.

7. The method according to claim 1, characterized in that After acquiring the real-time image of the outer surface of the docked drone, the method further comprises: Performing time domain filtering on the real-time image by using a sliding smoothing filter; A first-order discrete low-pass filter is used to perform frequency domain filtering on the real-time image after time domain filtering to obtain a denoised real-time image.

8. The method according to claim 1, characterized in that After parsing and obtaining the complete target QR code information, the method further includes: Obtaining a set of corner point coordinates corresponding to each QR code according to the target QR code information; Calculate the square difference between the corner point coordinates of each target two-dimensional code information within a specified time period and the corner point coordinate set in the target two-dimensional code information; The accuracy of the target two-dimensional code information is determined according to a relationship between the square difference and an error threshold, and the target two-dimensional code information is corrected according to the accuracy.

9. The method according to claim 1, characterized in that During the docking process between the demand UAV and the docking UAV, the method includes: Counting a first number of a plurality of target QR codes in the real-time image; Comparing the corner point coordinates of each target two-dimensional code in the plurality of target two-dimensional codes with the corner point coordinate set, and calculating a second number of target two-dimensional codes whose corner point coordinates are different from the corner point coordinate set; determining a false detection rate of the camera according to a ratio of the second number to the first number; The docking operation progress is adjusted according to the numerical relationship between the false detection rate and the false detection threshold.

10. A drone aerial docking device based on multiple QR codes, characterized in that: The device includes a layout module, a calculation module and a control module; wherein, The arrangement module is used to determine the blank area on the outer surface of the docking drone; The arrangement module is further configured to arrange a plurality of QR codes in the plurality of target locations in the blank area, the QR codes having the same size as the blank area of the target location but having different sizes and numbers; The calculation module is used to generate a set of corner point coordinates of each of the two-dimensional codes according to the position of each of the two-dimensional codes; The computing module is further configured to require the drone to obtain a real-time image of the outer surface of the docked drone, and parse the real-time image to obtain the complete target QR code information; The calculation module is further configured to determine the relative position of the desired UAV and the docking UAV based on the target QR code information and the corner point coordinate set corresponding to the target QR code; The control module is used to control the demand UAV to dock with the docking UAV according to the relative position.

Images