Unmanned aerial vehicle centering method and system and storage medium

By performing feature component annotation and target detection network training on drone video data, the average area ratio and high-width ratio range are calculated, the problem of judging the parking position and angle of the drone is solved, and the reliability of the unmanned hangar system is improved. , prevent misoperation.

CN119007049APending Publication Date: 2024-11-22CHENGDU ZONGHENG PENGFEI TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411257814.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-09
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

In an unmanned hangar system, the diversity of the landing position and angle of the drone makes it impossible to determine whether it is parked in a safe stopable area surrounded by the center pole, which may lead to the center pole being accidentally clamped or the drone being caught in the hatch door. Clip, causing damage. Existing object detection algorithms and instance segmentation algorithms cannot directly detect relative position relationships.

Method used

By collecting drone video data, selecting feature components for labeling, generating component area masks, inputting to target detection network for training, outputting predicted drone component target boxes, calculating the average area ratio and high-width ratio range, and judging that no one is Whether the machine is in a safe stopable area and the angle meets the internal operating range.

Benefits of technology

Effectively determine whether the drone is located in an irregular safe and stopable area surrounded by the central pole, improve the reliability of the unmanned hangar system, and prevent damage to the central pole, drone and hatch door.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007049A_ABST
    Figure CN119007049A_ABST
Patent Text Reader

Abstract

The invention relates to the field of unmanned aerial vehicle technology and computer vision, in particular to an unmanned aerial vehicle centering method and system and a storage medium, and the method comprises the steps: firstly collecting the video data of an unmanned aerial vehicle, selecting feature parts of the unmanned aerial vehicle for marking, and obtaining a tag file; generating a part region mask according to the label file, assigning a value to a region which is not covered by the part region mask, and generating a part target frame ratio domain; inputting the label file and the real-time image received by the unmanned aerial vehicle into a target detection network for training, and outputting to obtain a predicted unmanned aerial vehicle part target frame; screening a predicted unmanned aerial vehicle component target frame, and calculating an average area ratio; according to the average area ratio, whether the unmanned aerial vehicle is located in a safe stoppable area defined by the centering rods or not is judged, and whether centering operation is carried out or not is determined; according to the method, whether the unmanned aerial vehicle meets the centering and cabin door closing requirements or not is detected in combination with the target detection network, the component area mask and the height-width ratio of the target frame, and the reliability of an unattended hangar system is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the fields of unmanned aerial vehicle technology and computer vision, and in particular to a method, system and storage medium for centering an unmanned aerial vehicle. Background Art

[0002] In an unmanned hangar system, after the drone completes its flight mission and lands on the hangar platform, it will be centered and the hatch will be closed. However, the landing position and angle of the drone are often diverse. The in-place sensor often cannot determine whether the drone is parked in the irregular safe parking area surrounded by the centering pole. If the centering pole is used to center the drone and the hatch is closed without fully confirming that the drone is parked in the safe parking area surrounded by the centering pole, the centering pole may accidentally clamp the drone or the drone may be clamped by the hatch, resulting in damage to the centering pole, the drone, the hatch, etc.

[0003] At present, the most common method is to use advanced visual detection algorithms such as target detection algorithms and instance segmentation algorithms. However, in actual applications, when the camera is fixed, the contact surface between the drone and the safe parking area surrounded by the centering pole may be blocked, and the positional relationship between the detected object and the area cannot be intuitively seen in the image. At this time, target detection algorithms, instance segmentation algorithms, etc. cannot directly detect the relative position relationship. Summary of the invention

[0004] The present invention aims to provide a method, system and storage medium for centering a drone, so as to solve the problem of effectively judging whether the drone is located in an irregular safe parking area surrounded by a centering pole and whether the parking angle conforms to the centering operation when the target detection algorithm, instance segmentation algorithm, etc. cannot directly detect the relative position relationship, so as to improve the reliability of the unmanned hangar system.

[0005] In order to achieve the above object, the present invention provides the following method:

[0006] A method for centering a drone provided by the present invention is as follows:

[0007] S1: Collect drone video data, select characteristic parts of the drone for annotation, and obtain label files;

[0008] S2: Generate a component area mask according to the label file, assign a value to the area not covered by the component area mask, and generate a component target frame ratio domain;

[0009] S3: inputting the label file and the real-time image received by the drone into the target detection network for training, and outputting the predicted drone component target frame;

[0010] S4: Filter the predicted drone component target frame and calculate the average area ratio;

[0011] S5: judging whether the UAV is located in a safe stoppable area surrounded by the centering pole and within an angle range of a centering operation according to the average area ratio and the component target frame ratio domain, and deciding whether to perform a centering operation on the UAV.

[0012] Preferably, the step of collecting drone video data, selecting characteristic components of the drone for annotation, and obtaining a label file includes: selecting each frame of the collected video data for component target annotation to obtain the label file, wherein a line in the label file represents a component target box.

[0013] Preferably, the step of generating a component area mask according to the label file and assigning a value to an area not covered by the component area mask comprises: generating n component area masks corresponding to the characteristic components according to the label file; and assigning a value of 1 to an area not covered in the middle of the component area mask.

[0014] Preferably, the step of generating n component area masks corresponding to the characteristic components according to the label file includes: generating an all-zero array mask of the same size as the image; traversing each line of data of each label file in turn; determining the class_id of each line of data of the label file and selectively assigning values ​​to the elements; after completing the traversal of all the label file data, generating the component area mask.

[0015] Preferably, the step of assigning a value of 1 to the uncovered area in the middle of the component area mask, includes: extracting the outer contour of the covered area from the component area mask; sorting according to the area of ​​the extracted outer contour; determining whether there is a small contour within the extracted outer contour, and performing outer contour processing operations; assigning a value of 1 to the uncovered area in the middle of the component area mask.

[0016] Preferably, the step of generating the component target frame ratio range includes: traversing each line of data of each label file in turn; if the class_id of the label file does not match the category number corresponding to the component in the preset aspect ratio range, then the aspect ratio tanθ is not calculated; if the class_id of the label file matches the category number corresponding to the component in the preset aspect ratio range, then the aspect ratio tanθ is calculated by the height h and width w of the corresponding component target frame;

[0017]

[0018] Among them, θ represents the angle between the diagonal line of the component target box and the lower frame line;

[0019] All the values ​​of the aspect ratio tanθ are saved in a list to obtain a list of aspect ratio tanθ values;

[0020] After completing the traversal of all the label files, the minimum and maximum values ​​of the aspect ratio tanθ in the aspect ratio tanθ value list are obtained to form the aspect ratio value range of each component target frame.

[0021] Preferably, the step of screening the predicted drone component target frame and calculating the average area ratio includes: calculating the area ratio P of the intersection of n predicted drone component target frames and the corresponding component area mask to the predicted component target frame. i :

[0022]

[0023] Then calculate the average area ratio μ:

[0024]

[0025] Preferably, the step of judging whether the UAV is located in the safe stoppable area surrounded by the centering pole and the angle range of the centering operation according to the average area ratio and the component target frame ratio domain comprises: if the value of μ is equal to 1, then the n predicted component target frames are all located inside the corresponding component area mask, and the UAV is located in the safe stoppable area surrounded by the centering pole; if the value of μ is not equal to 1, then at least one component target frame intersects with the corresponding component area mask or is outside the corresponding component area mask, and the UAV is outside the safe stoppable area surrounded by the centering pole; if the aspect ratio tanθ of the predicted component target frame is within the preset aspect ratio value range, the parking angle of the UAV meets the angle range of the centering operation; if the aspect ratio tanθ of the predicted component target frame is outside the preset aspect ratio value range, the parking angle of the UAV does not meet the angle range of the centering operation.

[0026] Preferably, the step of deciding whether to perform the drone centering operation includes: if the drone is parked in a safe parking area and meets the angle range of the centering operation, then performing the drone centering operation of the centering pole; if the drone is not parked in a safe parking area or meets the angle range of the centering operation, then terminating the drone centering operation of the centering pole and issuing a warning.

[0027] The present invention provides a centering system for a drone:

[0028] Collection module: used to collect video data of the drone parked on the boundary of the irregular safe parking area surrounded by the centering pole, and the drone's nose is facing the angle range that meets the centering operation;

[0029] Annotation module: annotate the characteristic parts of the drone;

[0030] Assignment module: assign a value to the area not covered by the mask in the middle of the component area;

[0031] Prediction module: inputs the real-time image received by the drone into the trained network and outputs the predicted target box of the drone parts;

[0032] Calculation module: Calculate the average area ratio of the intersection of the drone component target frame and the corresponding component area mask to the predicted component target frame;

[0033] Judgment module: judge whether the drone is in the safe parking area surrounded by the centering pole and the angle range of the centering operation;

[0034] Operation module: perform centering operations on the drone.

[0035] The centering system of a drone provided by the present invention also includes:

[0036] A memory storing executable program code;

[0037] a processor coupled to the memory;

[0038] The processor calls the executable program code stored in the memory to execute the centering method of the drone.

[0039] The computer storage medium of a drone provided by the present invention is:

[0040] The computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the centering method of the drone.

[0041] The beneficial effects of the present invention are embodied in: the present invention combines the target detection network, the component area mask, and the aspect ratio of the target frame to detect whether the drone is parked in the irregular safe parking area surrounded by the centering pole and whether the parking angle conforms to the centering operation; improves the reliability of the unmanned hangar system, and provides an additional guarantee for the safety of the drone and the hangar; inputs the label file into the target detection network for training, inputs the real-time image into the trained network, and outputs the predicted drone component target frame; inputs the label file into the target detection network for training to obtain the drone component target detection model; pre-processes the real-time image, inputs it into the trained drone component target detection model, and outputs the predicted drone component target frame; each predicted drone component target frame includes coordinates and confidence levels belonging to each category. The technical solution provided by the present invention can effectively determine whether the drone is located in the irregular safe parking area surrounded by the centering pole, thereby improving the reliability of the unmanned system. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0043] Figure 1 The present invention is a flowchart of a method for centering a drone provided by an embodiment of the present invention.

[0044] Figure 2 The present invention is a flowchart of a UAV centering system provided by an embodiment of the present invention.

[0045] Figure 3 It is a schematic diagram of generating a component area mask with only one outline provided by an embodiment of the present invention.

[0046] Figure 4 It is a schematic diagram of the positional relationship between components and regional masks provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the technical personnel in the technical field better understand the scheme of the present invention, the technical scheme in the embodiment of the present invention will be clearly and completely described below in combination with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0049] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0050] At present, the most common method is to use advanced visual detection algorithms such as target detection algorithms and instance segmentation algorithms. However, in actual applications, when the camera is fixed, the contact surface between the drone and the safe parking area surrounded by the centering pole may be blocked, and the positional relationship between the detected object and the area cannot be intuitively seen in the image. At this time, target detection algorithms, instance segmentation algorithms, etc. cannot directly detect the relative position relationship.

[0051] The present invention aims to provide a method, system and storage medium for centering a drone, so as to solve the problem of effectively judging whether the drone is located in an irregular safe parking area surrounded by a centering pole and whether the parking angle conforms to the centering operation when the target detection algorithm, instance segmentation algorithm, etc. cannot directly detect the relative position relationship, so as to improve the reliability of the unmanned hangar system.

[0052] A specific embodiment of the present invention provides a method for centering a drone. Figure 1 As shown, the following steps are included:

[0053] In the unmanned hangar, determine whether the drone is located in the irregular safe parking area surrounded by the centering pole, and whether the parking angle of the drone is within the angle range of the centering operation. Then, based on the landing and parking position and angle of the drone, decide whether to perform the centering operation and the door closing operation. The detailed process is as follows:

[0054] In step S1, drone video data is collected, characteristic components of the drone are selected for annotation, and a label file is obtained.

[0055] In the embodiment of the present invention, the centering rod is opened on the hangar platform, and the drone is parked on the boundary of the irregular safe parking area surrounded by the centering rod, and the nose of the drone is oriented within the angle range that meets the centering operation, and the drone is moved along the boundary of the safe parking area to collect video data;

[0056] Select the characteristic parts of the drone for labeling, where the characteristic parts of the drone can be the landing gear of the drone, the left wing of the drone, the right wing of the drone, the tail wing of the drone or the nose of the drone. In order to obtain a more accurate judgment of the parking and centering of the drone, the selected characteristic parts are greater than or equal to 2, and the selected characteristic parts are different according to the direction of the drone's movement. For example, if the drone is parked on the left side of the safe parking area, the left wing, landing gear and nose of the drone can be selected as characteristic parts. If the drone is parked on the right side of the safe parking area, the right wing, landing gear and nose of the drone can be selected as characteristic parts.

[0057] Get the images in the video data. Each image corresponds to a label file. The color of the image is not required. It can be a color image or a black and white image. One line in the label file represents a component target frame. The format of each line is: class_id xywh. Among them, class_id represents the category of the component target frame, (x, y) represents the coordinates of the center point of the component target frame, and h and w represent the height and width of the component target frame respectively.

[0058] In step S2, a component area mask is generated according to the label file, and a value is assigned to the area not covered by the component area mask to generate a component target frame ratio domain.

[0059] In the embodiment of the present invention, taking the generation of a drone landing gear area mask as an example:

[0060] First, generate three component area masks corresponding to the characteristic components according to the label file:

[0061] Generate an all-zero array mask of the same size as the image;

[0062] Iterate through each line of each label file in turn;

[0063] If the class_id of a row in the label file is the category number corresponding to the drone landing gear, and the category number of the drone landing gear is pre-set, the elements in the area formed by the corresponding target box are assigned a value of 1; if the class_id of a row in the label file does not match the category number corresponding to the drone landing gear, the elements in the area formed by the corresponding target box are not assigned a value; after completing the traversal of all label files, a binary mask of the drone landing gear area will be generated.

[0064] Secondly, if Figure 3 As shown, the area not covered by the component area mask is assigned a value of 1:

[0065] Extract contours from the component region mask;

[0066] Sort by contour area;

[0067] If there is a small contour, draw the small contour area and assign the elements in the area to 1;

[0068] The component area mask and the small outline area mask are bitwise ORed to obtain a component area mask with only one large outline.

[0069] If no small contour exists, the assignment step is skipped;

[0070] Take the generation of the height-to-width ratio range of the target frame of a drone landing gear as an example:

[0071] Iterate through each line of each label file in turn;

[0072] If the class_id of a row is the category number corresponding to the drone landing gear, the aspect ratio tanθ is calculated for the height h and width w of the corresponding target box:

[0073]

[0074] Among them, θ represents the angle formed by the diagonal line of the component target box and the lower frame line.

[0075] Save all the values ​​of aspect ratio tanθ into a list;

[0076] After traversing all label files, find the minimum and maximum values ​​of the aspect ratio in the list to form the aspect ratio value range of the drone landing gear target frame.

[0077] In step S3, the label file and the real-time image received by the drone are input into the target detection network for training, and the predicted drone component target box is output.

[0078] In an embodiment of the present invention, the label file is input into the target detection network for training to obtain a drone component target detection model;

[0079] Preferably, the target detection model is a YOLOv8 target detection model.

[0080] Preprocess the real-time image, input it into the trained drone component target detection model, and output the predicted drone component target box;

[0081] Each predicted drone part object box includes coordinates and confidence that it belongs to each category.

[0082] In the predicted drone component target boxes, there may be multiple component target boxes belonging to the same category (component).

[0083] In step S4, the predicted drone component target boxes are screened and the average area ratio is calculated.

[0084] In the embodiment of the present invention, the predicted drone component target frames are filtered, and only one component target frame with the largest confidence value is retained for each category (component). Taking the prediction of the three components of the drone landing gear, right aileron, and right rear rotor as an example, three component target frames are obtained: the drone landing gear target frame, the right aileron target frame, and the right rear rotor target frame; the three predicted component target frames Box are calculated respectively. i The area and the corresponding component area Mask i The ratio of the intersection of the area of ​​​​the predicted component target box area P i :

[0085]

[0086] Then find the average area ratio μ:

[0087]

[0088] In step S5, based on the average area ratio and the component target frame ratio domain, it is determined whether the drone is located in the safe parking area surrounded by the centering pole and the angle range of the centering operation, and whether to perform the drone centering operation, such as Figure 4 As shown, the possible positional relationship between the predicted component target and the component area mask is as follows Figure 2 As shown in the figure, the red box represents the predicted part target box, and the white area represents the part area.

[0089] In the embodiment of the present invention, if the value of μ is equal to 1, it means that the three predicted component targets are all located inside the corresponding component area mask, and the drone is located in the safe parking area surrounded by the centering pole;

[0090] If the value of μ is not equal to 1, it means that the three predicted component targets intersect with the corresponding component area mask or are outside the corresponding component area mask, and the drone is not located in the safe parking area surrounded by the centering pole.

[0091] Calculate the aspect ratios of the three predicted component target boxes respectively; that is:

[0092]

[0093] Among them, θ represents the angle between the diagonal line of the component target box and the lower frame line;

[0094] Determine whether the parking angle of the drone is within the angle range of the centering operation based on the aspect ratio;

[0095] If the predicted aspect ratio tanθ of the component target frame is within the pre-obtained aspect ratio range, the parking angle of the drone is within the angle range of the centering operation;

[0096] On the contrary, if the predicted aspect ratio tanθ of the component target frame is outside the pre-obtained aspect ratio value range, the parking angle of the drone does not meet the angle range of the centering operation;

[0097] Whether to perform the centering operation or the door closing operation is determined based on the landing and parking position and angle of the drone.

[0098] In the embodiment of the present invention, if the drone is parked in a safe parking area and within the centering operation angle range, the centering operation of the centering pole and the door closing operation can be safely performed;

[0099] If the UAV is not parked in a safe parking area or within the centering operation angle range, the centering operation of the centering pole is terminated, the hatch is closed and a warning is issued. The warning can be issued by setting an indicator light on the hangar to provide a prompt; or it can be issued in the UAV management platform system, where a warning of a fault or failure to center the UAV is popped up.

[0100] The present invention provides a drone landing hangar position and angle detection system, such as Figure 2 As shown:

[0101] Collection module: used to collect video data of the drone parked on the boundary of the irregular safe parking area surrounded by the centering pole, and the drone's nose is facing the angle range that meets the centering operation;

[0102] Annotation module: annotate the characteristic parts of the drone;

[0103] Assignment module: Assign values ​​to the uncovered area in the middle of the component area mask;

[0104] Prediction module: inputs the real-time image received by the drone into the trained network and outputs the predicted target box of the drone parts;

[0105] Calculation module: Calculate the average area ratio of the intersection of the drone component target box and the corresponding component area mask to the predicted component target box;

[0106] Judgment module: judge whether the drone is in the safe parking area surrounded by the centering pole and the angle range of the centering operation;

[0107] Operation module: perform centering operations on the drone.

[0108] The centering system of a drone provided by the present invention also includes:

[0109] A memory storing executable program code;

[0110] a processor coupled to the memory;

[0111] The processor calls the executable program code stored in the memory to execute the centering method of the drone.

[0112] The computer storage medium of a drone provided by the present invention is:

[0113] The computer storage medium stores computer instructions, which are used to execute the centering method of the drone when the computer instructions are called.

[0114] Embodiment 1

[0115] In the unmanned hangar, determine whether the drone is located in the irregular safe parking area surrounded by the centering pole, and whether the parking angle of the drone meets the angle range required for centering. Then, based on the landing and parking position and angle of the drone, decide whether to perform the centering operation and the door closing operation. The detailed process is as follows:

[0116] Simulate the drone to park in a safe parking area and within the required centering angle range to collect data;

[0117] Open the centering pole on the hangar platform and park the drone on the boundary of the irregular safe parking area surrounded by the centering pole, with the nose facing within the angle range that meets the centering requirements. Move the drone and collect video data.

[0118] Three characteristic parts of the drone are selected, namely the tripod, right aileron and right rear rotor of the drone. The component targets are annotated for each color image to obtain a label file.

[0119] Each color image corresponds to a label file. One line in the label file represents a component target box. The format of each line is: class_id xywh. Among them, class_id represents the category to which the component target box belongs, (x, y) represents the coordinates of the center point of the component target box, and h and w represent the height and width of the component target box respectively.

[0120] Generate three component area masks corresponding to the characteristic components according to the label file, namely the drone tripod area mask, the right aileron area mask, and the right rear rotor area mask;

[0121] Take the generation of a drone tripod area mask as an example:

[0122] First, generate n component area masks corresponding to the characteristic components according to the label file:

[0123] Generate an all-zero array mask of the same size as the image;

[0124] Iterate through each line of each label file in turn;

[0125] If the class_id of a row is the category number corresponding to the drone tripod, the elements in the area formed by the corresponding target box are assigned a value of 1;

[0126] After traversing all label files, a binary mask of the drone tripod area will be generated.

[0127] Secondly, assign a value of 1 to the area not covered by the component area mask:

[0128] Extract contours from the component region mask;

[0129] Sort by contour area;

[0130] If there is a small contour, draw the small contour area and assign the elements in the area to 1;

[0131] The component area mask and the small outline area mask are bitwise ORed to obtain a component area mask with only one large outline.

[0132] If no small contours exist, steps (3) and (4) are skipped.

[0133] Generate three component target frame aspect ratio value ranges corresponding to the characteristic components according to the label file, which are the target frame aspect ratio value range of the UAV tripod, the target frame aspect ratio value range of the right aileron, and the target frame aspect ratio value range of the right rear rotor;

[0134] Take the generation of the height-to-width ratio range of the drone tripod target frame as an example:

[0135] Iterate through each line of each label file in turn;

[0136] If the class_id of a row is the category number corresponding to the drone tripod, the aspect ratio tanθ is calculated for the height h and width w of the corresponding target box:

[0137]

[0138] Among them, θ represents the angle formed by the diagonal line of the component target box and the lower frame line.

[0139] Save all the values ​​of aspect ratio tanθ into a list;

[0140] After traversing all label files, find the minimum and maximum values ​​of the aspect ratio in the list to form the aspect ratio value range of the drone tripod target frame.

[0141] Input the label file into the target detection network for training, input the real-time image into the trained network, and output the predicted target box of the drone parts;

[0142] Input the label file into the target detection network for training to obtain the drone component target detection model;

[0143] Preferably, the target detection model is a YOLOv8 target detection model.

[0144] Preprocess the real-time image, input it into the trained drone component target detection model, and output the predicted drone component target box;

[0145] Each predicted drone part object box includes coordinates and confidence that it belongs to each category.

[0146] In the predicted drone component target boxes, there may be multiple component target boxes belonging to the same category (component).

[0147] The predicted drone component target frames are filtered, and only one component target frame with the largest confidence value is retained for each category (component). Three component target frames are obtained: the drone tripod target frame, the right aileron target frame, and the right rear rotor target frame;

[0148] Calculate the target boxes of the three predicted components respectively i The area and the corresponding component area Mask i The ratio of the intersection of the area of ​​​​the predicted component target box area P i :

[0149]

[0150] Then find the average:

[0151]

[0152] According to the average ratio μ, it is judged whether the UAV is in the safe parking area surrounded by the centering pole;

[0153] If the value of μ is equal to 1, it means that the three predicted component targets are all located inside the corresponding component area mask, and the drone is located in the safe parking area surrounded by the centering pole;

[0154] If the value of μ is not equal to 1, it means that the three predicted component targets intersect with the corresponding component area mask or are outside the corresponding component area mask, and the drone is not located in the safe parking area surrounded by the centering pole.

[0155] Calculate the aspect ratios of the n predicted component target boxes respectively; that is:

[0156]

[0157] Among them, θ represents the angle formed by the diagonal line of the component target box and the lower frame line.

[0158] Determine whether the parking angle of the drone meets the angle range required for centering based on the height-to-width ratio;

[0159] If the predicted aspect ratio tanθ of the component target frame is within the pre-obtained aspect ratio range, the parking angle of the drone meets the angle range required for centering;

[0160] On the contrary, if the predicted aspect ratio tanθ of the component target frame is outside the pre-obtained aspect ratio value range, the parking angle of the drone does not meet the angle range required for centering;

[0161] Whether to perform the centering operation or the door closing operation is determined based on the landing and parking position and angle of the drone.

[0162] If the drone is parked in a safe parking area and within the required angle range for centering, the centering pole can be safely used for centering operations;

[0163] If the drone is not parked in a safe parking area or within the required angle range for centering, the centering operation of the centering pole will be terminated and a warning will be issued.

[0164] The beneficial effects of the present invention are embodied in: the present invention combines the target detection network, the component area mask, and the aspect ratio of the target frame to detect whether the drone is parked in the irregular safe parking area surrounded by the centering pole and whether the parking angle conforms to the centering operation; improves the reliability of the unmanned hangar system, and provides an additional guarantee for the safety of the drone and the hangar; inputs the label file into the target detection network for training, inputs the real-time image into the trained network, and outputs the predicted drone component target frame; inputs the label file into the target detection network for training to obtain the drone component target detection model; pre-processes the real-time image, inputs it into the trained drone component target detection model, and outputs the predicted drone component target frame; each predicted drone component target frame includes coordinates and confidence levels belonging to each category. The technical solution provided by the present invention can effectively determine whether the drone is located in the irregular safe parking area surrounded by the centering pole, thereby improving the reliability of the unmanned system.

[0165] The above is only an embodiment of the present invention, and the common knowledge such as the known specific technical solutions or characteristics in the solution is not described in detail here. It should be pointed out that for those skilled in the art, without departing from the solution of the present invention, several modifications and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A method for centering a drone, characterized in that: The method comprises: S1: Collect drone video data, select characteristic parts of the drone for annotation, and obtain label files; S2: Generate a component area mask according to the label file, assign a value to the area not covered by the component area mask, and generate a component target frame ratio domain; S3: inputting the label file and the real-time image received by the drone into the target detection network for training, and outputting the predicted drone component target frame; S4: Filter the predicted drone component target frame and calculate the average area ratio; S5: judging whether the UAV is located in a safe stoppable area surrounded by the centering pole and within an angle range of a centering operation according to the average area ratio and the component target frame ratio domain, and deciding whether to perform a centering operation on the UAV.

2. A method for centering a drone according to claim 1, characterized in that: The step of collecting drone video data, selecting characteristic parts of the drone for annotation, and obtaining a label file includes: Each frame of the image in the collected video data is selected to perform component target annotation to obtain the label file, wherein a line in the label file represents a component target frame.

3. A method for centering a drone according to claim 2, characterized in that: The step of generating a component area mask according to the label file and assigning a value to an area not covered by the component area mask comprises: Generating n component area masks corresponding to the characteristic components according to the label file; The area in the middle of the component area mask that is not covered is assigned a value of 1.

4. A method for centering a drone according to claim 3, characterized in that: The step of generating n component area masks corresponding to the characteristic components according to the label file comprises: Generate an all-zero array mask of the same size as the image; Traverse each line of data of each of the label files in turn; Determine the class_id of each row of data in the label file and selectively assign values ​​to the elements; After completing the traversal of all the label file data, the component area mask is generated.

5. A method for centering a drone according to claim 3, characterized in that: The step of assigning a value of 1 to an area not covered in the middle of the component area mask comprises: Extracting the outer contour of the coverage area from the component area mask; Sort by the extracted outer contour area; Determine whether there is a small contour within the extracted outer contour, and perform processing operations on the outer contour; The area not covered in the middle of the component area mask is assigned a value of 1.

6. A method for centering a drone according to claim 4, characterized in that: The step of generating a component target frame ratio domain comprises: Traverse each line of data of each of the label files in turn; If the class_id of the label file does not match the class number corresponding to the component in the preset aspect ratio range, the aspect ratio tanθ is not calculated; If the class_id of the label file matches the category number corresponding to the component in the preset aspect ratio value range, the aspect ratio tanθ is calculated by the height h and width w of the corresponding component target frame; wherein θ represents the angle formed by the diagonal line of the component target frame and the lower frame line; All the values ​​of the aspect ratio tanθ are saved in a list to obtain a list of aspect ratio tanθ values; After completing the traversal of all the label files, the minimum and maximum values ​​of the aspect ratio tanθ in the aspect ratio tanθ value list are obtained to form the aspect ratio value range of each component target frame.

7. A method for centering a drone according to claim 6, characterized in that: The step of screening the predicted drone component target frame and calculating the average area ratio includes: Calculate the area ratio P of the intersection of n predicted drone component target frames and the corresponding component area mask mask to the predicted drone component target frame i ; Then calculate the average area ratio μ.

8. A method for centering a drone according to claim 7, characterized in that: The step of judging whether the drone is located in a safe stoppable area surrounded by the centering pole and in an angle range of a centering operation according to the average area ratio and the component target frame ratio domain comprises: If the value of μ is equal to 1, the n predicted component target boxes are all located inside the corresponding component area mask, and the drone is located in the safe parking area surrounded by the centering pole; If the value of μ is not equal to 1, at least one of the component target frames intersects with the corresponding component area mask or is outside the corresponding component area mask, and the drone is outside the safe parking area surrounded by the centering pole; If the predicted aspect ratio tanθ of the component target frame is within the preset aspect ratio value range, the parking angle of the drone meets the angle range of the centering operation; If the predicted aspect ratio tanθ of the component target frame is outside the preset aspect ratio value range, the parking angle of the drone does not meet the angle range of the centering operation.

9. A method for centering a drone according to claim 8, characterized in that: The step of determining whether to perform the drone centering operation includes: If the drone is parked in a safe parking area and meets the angle range of the centering operation, the drone centering operation will be performed using the centering pole; If the drone is not parked in a safe parking area or within the angle range for the homing operation, the drone homing operation of the homing pole will be terminated and a warning will be issued.

10. A centering system for a drone, characterized in that: The system comprises: Collection module: used to collect video data of the drone parked on the boundary of the irregular safe parking area surrounded by the centering pole, and the drone's nose is facing the angle range that meets the centering operation; Annotation module: annotate the characteristic parts of the drone; Assignment module: assign a value to the area not covered by the mask in the middle of the component area; Prediction module: inputs the real-time image received by the drone into the trained network and outputs the predicted target box of the drone parts; Calculation module: Calculate the average area ratio of the intersection of the drone component target frame and the corresponding component area mask to the predicted component target frame; Judgment module: judge whether the drone is in the safe parking area surrounded by the centering pole and the angle range of the centering operation; Operation module: perform centering operations on the drone.

11. A centering system for a drone according to claim 10, characterized in that: The system further comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the centering method of the drone as described in any one of claims 1-9.

12. A computer storage medium for a drone, characterized in that: The computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the centering method of the drone as described in any one of claims 1-9.

Citation Information

Patent Citations

  • Unmanned aerial vehicle warehousing identification method, unmanned aerial vehicle system and vehicle

    CN118397244A

  • Positioning method and positioning device for unmanned aerial vehicle

    US20170193271A1

  • Object Detection Using Skewed Polygons Suitable For Parking Space Detection

    US20200294310A1

  • Dynamic take-off and landing device and method for unmanned aerial vehicle

    WO2023206864A1