Unmanned aerial vehicle runway obstacle recognition method and obstacle avoidance method based on improved YOLO v5

Abstract

The invention provides an unmanned aerial vehicle runway obstacle recognition method and an obstacle avoidance method based on improved YOLO v5, and belongs to the technical field of obstacle recognition. The unmanned aerial vehicle runway obstacle recognition method comprises the steps of obtaining a data image; and segmenting the data image by using a sky segmentation method to obtain a segmented data image. An improved YOLO v5 vision model is configured, a target recognition sub-model in a backbone network Backbone included in the improved YOLO v5 vision model is Center Net, and an MSN component is deleted from a detection head Head. And inputting the segmented data image into the improved YOLO v5 visual algorithm model, and outputting an obstacle recognition result. The invention provides an unmanned aerial vehicle runway obstacle identification method based on improved YOLOv5 for an actual application scene in which the types and the number of obstacles on a runway are relatively small in unmanned aerial vehicle runway obstacle identification, so as to solve the problems that the YOLOv5 is complex in computational power, low in efficiency, not suitable for an unmanned aerial vehicle runway and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of obstacle recognition, and in particular to an unmanned aerial vehicle runway obstacle recognition method and an obstacle avoidance method based on an improved YOLO v5. Background Art

[0002] With the rapid development of visual sensors, their application scenarios are increasing, and obstacle recognition is one of the core applications in the field of machine vision. At the same time, with the development of AI technology, more and more intelligent algorithms are being built and applied in machine vision systems as core algorithms for obstacle recognition. YOLO v5 was built in this context. YOLOv5 is an improvement on its predecessor YOLO v4, which has the advantages of fast speed, high accuracy, and light weight. However, it also faces many problems such as poor results for small and dense targets, and too large training data sets. Therefore, it is necessary to improve the YOLO v5 algorithm and eliminate the problems it faces as much as possible while retaining its advantages. Summary of the invention

[0003] The purpose of the present invention is to provide a drone runway obstacle recognition method and an obstacle avoidance method based on an improved YOLO v5. Aiming at the actual application scenario in which the types and numbers of obstacles on the runway are small in drone runway obstacle recognition, based on the fact that the YOLOv5 algorithm has a relatively small data set and is difficult to recognize small targets on the end side, especially the poor recognition effect of dense targets, a drone runway obstacle recognition method based on an improved YOLOv5 is proposed to solve the problems of complex computing power, low efficiency and unsuitability for drone runways faced by YOLOv5.

[0004] In order to achieve the above object, the present invention provides the following technical solutions: In a first aspect, the present invention provides a method for identifying obstacles on a runway of a UAV based on an improved YOLO v5, characterized in that the method comprises the following steps: S10. Acquiring data image; S11. Applying the sky segmentation method to segment the data image to obtain a segmented data image; S12. Configure an improved YOLO v5 visual model, where the target recognition sub-model in the backbone network Backbone included in the improved YOLO v5 visual algorithm model is CenterNet, and the MSN component is deleted from the detection head Head; S13. Input the segmented data image into the improved YOLO v5 visual algorithm model and output the obstacle recognition result.

[0005] As a possible implementation manner, the backbone network Backbone sequentially sets three layers of components including CBL and CSP-x after CenterNeT, where x represents the number of repetitions.

[0006] As a possible implementation method, the Neck network in the improved YOLO v5 visual model includes a backbone network Backbone, Neck and Prediction; wherein the target recognition submodel in the backbone network Backbone is CenterNet, and the MSN component is deleted in the detection head Head.

[0007] In a second aspect, the present invention further provides an obstacle avoidance method, which uses the drone runway obstacle recognition method based on the improved YOLO v5 described in the first aspect to identify obstacles; the obstacle avoidance method comprises the following steps: S20. Determine the threat level of obstacles; S21. Determine obstacle avoidance paths based on threat levels.

[0008] As a possible implementation manner, the threat level specifically includes obstacles that do not affect the flight of the aircraft, obstacles that may affect the flight of the aircraft, and obstacles that will definitely affect the flight of the aircraft.

[0009] As a possible implementation method, the obstacle avoidance path is determined according to the threat level, including: When the threat level is judged to be that there is an obstacle but it does not affect the flight of the aircraft, the path planning module does not re-plan the path, that is, it provides the original path to the aircraft and continues to execute the flight mission; When the threat level is determined to be that there is an obstacle and it may affect the flight of the aircraft, the obstacle recognition module sends the recognition information to the path planning module, and the path planning module replans the path and provides the new route to the aircraft to continue the flight mission. When the threat level is judged to be that there is an obstacle and it will definitely affect the flight of the aircraft, and when the path planning does not provide a new path, an emergency stop message will be sent to the control system of the drone to brake the aircraft.

[0010] Compared with the prior art, the present invention has the following effects: 1. Aiming at the special application scenarios of airports, the present invention adopts the optimized YOLOv5 visual algorithm to realize the runway obstacle recognition and safety warning of unmanned aerial vehicles, which effectively solves the problem of difficulty in recognizing small targets on the end side in relatively small data sets, especially the poor recognition effect of dense targets.

[0011] 2. By modifying the target recognition algorithm of the backbone network Backbone of YOLO v5, that is, using CenterNet instead of FCOS, this structure can omit the MSN component in the native YOLOv5 Head, effectively reducing the complexity of the algorithm and meeting the requirements of limited space, computing power and real-time performance; 3. The present invention applies sky segmentation and Reduced YOLO v5 algorithm to fuse the runway obstacle recognition algorithm. After fusing the sky segmentation, the visual image can be quickly processed and the influence of the noise brought by the sky part on the subsequent obstacle recognition results can be eliminated.

[0012] 4. The present invention constructs a three-level safety obstacle avoidance rule to support the intelligent path planning of the unmanned aerial vehicle platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings described herein are used to provide further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0014] Figure 1 The overall flow chart of unmanned aircraft runway obstacle recognition based on the improved YOLO v5 algorithm provided in the embodiment of the present invention; Figure 2 It is an improved YOLO v5 algorithm framework; Figure 3 There are three different obstacle avoidance modes. DETAILED DESCRIPTION

[0015] In order to clearly describe the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, words such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and effects. For example, the first threshold and the second threshold are only used to distinguish different thresholds, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0016] It should be noted that, in the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0017] In the present invention, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, the combination of a and b, the combination of a and c, the combination of b and c, or the combination of a, b and c, where a, b, c can be single or multiple.

[0018] See also Figure 1 and Figure 2 In a first aspect, an embodiment of the present invention provides a method for identifying obstacles on a runway of a UAV based on an improved YOLO v5, comprising the following steps: S10. Acquiring data image; S11. Apply the sky segmentation method to segment the data image to obtain a segmented data image. Eliminate the sky information that occupies a large area of ​​information in the image to adapt to the improved YOLO v5 visual algorithm model and accurately realize the recognition of runway obstacles. The present invention uses sky segmentation and Reduced YOLO v5 algorithm to fuse the runway obstacle recognition algorithm; after fusing the sky segmentation, the visual image can be quickly processed, and the influence of the noise brought by the sky part on the subsequent obstacle recognition results can be eliminated.

[0019] S12. Configure an improved YOLO v5 visual model, in which the target recognition submodel in the backbone network Backbone included in the improved YOLO v5 visual algorithm model is CenterNet, and the MSN component is deleted in the detection head Head. The efficiency of the YOLO v5 algorithm is improved by replacing FCOS with CenterNet. By replacing FCOS with CenterNet as the framework for obstacle recognition, the recognition of obstacles on the runway line is quickly realized by fusing the length and width of a center point of CenterNet. The modification of the algorithm is mainly because the obstacles on the runway are relatively simple compared to common obstacles, and the number is relatively small, so a complex target monitoring framework is not required, and the application scenario in the embodiment of the present invention is more suitable for using CenterNet as a target monitoring method. After using CenterNet, the framework of the unmanned aircraft runway obstacle recognition method can cancel the NMS framework in the Head part, which can further reduce the entire YOLO v5 framework, effectively reduce the complexity of the algorithm, and meet the requirements of limited space, computing power and real-time performance.

[0020] In addition, the present invention adopts the optimized YOLOv5 visual algorithm to realize the obstacle recognition and safety warning of the runway of unmanned aerial vehicles for the special application scenario of the airport, and effectively solves the problem that the recognition of small targets on the end side is difficult in a relatively small data set, especially the poor recognition of dense targets. Specifically, CenterNet is mainly based on the key point detection method. Compared with the traditional FCOS, CenterNet converts the target detection problem into a regression task of predicting the center point and size of the object. It uses a heatmap to represent the center position of the object, and combines the width and height to regress the bounding box. Through the center point positioning, the model can effectively avoid the complex design (such as anchor boxes and multi-scale feature pyramids) in traditional target detection. Since it detects objects directly from the heat map, it can better capture the position of the object, especially in the detection of small objects. In addition, it is more suitable for the detection of irregular objects and non-rectangular areas, because it detects by regressing the center and size of the object, rather than by regressing the four bounding box coordinates. It is more suitable for the application scenario of the runway; furthermore, since it does not involve complex anchor configuration, training is relatively easy, and the model is less sensitive to hyperparameters.

[0021] S13. Input the segmented data image into the improved YOLO v5 visual algorithm model and output the obstacle recognition result.

[0022] As a possible implementation manner, the backbone network Backbone sequentially sets three layers of components including CBL and CSP-x after CenterNe, where x represents the number of repetitions, and as an example, x=3.

[0023] As a possible implementation method, the Neck network in the improved YOLO v5 visual model includes a backbone network Backbone, Neck and Prediction; wherein the target recognition submodel in the backbone network Backbone is CenterNet, and the MSN component is deleted in the detection head Head.

[0024] In a second aspect, an embodiment of the present invention further provides an obstacle avoidance method, which uses the drone runway obstacle recognition method based on the improved YOLO v5 described in the first aspect to identify obstacles; the obstacle avoidance method comprises the following steps: S20. Determine the threat level of the obstacle; the threat level specifically includes obstacles that do not affect the flight of the aircraft, obstacles that may affect the flight of the aircraft, and obstacles that will definitely affect the flight of the aircraft.

[0025] S21. Determine the obstacle avoidance path according to the threat level. When the threat level is judged to be that there is an obstacle but it does not affect the flight of the aircraft, the path planning module does not re-plan the path, that is, the original path is provided to the aircraft, and the flight mission continues to be executed. When the threat level is judged to be that there is an obstacle and it may affect the flight of the aircraft, the obstacle identification module sends the identification judgment information to the path planning module, and the path planning module re-plans the path and provides the new route to the aircraft, and the flight mission continues to be executed. When the threat level is judged to be that there is an obstacle and it will definitely affect the flight of the aircraft, and when the path planning does not provide a new path, an emergency stop message will be sent to the control system of the drone to achieve aircraft braking.

[0026] For specific implementation, see Figure 3 In (a), there is an obstacle but it does not affect the flight of the aircraft. The obstacle identification module is configured to identify the obstacle information and determine the proportion of the obstacle to the runway, for example, only one tenth. At this time, it does not affect the normal operation of the aircraft. That is, because the speed of the aircraft is very low, even personnel will not affect the normal operation of the aircraft.

[0027] See also Figure 3 In (b), there are obstacles, but the obstacles may affect flight safety. At this time, the obstacle recognition module will send the recognition judgment information to the path planning module. The path planning module will re-plan the path and provide the new route to the aircraft to continue the flight mission.

[0028] See also Figure 3In (c), there is an obstacle, and the location of the obstacle will affect the safe operation of the aircraft. When the path planning does not provide a new path, an emergency stop message will be sent to the control system of the drone to achieve aircraft braking.

[0029] The present invention constructs a three-level safety obstacle avoidance rule to support the intelligent path planning of the unmanned aerial vehicle platform.

[0030] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the drawings, etc. In the specification, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit can implement several functions listed in the specification. Certain measures are recorded in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0031] Although the present invention has been described in conjunction with specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present invention. Accordingly, this specification and the accompanying drawings are merely exemplary illustrations of the present invention and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present invention. Obviously, those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, the present invention is intended to include such modifications and variations if they fall within the scope of the present invention and its equivalents.

Claims

1. A method for identifying obstacles on a drone runway based on an improved YOLO v5, characterized in that: The steps include: S10. Acquiring data image; S11. Applying the sky segmentation method to segment the data image to obtain a segmented data image; S12. Configure an improved YOLO v5 visual model, where the target recognition sub-model in the backbone network Backbone included in the improved YOLO v5 visual algorithm model is CenterNet, and the MSN component is deleted from the detection head Head; S13. Input the segmented data image into the improved YOLO v5 visual algorithm model and output the obstacle recognition result.

2. The method for identifying obstacles on a drone runway based on improved YOLO v5 according to claim 1, characterized in that: The backbone network Backbone sequentially sets three layers of components including CBL and CSP-x after CenterNeT, wherein x represents the number of repetitions.

3. The method for identifying obstacles on a drone runway based on improved YOLO v5 according to claim 1, characterized in that: The Neck network in the improved YOLO v5 visual model includes a backbone network Backbone, Neck and Prediction; wherein the target recognition submodel in the backbone network Backbone is CenterNet, and the MSN component is deleted in the detection head Head.

4. An obstacle avoidance method, characterized in that: Apply the drone runway obstacle recognition method based on improved YOLO v5 as described in any one of claims 1 to 3 to identify obstacles; the obstacle avoidance method comprises the following steps: S20. Determine the threat level of obstacles; S21. Determine obstacle avoidance paths based on threat levels.

5. The obstacle avoidance method according to claim 4, characterized in that: The threat levels specifically include obstacles that do not affect the flight of the aircraft, obstacles that may affect the flight of the aircraft, and obstacles that will definitely affect the flight of the aircraft.

6. The obstacle avoidance method according to claim 5, characterized in that: Determine the obstacle avoidance path based on the threat level, including: When the threat level is judged to be that there is an obstacle but it does not affect the flight of the aircraft, the path planning module does not re-plan the path, that is, it provides the original path to the aircraft and continues to execute the flight mission; When the threat level is determined to be that there is an obstacle and it may affect the flight of the aircraft, the obstacle recognition module sends the recognition information to the path planning module, and the path planning module replans the path and provides the new route to the aircraft to continue the flight mission. When the threat level is judged to be that there is an obstacle and it will definitely affect the flight of the aircraft, and when the path planning does not provide a new path, an emergency stop message will be sent to the control system of the drone to brake the aircraft.

Images