Monocular obstacle avoidance method for fixed-wing aircraft

Abstract

The invention provides a monocular obstacle avoidance method for a fixed-wing aircraft. The monocular obstacle avoidance method comprises the following steps: providing an image acquisition module to obtain a monocular image; providing an image processing module, performing data processing on the monocular image acquired by the image acquisition module within a preset time interval, and generating first estimated collision time and a first sight angle; providing an obstacle avoidance module to calculate and generate a navigation control quantity; and providing a bottom layer control module to control the state change of the fixed-wing aircraft. According to the monocular obstacle avoidance method for the fixed-wing aircraft, the flying obstacle is recognized based on image processing and analysis, the processing method is low in cost, and the safety performance is good.

Description

Technical Field

[0001] The present invention relates to the technical field of aircraft obstacle avoidance, and in particular to a monocular obstacle avoidance method for fixed-wing aircraft. Background Art

[0002] Aircraft operations are a vital component of the low-altitude economy. Compared to traditional ground vehicles, aircraft operations require a higher level of safety. A collision can impact flight stability at best, or even lead to a crash. Furthermore, due to the high altitudes at which they fly, a fall can also cause secondary injuries. Therefore, automatic obstacle avoidance is a crucial component of flight safety.

[0003] Existing technologies include ultrasonic ranging, infrared obstacle avoidance, laser ranging, lidar, millimeter-wave radar, and visual obstacle avoidance. Ultrasonic ranging is suitable for short-range obstacle avoidance but is less effective against approaching obstacles at high speeds. Infrared obstacle avoidance and laser ranging have high requirements for reflective objects, and their measurement accuracy is easily affected by the environment and the material of the obstacle itself. Lidar and millimeter-wave radar are relatively expensive, making them uneconomical for fixed-wing aircraft. These sensors offer good perception at close range, but their reliability decreases significantly at longer distances.

[0004] For example, prior art Chinese patent CN115602003A describes a method, device, and drone for dynamic obstacle avoidance using a monocular camera. The method involves measuring the distance between dynamic obstacles and the drone using a millimeter-wave radar module. Millimeter-wave radar, like binocular cameras and lidar, can obtain depth information about obstacles, but its perception at long distances is poor.

[0005] A few obstacle avoidance solutions still use monocular depth estimation methods based on deep learning. These methods have high depth estimation reliability for small obstacles at close range and large, stationary obstacles at long range (such as hills and buildings). However, the depth estimation accuracy for distant moving aircraft is very poor, and they can only achieve recognition, so their depth information cannot be used. Summary of the Invention

[0006] The present invention provides a monocular obstacle avoidance method for fixed-wing aircraft for solving the above-mentioned problems of high cost and insufficient precision.

[0007] The present invention provides a fixed-wing aircraft monocular obstacle avoidance method, comprising the following steps: S1: providing an image acquisition module, and acquiring a monocular image through the image acquisition module, wherein the monocular image contains a flight obstacle; S2: providing an image processing module, which processes the monocular image acquired by the image acquisition module within a preset time interval and generates a first estimated collision time and a first sight angle corresponding to the flight obstacle; S3: Providing an obstacle avoidance module, wherein the obstacle avoidance module collects the first estimated collision time and the first sight angle generated by the image processing module in real time, and calculates and generates a navigation control variable in combination with flight parameters of the fixed-wing aircraft; S4: Providing a bottom-level control module, wherein the bottom-level control module controls the state change of the fixed-wing aircraft based on the navigation control amount.

[0008] Specifically, in S1, the image acquisition module includes at least three monocular cameras, which are respectively installed in front, on the left and on the right side of the fixed-wing aircraft. The horizontal shooting angle of at least three monocular cameras is not less than 225°, and the vertical shooting angle is not less than 30°.

[0009] Specifically, the S2 includes: S21: Acquire a first image with an obstacle at time t1; S22: Acquire a second image having the flight obstacle at time t2; S23: Processing the first image and the second image to generate the first estimated collision time and the first sight angle based on a change in pixel area of the flight obstacle in the image; Among them, in S23, data processing is performed on the first image and the second image, including using the YOLO algorithm and the Kalman filter algorithm to identify the image features of the flight obstacle and obtain the first estimated collision time.

[0010] Specifically, the S2 further includes: S24: The state of the fixed-wing aircraft continues to change, and the image processing module smoothes the first estimated collision time and the first sight angle that have changed beyond a first threshold within a predetermined time, and transmits them to the obstacle avoidance module.

[0011] Specifically, the S3 includes: S31: Based on the first estimated collision time, obtain a first collision time corresponding to at least one of the flight obstacles ; S32: Based on the first sight angle, obtain a first azimuth angle corresponding to at least one of the flight obstacles and the first pitch angle ; S33: Acquire the flight parameters, including target point position information, fixed-wing aircraft position information, and fixed-wing aircraft speed information; S34: Calculate and generate the navigation control amount ; in, It refers to the existence of said flight obstacles.

[0012] Specifically, the step S34 further includes: determining the obstacle avoidance priority, controlling the obstacle avoidance trend, wherein the first collision time is set The second threshold, when the first collision time Below the second threshold, maintain the navigation control amount obstacle avoidance trend, reducing the influence weight of the target point position information.

[0013] Specifically, the S33 further includes: S331: Obtain a 2.5D terrain map, construct a simulated radar, calculate terrain obstacles in front of the fixed-wing aircraft, and obtain a second estimated collision time and a second line of sight angle; S332: Based on the second estimated collision time, obtain a second collision time corresponding to at least one of the terrain obstacles ; S333: Based on the second sight angle, obtain a second azimuth angle corresponding to at least one of the terrain obstacles and the second pitch angle .

[0014] Specifically, in the S34, the navigation control amount Including the angular rate of the fixed-wing aircraft's three-dimensional track in the horizontal direction and the angular rate in the vertical direction, the calculation formula is as follows:

[0015] Wherein, N is the total number of the flight obstacles, and M is the total number of the terrain obstacles; is its own flying speed, is the flight obstacle avoidance control quantity, is the terrain obstacle avoidance control quantity, is the target point control amount, its own position information , target point location information .

[0016] Specifically, in S331, the 2.5D terrain map is pre-built and / or acquired in real time through a SLAM model.

[0017] Specifically, performing layered convex hull processing on the acquired 2.5D terrain map includes the following steps: S3311: Obtain the original 2.5D map, perform layering on the 2.5D map, and cluster the singly connected obstacles in it; S3312: Perform convex hull processing on each layer of simply connected regions; S3313: Output the 2.5D terrain map.

[0018] Compared to existing technologies, the monocular obstacle avoidance method for fixed-wing aircraft provided by the present invention uses image processing and analysis to identify flying obstacles. It then obtains the first estimated collision time and the first sight angle of the flying obstacle, and performs obstacle avoidance control in conjunction with the fixed-wing aircraft's flight parameters. This method can acquire rich environmental information and identify complex objects and dynamic flying obstacles. It also offers low-cost processing and eliminates the need for depth cameras, infrared cameras, radar, and other equipment.

[0019] In addition, this method does not need to obtain relevant depth information and has high computational efficiency. It can perform real-time calculations using only the first estimated collision time and the first sight angle, and can effectively respond when a flying obstacle approaches at high speed.

[0020] After 500,000 Monte Carlo simulations, it was found that the fixed-wing aircraft used the method provided by the present invention to avoid obstacles, with a very low probability of collision and good safety performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which: Figure 1 This is a flowchart of the fixed-wing aircraft monocular obstacle avoidance method provided by the present invention; Figure 2 This is a structural block diagram of the fixed-wing aircraft monocular obstacle avoidance method provided by the present invention; Figure 3 The fixed-wing aircraft monocular obstacle avoidance method in one embodiment calculates the navigation control amount Flowchart of the process; Figure 4 yes Figure 1 The flowchart of S2 shown; Figure 5 yes Figure 1 The flowchart of S3 is shown; Figure 6 This is a schematic diagram of a fixed-wing aircraft flight in one embodiment; Figure 7 yes Figure 5 The flowchart of S33 is shown; Figure 8 This is a schematic diagram of a 2.5D original map in one embodiment; Figure 9 yes Figure 7 The flowchart of S331 is shown; Figure 10 yes Figure 8 The 2.5D original map shown is a 2D grid map after single-connected clustering processing; Figure 11 yes Figure 10 The 2D grid map shown is a 2D grid map after convex hull processing; Figure 12 yes Figure 11 The 2D raster map shown is a 2.5D terrain map generated by adding height. DETAILED DESCRIPTION

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0023] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0024] In addition, the terms "first," "second," and so on, used in this disclosure are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referenced. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this disclosure, "plurality" means at least two, such as two or three, unless otherwise specifically defined.

[0025] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in this field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0026] See also Figure 1 and Figure 2 , Figure 1 This is a flowchart of the fixed-wing aircraft monocular obstacle avoidance method provided by the present invention. Figure 2 This is a block diagram of the fixed-wing aircraft monocular obstacle avoidance method provided by the present invention. The fixed-wing aircraft monocular obstacle avoidance method comprises the following steps: S1: providing an image acquisition module, and acquiring a monocular image through the image acquisition module, wherein the monocular image contains a flight obstacle; S2: providing an image processing module, which processes the monocular image acquired by the image acquisition module within a preset time interval and generates a first estimated collision time and a first sight angle corresponding to the flight obstacle; S3: Providing an obstacle avoidance module, wherein the obstacle avoidance module summarizes the first estimated collision time parameter and the first sight angle generated by the image processing module in real time, and calculates and generates a navigation control variable in combination with flight parameters of the fixed-wing aircraft; S4: Providing a bottom-level control module, wherein the bottom-level control module controls the state change of the fixed-wing aircraft based on the navigation control amount.

[0027] It should be noted that in S1, the flight obstacles include intruding aircraft, balloons, or flying birds, etc., and are not limited thereto. Using the fixed-wing aircraft monocular obstacle avoidance method, the image acquisition module and the image processing module are capable of identifying and processing these obstacles. Of course, the image acquisition module acquires the monocular image in real time and filters images that do not contain the flight obstacles to reduce image processing data.

[0028] To avoid collisions with lateral obstacles, in S1, the image acquisition module includes at least three monocular cameras, mounted in front, on the left, and on the right sides of the fixed-wing aircraft. The at least three monocular cameras have a horizontal shooting angle of no less than 225° and a vertical shooting angle of no less than 30°. A monocular camera is a type of imaging device that utilizes only a single optical sensor and lens, primarily used to capture images or videos from a single perspective. There are no specific requirements for its model or size. By placing these three monocular cameras in the front, left, and right directions, the situation in front of and to the sides of the fixed-wing aircraft can be effectively identified. Typical monocular cameras can monitor a 30° vertical angle, eliminating the need for installations on the top and bottom of the fixed-wing aircraft. This ensures monitoring requirements while minimizing costs. Horizontal monitoring is adequately covered by the three monocular cameras. The horizontal shooting angle of no less than 225° ensures that the three cameras can cross-monitor the situation in front of and to the left and right sides of the fixed-wing aircraft, avoiding blind spots. Obstacle avoidance is not considered for the area behind the fixed-wing aircraft; obstacle avoidance is based on the aircraft's own capabilities. If additional monitoring is required, a rear monocular camera can be added accordingly, which will not be described in detail here.

[0029] Furthermore, it should be noted that the monocular camera has a horizontal shooting angle of no less than 225° and a vertical shooting angle of no less than 30°. These angles are symmetrical when a fixed-wing aircraft is in horizontal flight. In other words, the pitch angles are symmetrical by at least 15° relative to the horizontal plane of the fixed-wing aircraft, and symmetrical by at least 112.5° relative to the vertical plane in which the fixed-wing aircraft's nose is pointed.

[0030] Please continue reading Figure 3 and Figure 4 The image processing module processes the monocular image acquired by the image acquisition module within a preset time interval and generates a first estimated collision time and a first sight angle corresponding to the flight obstacle. Specifically, S2 includes the following steps: S21: Acquire a first image with an obstacle at time t1; S22: Acquire a second image having the flight obstacle at time t2; S23: Processing the first image and the second image to generate the first estimated collision time and the first sight angle based on a change in pixel area of the flight obstacle in the image; S24: The state of the fixed-wing aircraft continues to change, and the image processing module smoothes the first estimated collision time and the first sight angle that have changed beyond a first threshold within a predetermined time, and transmits them to the obstacle avoidance module.

[0031] In S23, data processing is performed on the first image and the second image, including using the YOLO algorithm and the Kalman filter algorithm to identify the image features of the flight obstacle and estimate the first estimated collision time.

[0032] In this embodiment, the preset time interval is equal to the time difference between t2 and t1. The preset time interval is pre-set and its specific value can be determined based on the safety requirements of the fixed-wing aircraft and the acquisition frequency of the monocular camera, and is not specifically limited herein. Of course, the preset time interval should be a specific and clear duration. After the image processing module identifies changes in the image features of the flight obstacle in the first and second images, it can calculate the preset time interval to effectively generate the first estimated collision time and the first line of sight angle.

[0033] Among them, the image processing module mainly uses the YOLO algorithm and filter to identify the image features of the flight obstacle.

[0034] The YOLO (You Only Look Once) algorithm is an image object recognition algorithm proposed by Joseph Redmon et al. in 2015. Unlike traditional two-stage object detection methods, the YOLO algorithm simplifies the entire detection process into a single regression problem. It can simultaneously predict the location and category of objects in an image in a single pass, achieving very fast detection response speed and high overall accuracy.

[0035] There are various types of filters, such as the Kalman filter used in this embodiment. A Kalman filter is a highly efficient recursive filter that estimates the real-time state based on the first and second images acquired in real time, combined with the system's dynamic model, and exhibits excellent real-time performance. Simply put, the Kalman filter acts like an intelligent data processor, accurately extracting the difference between the first and second images from noisy data, specifically the changes in the flight obstacle relative to the fixed-wing aircraft.

[0036] The fixed-wing aircraft monocular obstacle avoidance method is based on the use of the YOLO algorithm and the filter. In a high-speed changing low-altitude environment, the image features of the flight obstacles can be effectively identified and analyzed in real time, which is conducive to the safe obstacle avoidance of the fixed-wing aircraft.

[0037] Of course, due to the preset time interval, the image processing module processes a large number of the first and second images within a certain period of time and generates multiple sets of the first estimated collision time and the first sight angle. If the position and attitude between the flight obstacle and the fixed-wing aircraft change rapidly, the first estimated collision time and the first sight angle will fluctuate widely.

[0038] Therefore, when the state of the fixed-wing aircraft changes too quickly, the image processing module smoothes the first estimated collision time and the first sight angle that change beyond a first threshold within a predetermined time, and transmits them to the obstacle avoidance module.

[0039] Among them, the predetermined time is set in advance and is relatively greater than the preset time interval. The specific value is set according to the self-application situation and safety performance requirements of the fixed-wing aircraft and is not specifically limited. That is, within the predetermined time range, the first estimated collision time and the first sight angle generated by multiple preset time interval settings are monitored. When large-scale fluctuations occur, the first estimated collision time and the first sight angle are smoothed to ensure that the input and output of the obstacle avoidance module can be more stable, and the underlying control module will not have multiple frequencies and complex controls, thereby enhancing the safety of fixed-wing aircraft control. Of course, the smoothing method is an existing technology, such as based on the mean algorithm, median algorithm, normal distribution algorithm or variance algorithm, etc., which will not be explained in detail here.

[0040] Please continue to refer to Figure 5 and Figure 6 , for the calculation of the navigation control amount, the S3 includes the following steps: S31: Based on the first estimated collision time, obtaining a first collision time of an intruding obstacle corresponding to at least one of the flight obstacles ; S32: Based on the first sight angle, obtain a first azimuth angle corresponding to at least one of the flight obstacles and the first pitch angle ; S33: Acquire the flight parameters, including target point position information, fixed-wing aircraft position information, and fixed-wing aircraft speed information; S34: Calculate and generate the navigation control amount .

[0041] It should be noted that, due to the rapid development of the low-altitude economy, in the future low-altitude environment, multiple flying obstacles may appear simultaneously around the flight path of a fixed-wing aircraft. Therefore, the fixed-wing aircraft monocular obstacle avoidance method needs to consider the obstacle avoidance needs in complex situations in advance. Based on at least one of the flying obstacles, at least one set of the first estimated collision time and the first sight angle is generated, and the first collision time of the intruding obstacle corresponding to at least one set of the flying obstacles is obtained. , the first azimuth angle and the first pitch angle .in, Represents the existence i The flight obstacles are determined according to the number of intruding aircraft or other dynamic obstacles. If there is only one intruding aircraft, then =1; if there are two invading aircraft, then =1,2.

[0042] In this embodiment, the navigation control amount is calculated and generated When controlling the flight parameters, it is also necessary to ensure control accuracy and efficiency. The target point location information is determined by the fixed-wing aircraft before takeoff, typically by loading the GPS location information corresponding to the target location. The inclusion of this target point location information ensures that the fixed-wing aircraft considers the shortest possible flight path toward the target point while avoiding obstacles. This is based on the principle that a straight line is the shortest distance between two points, and the weights of obstacle avoidance and flight to the target point are combined for control, ensuring that the shortest possible distance to the target point is achieved while avoiding obstacles.

[0043] The flight parameters also include the fixed-wing aircraft's position, attitude, and speed. These parameters can be accurately acquired through sensors like GPS and IMU. These parameters can be retrieved during normal flight without requiring additional data. These parameters can then be temporarily retrieved when the obstacle avoidance module is calculating and selecting a strategy, serving as a basis for decision-making.

[0044] Furthermore, in said S34, it also includes: judging the obstacle avoidance priority, controlling the obstacle avoidance trend, wherein the first collision time is set The second threshold, when the first collision time Below the second threshold, maintain the navigation control amount The obstacle avoidance trend in the target point is reduced, and the influence weight of the target point position information is reduced.

[0045] Here is an example. When the flight obstacle is at a relatively far position, that is, the first collision time When the collision time is large and the fixed-wing aircraft has enough decision time to avoid the obstacle, the obstacle avoidance weight can be low and the aircraft still flies towards the target point. When the value is smaller, the obstacle avoidance weight is higher, and obstacle avoidance is given priority. The latest situation of the flight obstacle is further monitored. After the obstacle avoidance is adjusted and confirmed to be safe, the flight is adjusted to the target point. When the flight is very close to the target point, obstacle avoidance is given the highest weight, and the target point position information is not considered. Adjustments are made only after safety is ensured.

[0046] The first collision time The judgment is mainly based on the second threshold. Of course, the setting of the second threshold is based on the fixed-wing aircraft's own conditions and safety performance needs. When the threshold is lower than the second threshold, it can be considered as a very critical situation, so there is no need to collect multiple times, and the navigation control amount is directly maintained. The vehicle will continue to avoid obstacles for a certain period of time. It will no longer be pulled by the target point and will then determine whether it is safe or far away from the target point in the new state.

[0047] Therefore, the navigation control amount is calculated and generated by combining the flight parameters and the corresponding weight relationship. It can achieve the most reasonable flight route and reduce energy consumption while ensuring safety.

[0048] Please continue reading Figure 7 , the S33 further comprises the following steps: S331: Obtain a 2.5D terrain map, construct a simulated radar, calculate terrain obstacles in front of the fixed-wing aircraft, and obtain a second estimated collision time and a second line of sight angle; S332: Based on the second estimated collision time, obtain a second collision time corresponding to at least one of the terrain obstacles ; S333: Based on the second sight angle, obtain a second azimuth angle corresponding to at least one of the terrain obstacles and the second pitch angle .

[0049] It's important to note that 2.5D terrain maps are a type of visualization that's somewhere between 3D and 2D terrain maps. Through techniques like tilted viewing angles, height stretching, or shadow rendering, they simulate a 3D effect while retaining the accuracy of a 2D map. This is achieved by adding height information to the 2D terrain map data to create a data map.

[0050] The 2.5D terrain map allows fixed-wing aircraft to avoid both flight obstacles and terrain obstacles simultaneously. Furthermore, because the 2.5D terrain map includes additional altitude information compared to a 2D terrain map, it facilitates identification of terrain obstacles and occupies significantly less storage space than a 3D terrain map, enabling more efficient and rapid access.

[0051] Construct a simulated radar to calculate the terrain obstacles in front of the fixed-wing aircraft, and obtain the second estimated collision time and the second line of sight angle. The terrain obstacles do not need to be detected, identified, and calculated at the physical level, but can be achieved through internal algorithms. Specifically, based on the 2.5D terrain map and the flight parameters of the fixed-wing aircraft, the flight is simulated internally, and the fixed-wing aircraft is simulated to transmit radar signals to the terrain obstacles in the 2.5D terrain map, so as to simulate and calculate the second estimated collision time and the second line of sight angle. This method can accurately generate the second estimated collision time and the second line of sight angle internally, without being affected by the detection sensors, flight environment, and data processing modules at the physical level, and can be directly simulated and calculated internally.

[0052] In addition, the second estimated collision time and the second sight angle are used to calculate and generate the second collision time. , the second azimuth and the second pitch angle , the first collision time corresponding to the flight obstacle , the first azimuth angle and the first pitch angle The parameter types remain consistent, allowing the obstacle avoidance module to directly identify and process obstacles without requiring additional processing steps. When the 2.5D terrain map is known, obstacles with unknown distances are integrated into the obstacle avoidance framework, eliminating complex obstacle avoidance priority determination.

[0053] In order to comprehensively consider the situation of the flight obstacle and the terrain obstacle, the obstacle avoidance control is performed. In the S34, the navigation control amount This includes the horizontal and vertical angular rates of the fixed-wing aircraft's three-dimensional trajectory. When the aircraft enters fixed-wing flight, monitoring begins. If an approaching intruder is detected, the TTC obstacle avoidance system is activated.

[0054] The calculation steps and formulas are as follows:

[0055] The navigation control amount Based on its own parameters, the first collision time , the first azimuth angle , the first pitch angle The second collision time , the second azimuth and the second pitch angle Calculation is performed. Where i refers to at least one of the possible flight obstacles or terrain obstacles, N is the total number of the flight obstacles, and M is the total number of the terrain obstacles.

[0056] The second threshold is a parameter set in advance for evaluating the first collision time Is it lower than the second threshold? Only when the first collision time Below the second threshold Only when the flight obstacle avoidance control is involved The first threshold A parameter set in advance for evaluating the second collision time Is it lower than the first threshold? Only when the second collision time Below the first threshold When the terrain obstacle avoidance control amount is involved The definition of specific parameters is based on the specific environment needs and safety requirements and is not limited here.

[0057] The navigation control amount Including flight obstacle avoidance control , terrain obstacle avoidance control amount and target point control amount , add the three components to get the navigation control amount , so that all relevant flight obstacles and terrain obstacles can be included in the obstacle avoidance decision and comprehensively considered to ensure the accuracy of the obstacle avoidance decision and the safety of the fixed-wing aircraft flight.

[0058] Among them, the flight obstacle avoidance control amount Mainly based on the first collision time of at least one of the flight obstacles , the first azimuth angle , the first pitch angle and its own flight speed Calculate the obstacle avoidance control amount and sum it up. Mainly based on the second collision time of at least one of the terrain obstacles , the second azimuth and the second pitch angle and its own flight speed Calculate the obstacle avoidance control amount and sum it up. Mainly based on its own location information , target point location information and its own flight speed Calculate the flight obstacle avoidance control amount , the terrain obstacle avoidance control amount And the target point control amount Each includes the horizontal and vertical angular rates of its own three-dimensional trajectory. The resulting horizontal and vertical angular rates of the three-dimensional trajectory are delivered to the underlying controller, which calculates the corresponding attitude and thrust and distributes them to the various actuators for execution.

[0059] In the S331, the 2.5D terrain map is pre-built and / or acquired in real time through the SLAM model. It should be noted that the construction of the 2.5D terrain map can be pre-built and saved in the fixed-wing aircraft in advance, and directly called during the flight; of course, it can also be acquired in real time during the flight phase through the SLAM model. In this embodiment, a solution where both exist at the same time is preferred. That is to say, the existing 2.5D terrain map is pre-stored inside the fixed-wing aircraft, and during the flight, it is acquired in real time through the SLAM model as needed, ensuring the accuracy and completeness of the information while reducing the call to the SLAM model as much as possible, thereby reducing the number of work steps, shortening the response time, reducing energy consumption and ensuring flight safety.

[0060] The SLAM model, short for Simultaneous Localization and Mapping, stands for simultaneous localization and mapping. This means that a fixed-wing aircraft can locate itself based on its position and environment while on the move, while simultaneously constructing a map of the surroundings. The primary purpose of SLAM is to map and estimate position in an unknown environment, thereby guiding subsequent actions such as navigation, obstacle avoidance, and grasping. The specific model construction and algorithms are not described here.

[0061] Please continue reading Figures 8 to 11 During flight, fixed-wing aircraft may encounter non-convex terrain (such as cols, dead ends, etc.) that is impossible to traverse directly. This non-convex terrain can create local optimal traps during the algorithm calculation process, preventing the fixed-wing aircraft from escaping such obstacles and causing collisions. Therefore, the fixed-wing aircraft monocular obstacle avoidance method can also perform layered convex hull processing on the acquired 2.5D terrain map, specifically including the following steps: S3311: Obtain the original 2.5D map, perform layering on the 2.5D map, and cluster the singly connected obstacles in it; S3312: Perform convex hull processing on each layer of simply connected regions; S3313: Output the 2.5D terrain map.

[0062] Among them, the layered processing of 2.5D map mainly includes obtaining the original 2.5D original map map ori , starting from its lowest point H_min, with slice resolution k (Unit: m), perform horizontal sectioning until the slice height h p Until it is greater than the highest point H_max of the original map. h(map ori )=h p What you get map layeri It is a 2D grid map. This grid map can be regarded as a 2.5D map in height. h p The part of the cut surface that touches the terrain can be regarded as the occupied part of the 2D grid map, and the part of the cut surface that does not touch the terrain can be regarded as the unoccupied part of the 2D grid map. Due to the characteristics of the 2.5D map obstacles, the bottom obstacle must contain the upper obstacle, so it can be considered that the original 2.5D map height is h p and h p +k The sections between them can be highly h p This is a more conservative setting, represented by the cross-section, which actually increases more inaccessible areas, but provides protection for the safety of fixed-wing aircraft.

[0063] Clustering is performed on the single connected obstacles, mainly using calsimpleconnect The function performs single-connected clustering, and the specific calculation formula is: map layeri_sc =Calsimpleconnect ( map layeri ); Perform single-connected clustering on the obstacle area of each layer of the 2D grid map to generate a 2D grid map after single-connected clustering processing, such as Figure 10As shown in the figure, simply connected clustering means that if an obstacle point can be connected to another obstacle point by a line segment that does not cross the non-obstacle area, then the two obstacle points belong to the same cluster. Simply connected clustering is performed because we do not want unconnected obstacles to be included in the same convex hull. This would cause a large amount of non-obstacle area to be considered as obstacle areas, resulting in a decrease in the traversable area and an overly conservative obstacle avoidance system.

[0064] The convex hull processing of the single connected area of each layer is performed by mincovhull Calculate the minimum convex hull. The specific formula is: map layeri_sc_cov =Mincovhull ( map layeri_sc ); Use the Quick Hull Algorithm to perform convex envelopment on each single-connected area and fill the non-obstacle area in it to generate a 2D grid map after convex hull processing, such as Figure 11 Show.

[0065] The obstacle area of the 2D grid map after convex hull enveloping is given height, so that it becomes a new 2.5D terrain map, such as Figure 12 As shown. The height is assigned as h p +k ,Right now: h ( map layeri_sc_cov ) =h+k After each layer is processed, multiple layered 2.5D maps are added together, and the height is the highest value in each layered 2.5D map. The calculation formula is: H ( map cov ) =h ( map layeri_sc_cov ∩ map cov &h ( map cov ) <h ( map layeri_sc_cov )).

[0066] By preprocessing the original 2.5D map and enveloping it with a layered convex hull, fixed-wing aircraft can avoid being trapped in local minima. In layman's terms, this is like sealing the entrance to a dead end in the terrain, preventing fixed-wing aircraft from entering a dead end and being trapped in a local minimum, unable to escape.

[0067] It should also be noted that the bottom control module mainly controls the state change of the fixed-wing aircraft based on the navigation control quantity. The bottom control module mainly calculates the specific flight attitude and thrust, and distributes the control to the mechanical structures such as the servos and motors installed in the fixed-wing aircraft, thereby realizing the change of the state of the fixed-wing aircraft. Including the angular rate of the fixed-wing aircraft's three-dimensional track in the horizontal direction and the angular rate in the vertical direction, that is, based on the navigation control amount By adjusting the efficiency of mechanical structures such as servos and motors, the purpose of obstacle avoidance is achieved. Therefore, the flight speed of fixed-wing aircraft generally does not change significantly. Obstacle avoidance is achieved by adjusting the angle, which has a very good guarantee for the stability of fixed-wing aircraft.

[0068] In addition, the fixed-wing aircraft monocular obstacle avoidance method described in this application is limited to the field of fixed-wing aircraft. Fixed-wing aircraft, referred to as fixed-wing aircraft, refers to a heavier-than-air aircraft that flies within the atmosphere, with forward thrust or pull generated by a power unit and lift generated by the fixed wings of the fuselage. Different from rotary-wing aircraft or multi-rotor drones, fixed-wing aircraft have their own particularities in obstacle avoidance. Obstacle avoidance is achieved through angle adjustment, which can effectively ensure the flight safety of fixed-wing aircraft. Therefore, the fixed-wing aircraft monocular obstacle avoidance method can perform obstacle avoidance in a safe, efficient and low-cost manner when the distance information of the intruding flight obstacle is unknown.

[0069] Compared to existing technologies, the monocular obstacle avoidance method for fixed-wing aircraft provided by the present invention uses image processing and analysis to identify flying obstacles. It obtains the first estimated collision time and the first sight angle of the flying obstacle, and then performs obstacle avoidance control in combination with the fixed-wing aircraft's flight parameters. This method can acquire rich environmental information and identify complex objects and dynamic flying obstacles. It also offers low-cost processing and eliminates the need for depth cameras, infrared cameras, radar, and other equipment.

[0070] In addition, this method does not need to obtain relevant depth information and has high computational efficiency. It can perform real-time calculations using only the first estimated collision time and the first sight angle, and can effectively respond when a flying obstacle approaches at high speed.

[0071] The fixed-wing aircraft monocular obstacle avoidance method also incorporates the 2.5D terrain map and constructs a simulated radar to calculate terrain obstacles in front of the fixed-wing aircraft, integrating both flight obstacles and terrain obstacles into the overall obstacle avoidance framework. This streamlines data acquisition, recognition, and processing, accelerating the obstacle avoidance module's real-time aggregation of data and generation of navigation control variables.

[0072] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0073] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A monocular obstacle avoidance method for a fixed-wing aircraft, characterized in that: The steps include: S1: providing an image acquisition module, and acquiring a monocular image through the image acquisition module, wherein the monocular image contains a flight obstacle; S2: providing an image processing module, which processes the monocular image acquired by the image acquisition module within a preset time interval and generates a first estimated collision time and a first sight angle corresponding to the flight obstacle; S3: Providing an obstacle avoidance module, wherein the obstacle avoidance module collects the first estimated collision time and the first sight angle generated by the image processing module in real time, and calculates and generates a navigation control variable in combination with flight parameters of the fixed-wing aircraft; S4: Providing a bottom-level control module, wherein the bottom-level control module controls the state change of the fixed-wing aircraft based on the navigation control amount.

2. The fixed-wing aircraft monocular obstacle avoidance method according to claim 1, characterized in that: In S1, the image acquisition module includes at least three monocular cameras, which are respectively installed in front, on the left and on the right side of the fixed-wing aircraft. The horizontal shooting angle of at least three monocular cameras is not less than 225°, and the vertical shooting angle is not less than 30°.

3. The fixed-wing aircraft monocular obstacle avoidance method according to claim 1, characterized in that: The S2 includes: S21: Acquire a first image with an obstacle at time t1; S22: Acquire a second image having the flight obstacle at time t2; S23: Processing the first image and the second image to generate the first estimated collision time and the first sight angle based on a change in pixel area of the flight obstacle in the image; Among them, in S23, data processing is performed on the first image and the second image, including using the YOLO algorithm and the Kalman filter algorithm to identify the image features of the flight obstacle and obtain the first estimated collision time.

4. The fixed-wing aircraft monocular obstacle avoidance method according to claim 3, characterized in that: Said S2 further comprises: S24: The state of the fixed-wing aircraft continues to change, and the image processing module smoothes the first estimated collision time and the first sight angle that have changed beyond a first threshold within a predetermined time, and transmits them to the obstacle avoidance module.

5. The fixed-wing aircraft monocular obstacle avoidance method according to claim 1, characterized in that: The S3 includes: S31: Based on the first estimated collision time, obtain a first collision time corresponding to at least one of the flight obstacles ; S32: Based on the first sight angle, obtain a first azimuth angle corresponding to at least one of the flight obstacles and the first pitch angle ; S33: Acquire the flight parameters, including target point position information, fixed-wing aircraft position information, and fixed-wing aircraft speed information; S34: Calculate and generate the navigation control amount ; in, i It refers to the existence of i said flight obstacles.

6. The fixed-wing aircraft monocular obstacle avoidance method according to claim 5, characterized in that: The step S34 also includes: determining the obstacle avoidance priority and controlling the obstacle avoidance trend, wherein the first collision time is set. The second threshold, when the first collision time Below the second threshold, maintain the navigation control amount obstacle avoidance trend, reducing the influence weight of the target point position information.

7. The fixed-wing aircraft monocular obstacle avoidance method according to claim 5, characterized in that: The S33 further includes: S331: Obtain a 2.5D terrain map, construct a simulated radar, calculate terrain obstacles in front of the fixed-wing aircraft, and obtain a second estimated collision time and a second line of sight angle; S332: Based on the second estimated collision time, obtain a second collision time corresponding to at least one of the terrain obstacles ; S333: Based on the second sight angle, obtain a second azimuth angle corresponding to at least one of the terrain obstacles and the second pitch angle .

8. The fixed-wing aircraft monocular obstacle avoidance method according to claim 7, characterized in that: In the S34, the navigation control amount Including the angular rate of the fixed-wing aircraft's three-dimensional track in the horizontal direction and the angular rate in the vertical direction, the calculation formula is as follows: ; Wherein, N is the total number of the flight obstacles, and M is the total number of the terrain obstacles; is its own flying speed, is the flight obstacle avoidance control quantity, is the terrain obstacle avoidance control quantity, is the target point control amount, its own position information , target point location information .

9. The fixed-wing aircraft monocular obstacle avoidance method according to claim 7, characterized in that: In S331 , the 2.5D terrain map is pre-built and / or acquired in real time through a SLAM model.

10. The fixed-wing aircraft monocular obstacle avoidance method according to claim 9, characterized in that: Performing layered convex hull processing on the acquired 2.5D terrain map includes the following steps: S3311: Obtain the original 2.5D map, perform layering on the 2.5D map, and cluster the singly connected obstacles in it; S3312: Perform convex hull processing on each layer of simply connected regions; S3313: Output the 2.5D terrain map.

Images