Monocular obstacle avoidance method for fixed-wing aircraft

Abstract

The present invention provides a monocular obstacle avoidance method for a fixed-wing aircraft, comprising providing an image acquisition module to acquire monocular images; providing an image processing module to process the monocular images acquired by the image acquisition module within a preset time interval and generate a first estimated collision time and a first line of sight angle; providing an obstacle avoidance module to calculate and generate navigation control variables; and providing a bottom-level control module to control state changes of the fixed-wing aircraft. The monocular obstacle avoidance method for a fixed-wing aircraft provided by the present invention uses image processing and analysis to identify flight obstacles, and has low processing costs and excellent safety performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of aircraft obstacle avoidance technology, in particular to a fixed-wing aircraft monocular obstacle avoidance method. BACKGROUND

[0002] The flight activities of various types of aircraft are an important part of low-altitude economy. Compared with the driving of traditional ground vehicles, the safety requirements of aircraft are higher. When an aircraft collides, it may affect the stability of the flight, or even cause the aircraft to crash, and since the flight height is high, the crash may also cause secondary damage. Therefore, aircraft automatic obstacle avoidance is a very important part of flight safety.

[0003] The prior art includes ultrasonic ranging, infrared obstacle avoidance, laser ranging, laser radar, millimeter wave radar, and visual obstacle avoidance schemes. Among them, ultrasonic ranging is suitable for short-range obstacle avoidance, and the effect is poor for high-speed flight near the flight obstacles; infrared obstacle avoidance and laser ranging have high requirements for reflective objects, and the measurement accuracy is easily affected by the environment and the material of the obstacle itself; laser radar and millimeter wave radar have high costs, and the cost performance is not high for fixed-wing aircraft. These sensors have good sensing effect at close range, but the sensing credibility is greatly reduced at long distance.

[0004] As mentioned in the prior art patent CN115602003A, a method, device and unmanned aerial vehicle for realizing dynamic obstacle avoidance of unmanned aerial vehicle using monocular camera are provided, wherein the method includes measuring the distance between the dynamic obstacle and the unmanned aerial vehicle through the millimeter wave radar module. Millimeter wave radar, similar to binocular camera and laser radar, can obtain depth information of the obstacle environment, but millimeter wave radar has poor perception for long distances.

[0005] A few obstacle avoidance schemes still use monocular depth estimation methods based on deep learning. Such methods have high depth estimation credibility for small obstacles at close range and large static obstacles (such as hills and buildings) at long distances. However, the depth estimation accuracy of moving aircraft at long distances is very poor, and only identification can be achieved, so the depth information cannot be used. SUMMARY

[0006] The present application provides a fixed-wing aircraft monocular obstacle avoidance method for solving the above problems of high cost and insufficient accuracy.

[0007] The present application provides a fixed-wing aircraft monocular obstacle avoidance method, comprising the following steps:

[0008] S1: providing an image acquisition module, acquiring a monocular image through the image acquisition module, and the monocular image contains a flight obstacle;

[0009] S2: providing an image processing module, the image processing module processes the monocular image acquired by the image acquisition module within a preset time interval, and generates a first estimated collision time and a first line of sight angle corresponding to the flight obstacle through data processing;

[0010] S3: providing an obstacle avoidance module, the obstacle avoidance module real-time summarizes the first estimated collision time and the first line of sight angle generated by the image processing module, and combines the flight parameters of the fixed-wing aircraft to calculate and generate a navigation control quantity;

[0011] S4: providing a bottom control module, the bottom control module controls the state change of the fixed-wing aircraft based on the navigation control quantity.

[0012] Specifically, in the S1, the image acquisition module includes at least three monocular cameras, which are respectively installed in the front, left and right sides of the fixed-wing aircraft. The horizontal direction shooting angle of at least three monocular cameras is not less than 225°, and the vertical direction shooting angle is not less than 30°.

[0013] Specifically, the S2 includes:

[0014] S21: acquiring a first image with an obstacle at t1;

[0015] S22: acquiring a second image with the flight obstacle at t2;

[0016] S23: data processing of the first image and the second image, generating the first estimated collision time and the first line of sight angle through the change of the pixel area of the flight obstacle in the image;

[0017] In the S23, the data processing of the first image and the second image includes that the first image and the second image identify the image features of the flight obstacle through YOLO algorithm and Kalman filter algorithm, and obtain the first estimated collision time.

[0018] Specifically, the S2 further includes:

[0019] S24: the state of the fixed-wing aircraft changes continuously, the image processing module performs smoothing processing on the first estimated collision time and the first line of sight angle which change beyond the first threshold value within a predetermined time, and delivers them to the obstacle avoidance module.

[0020] Specifically, the S3 includes:

[0021] S31: based on the first estimated collision time, acquiring a first collision time corresponding to at least one flight obstacle

[0022] ​S32: Obtain a first azimuth angle corresponding to at least one of the flight obstacles based on the first line-of-sight angle and a first pitch angle ;

[0023] S33: Obtain the flight parameters, including target point position information, fixed-wing aircraft position information, and fixed-wing aircraft speed information

[0024] S34: Calculate the navigation control quantity ;

[0025] wherein, refers to the first flight obstacle.

[0026] Specifically, the S34 further includes judging an obstacle avoidance priority and controlling an obstacle avoidance trend, wherein a second threshold value of the first collision time is set, and when the first collision time is lower than the second threshold value, the obstacle avoidance trend based on the navigation control quantity is maintained, and the influence weight of the target point position information is reduced.

[0027] Specifically, the S33 further includes:

[0028] S331: Obtain a 2.5D terrain map, construct a simulated radar, calculate the terrain obstacles in front of the fixed-wing aircraft, and obtain a second estimated collision time and a second line-of-sight angle;

[0029] S332: Obtain a second collision time corresponding to at least one of the terrain obstacles based on the second estimated collision time

[0030] S333: Obtain a second azimuth angle corresponding to at least one of the terrain obstacles based on the second line-of-sight angle .

[0031] Specifically, in the S34, the navigation control quantity includes an angular rate of the three-dimensional flight path of the fixed-wing aircraft in the horizontal direction and an angular rate in the vertical direction, and the calculation formula is as follows:

[0032]

[0033] wherein, N is the total number of the flight obstacles, and M is the total number of the terrain obstacles; is the flight speed of the fixed-wing aircraft, is the obstacle avoidance control quantity of the flight obstacle, is the terrain obstacle avoidance control quantity, Target point control quantity, self position information Target point position information .

[0034] Specifically, in the S331, the 2.5D terrain map is acquired in real time through a pre-construction and / or through a SLAM model.

[0035] Specifically, the acquired 2.5D terrain map is subjected to layered convex hull processing, including the following steps:

[0036] S3311: acquiring a 2.5D original map, performing layered processing on the 2.5D map, and performing clustering processing on single-connected obstacles therein;

[0037] S3312: performing convex hull processing on each layer of single-connected regions;

[0038] S3313: outputting the 2.5D terrain map.

[0039] Compared with the prior art, the fixed-wing aircraft monocular obstacle avoidance method provided by the application identifies flight obstacles based on image processing and analysis, and performs obstacle avoidance control by acquiring a first estimated collision time and a first line-of-sight angle of the flight obstacles and combining flight parameters of the fixed-wing aircraft. The fixed-wing aircraft monocular obstacle avoidance method can acquire rich environmental information, identify complex objects and dynamic flight obstacles, and has a low processing cost without the need to install and use a depth camera, an infrared camera, a radar and the like.

[0040] In addition, the method does not need to acquire relevant depth information, has high calculation efficiency, and can be calculated in real time only by using the first estimated collision time and the first line-of-sight angle. When a flight obstacle approaches at a high speed, an effective response can be made.

[0041] After 500,000 Monte Carlo simulations, the probability of collision of the fixed-wing aircraft using the method provided by the application for obstacle avoidance is very low, and the safety performance is good. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0043] Figure 1 is a flow chart of the fixed-wing aircraft monocular obstacle avoidance method provided by the application;

[0044] Figure 2is a structural block diagram of the fixed-wing aircraft monocular obstacle avoidance method provided by the application;

[0045] Figure 3 is a flowchart of the fixed-wing aircraft monocular obstacle avoidance method in the embodiment;

[0046] Figure 4 is a flowchart of the S2 shown in Figure 1

[0047] Figure 5 is a flowchart of the S3 shown in Figure 1

[0048] Figure 6 is a flight schematic diagram of the fixed-wing aircraft in the embodiment;

[0049] Figure 7 is a flowchart of the S33 shown in Figure 5

[0050] Figure 8 is a 2.5D original map in the embodiment;

[0051] Figure 9 is a flowchart of the S331 shown in Figure 7

[0052] Figure 10 is a 2D grid map after the 2.5D original map shown in Figure 8

[0053] Figure 11 is a 2D grid map after the 2D grid map shown in Figure 10

[0054] Figure 12 is a 2.5D terrain map generated after the 2D grid map shown in Figure 11 DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only a part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the application.

[0056] ​​​​​​​​It should be noted that all direction indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present application are only used to explain the relative position relationship, movement condition, etc. between components in a certain posture (as shown in the drawings), and if the certain posture changes, the direction indications will also change accordingly.

[0057] In addition, the description such as "first", "second" and the like in the present application is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first", "second" can be explicitly or implicitly included at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0058] In addition, the technical solutions of various embodiments of the present application can be combined with each other, but it must be based on the realization of ordinary skilled in the art, when the combination of technical solutions appears contradictory or cannot be realized, it should be considered that the combination of technical solutions does not exist, also not within the scope of protection required by the present application.

[0059] Please refer to Figure 1 and Figure 2 , Figure 1 is the flow chart of the monocular obstacle avoidance method of the fixed-wing aircraft provided by the present application, Figure 2 is the structure block diagram of the monocular obstacle avoidance method of the fixed-wing aircraft provided by the present application. The monocular obstacle avoidance method of the fixed-wing aircraft comprises the following steps:

[0060] S1: providing an image acquisition module, acquiring a monocular image through the image acquisition module, the monocular image containing a flight obstacle;

[0061] S2: providing an image processing module, the image processing module processing the monocular image acquired by the image acquisition module in a preset time interval, and performing data processing to generate a first estimated collision time and a first line of sight angle corresponding to the flight obstacle;

[0062] S3: providing an obstacle avoidance module, the obstacle avoidance module real-time summarizes the first estimated collision time parameter and the first line of sight angle generated by the image processing module, and combines the flight parameters of the fixed-wing aircraft to calculate and generate a navigation control quantity;

[0063] S4: providing a bottom control module, the bottom control module controls the state change of the fixed-wing aircraft based on the navigation control quantity.

[0064] It should be noted that in the S1, the flight obstacles include invading aircraft, balloons or birds, etc., which are not limited herein, and the image acquisition module and the image processing module can recognize and process them by using the fixed-wing aircraft monocular obstacle avoidance method. Of course, the image acquisition module acquires the monocular image in real time, and images not containing the flight obstacles are screened to reduce the data for image processing.

[0065] In order to avoid collision with the flight obstacles from the side, in the S1, the image acquisition module includes at least three monocular cameras, which are respectively installed at the front, left side and right side of the fixed-wing aircraft. The horizontal direction shooting angle of the at least three monocular cameras is not less than 225°, and the vertical direction shooting angle is not less than 30°. The monocular camera is a camera device using only one optical sensor and lens, mainly used for capturing images or videos of a single perspective, and the specific model and size are not specifically required. By using the three monocular cameras and setting them at the front, left side and right side, the situation in front of and on both sides of the fixed-wing aircraft can be effectively recognized. Generally, the monocular camera can monitor 30° in the vertical direction, so it is not necessary to be set at the top and bottom of the fixed-wing aircraft, so as to reduce the cost as much as possible while achieving the monitoring requirement. The horizontal direction monitoring can be covered by the three monocular cameras, and the horizontal direction shooting angle is not less than 225°, so that the three monocular cameras can cross monitor the front left and front right of the fixed-wing aircraft, avoiding the existence of dead angles. The rear of the fixed-wing aircraft is not considered in the obstacle avoidance, but is realized based on the obstacle avoidance of other aircraft itself. If additional monitoring is required, a rear monocular camera can be added, which is not described herein.

[0066] In addition, it should be noted that the horizontal direction shooting angle of the monocular camera is not less than 225°, and the vertical direction shooting angle is not less than 30°, wherein the angles are symmetrically set in the horizontal flight state of the fixed-wing aircraft. That is, the horizontal plane where the fixed-wing aircraft is located is taken as the reference, and the pitch angles are symmetrically at least 15°; the vertical plane where the nose of the fixed-wing aircraft is located is taken as the reference, and the horizontal sides are symmetrically at least 112.5°.

[0067] Please continue to refer to Figure 3 and Figure 4 The image processing module processes the monocular images acquired by the image acquisition module within a predetermined time interval to generate the first estimated collision time and the first line of sight angle corresponding to the flight obstacles. Specifically, the S2 includes the following steps:

[0068] S21: acquiring a first image with obstacles at t1;

[0069] S22: collect a second image having the flight obstacle at t2;

[0070] S23: data process the first image and the second image, generate the first estimated collision time and the first line of sight angle through the pixel area change of the flight obstacle in the image;

[0071] S24: the fixed-wing aircraft state continues to change, the image processing module performs smoothing processing on the first estimated collision time and the first line of sight angle which change beyond the first threshold value within a predetermined time, and delivers to the obstacle avoidance module.

[0072] In the S23, the data processing of the first image and the second image includes the first image and the second image identifying the image features of the flight obstacle and estimating the first estimated collision time through YOLO algorithm and Kalman filter algorithm.

[0073] In this embodiment, the preset time interval is equal to the time difference between t2 and t1, and the preset time interval is preset in advance. The specific data can be determined according to the safety needs of the fixed-wing aircraft itself and the acquisition frequency of the monocular camera, and is not limited here. Of course, the preset time interval should be a specific and explicit time length. After the image processing module identifies the image feature change of the flight obstacle in the first image and the second image, the first estimated collision time and the first line of sight angle can be effectively generated by combining the preset time calculation.

[0074] The image processing module mainly identifies the image features of the flight obstacle through the YOLO algorithm and the filter.

[0075] The YOLO (You only look once) algorithm is an image target recognition algorithm proposed by Joseph Redmon et al. in 2015. Unlike traditional two-stage target detection methods, YOLO algorithm simplifies the entire detection process into a single regression problem, which can simultaneously predict the position and category of objects in the image through one traversal, and the detection speed is very fast, and the overall precision is high.

[0076] The filter has various types, such as the Kalman filter used in this embodiment. Kalman filter is a kind of efficient recursive filter, which can estimate the real-time state according to the real-time acquisition of the first image and the second image, combined with the dynamic model of the system, and has excellent real-time performance. Simply speaking, Kalman filter is like an intelligent data processor, which can accurately extract the difference between the first image and the second image from the mixed noise data, that is, the change of the flight obstacle relative to the fixed-wing aircraft.

[0077] The fixed-wing aircraft monocular obstacle avoidance method based on the use of the YOLO algorithm and the filter can effectively identify and analyze the image features of the flight obstacles in real time in a low-altitude environment with high-speed changes, which is conducive to the safe obstacle avoidance of the fixed-wing aircraft.

[0078] Of course, due to the setting of the preset time interval, the image processing module will process a large number of first images and second images in a certain period of time and generate multiple sets of first estimated collision times and first line-of-sight angles. When the positions and postures of the flight obstacles and the fixed-wing aircraft change at a high speed, the first estimated collision times and the first line-of-sight angles will fluctuate greatly.

[0079] Therefore, when the state of the fixed-wing aircraft changes too fast, the image processing module performs smoothing processing on the first estimated collision times and the first line-of-sight angles that change beyond a first threshold value within a predetermined time, and delivers them to the obstacle avoidance module.

[0080] The predetermined time is set in advance and is relatively greater than the preset time interval. The specific value is set according to the self-application situation and safety performance of the fixed-wing aircraft, and is not specifically limited. That is, within the predetermined time range, the first estimated collision times and the first line-of-sight angles generated by multiple preset time intervals are monitored, and when they fluctuate greatly, the first estimated collision times and the first line-of-sight angles are smoothed to ensure that the input and output of the obstacle avoidance module are more stable, and the underlying control module will not appear multiple frequencies and complex control, thereby enhancing the safety of the fixed-wing aircraft control. Of course, the smoothing processing method is a prior art, such as a mean algorithm, a median algorithm, a normal distribution algorithm, or a variance algorithm, which is not specifically described here.

[0081] Please continue to refer to Figure 5 and Figure 6 For the calculation of the navigation control quantity, the S3 includes the following steps:

[0082] S31: Based on the first estimated collision time, obtain the first collision time of at least one intruding obstacle corresponding to the flight obstacle ;

[0083] S32: Based on the first line-of-sight angle, obtain the first azimuth angle and the first pitch angle corresponding to at least one flight obstacle ;

[0084] S33: Obtain the flight parameters, including target point position information, fixed-wing aircraft position information, and fixed-wing aircraft speed information; ​

[0085] S34: calculating the navigation control quantity .

[0086] It should be noted that, with the rapid development of low-altitude economy, in the future low-altitude environment, multiple flight obstacles may appear around the flight path of the fixed-wing aircraft. Therefore, the monocular obstacle avoidance method of the fixed-wing aircraft needs to consider the obstacle avoidance demand in complex situations in advance. Based on at least one flight obstacle, at least one set of first estimated collision time and first line of sight angle is generated, and at least one set of first collision time of the intruding obstacle corresponding to the flight obstacle is obtained , the first azimuth angle and the first pitch angle . Among them, represents the first i flight obstacle, that is, according to the number of intruding aircraft or other dynamic obstacles, if there is only one intruding aircraft, =1; if there are two intruding aircrafts, =1, 2.

[0087] In this embodiment, when calculating the navigation control quantity , the flight parameters also need to be combined to ensure the accuracy and efficiency of control. The target point position information is determined before the fixed-wing aircraft takes off, which is generally obtained by loading the GPS position information corresponding to the target position. The addition of the target point position information is to ensure that the fixed-wing aircraft considers the nearest flight route to the target point while avoiding obstacles. That is, based on the principle of two-point-one-line shortest, the weight of obstacle avoidance and flight to the target point is combined for control, so that the shortest distance is ensured while avoiding obstacles and flying to the target point.

[0088] In addition, the flight parameters also include fixed-wing aircraft position, attitude and speed parameters, and corresponding information can be obtained by GPS, IMU and other sensors. Of course, it can be obtained without additional retrieval in normal flight state; when the obstacle avoidance module performs strategy calculation and selection, it is temporarily retrieved as a decision basis.

[0089] Further, in the S34, it also includes judging the obstacle avoidance priority and controlling the obstacle avoidance trend, wherein a second threshold of the first collision time is set, when the first collision time is lower than the second threshold, the obstacle avoidance trend based on the navigation control quantity is maintained, and the influence weight of the target point position information is reduced.

[0090] Here is an example. When the flight obstacle is in a remote position, that is, the first collision time When the first collision time is larger, and the fixed-wing aircraft has enough decision time to avoid the obstacle, the obstacle avoidance weight can be lower, and the fixed-wing aircraft still flies towards the target point; when the first collision time is smaller , the obstacle avoidance weight is higher, and the fixed-wing aircraft gives priority to obstacle avoidance and further monitors the latest situation of the flight obstacle, and after the fixed-wing aircraft adjusts and confirms safety, the fixed-wing aircraft continues to adjust to fly towards the target point; when the fixed-wing aircraft is very close to the target point, the obstacle avoidance weight is the highest, and the fixed-wing aircraft does not consider the target point position information, and adjusts after ensuring safety.

[0091] The judgment of the first collision time is mainly based on the second threshold. Of course, the second threshold is set based on the self condition and safety performance of the fixed-wing aircraft. When the first collision time is lower than the second threshold, it can be considered as a very critical situation, so it is not necessary to collect multiple times, and the obstacle avoidance trend based on the navigation control quantity is directly maintained, and full obstacle avoidance is performed for a certain period of time. The fixed-wing aircraft is no longer pulled by the target point, and whether it is safe or far away from the target point is judged again under the new state.

[0092] Therefore, in combination with the flight parameters and the corresponding weight relationship, the navigation control quantity is calculated and generated. On the premise of ensuring safety, the fixed-wing aircraft can fly along the most reasonable route and reduce energy consumption.

[0093] Please continue to refer to Figure 7 , and the S33 further includes the following steps:

[0094] S331: Obtain a 2.5D terrain map, construct a simulated radar, calculate the terrain obstacles in front of the fixed-wing aircraft, and obtain a second estimated collision time and a second line-of-sight angle;

[0095] S332: Based on the second estimated collision time, obtain at least one second collision time corresponding to the terrain obstacle;

[0096] S333: Based on the second line-of-sight angle, obtain at least one second azimuth angle and second pitch angle corresponding to the terrain obstacle.

[0097] It should be noted that the 2.5D terrain map is a visualization map between a 3D terrain map and a 2D terrain map. Through technical means such as inclined viewing angle, height stretching, or shadow rendering, a stereoscopic effect is simulated while the accuracy of a two-dimensional map is retained. The specific implementation manner is to increase height information on the basis of 2D terrain map data to realize construction of a data map.

[0098] Through the 2.5D terrain map, the fixed-wing aircraft can simultaneously avoid the flight obstacles and the terrain obstacles. Moreover, compared with the 2D terrain map, the 2.5D terrain map increases the height information, which is helpful to identify the terrain obstacles, and the storage space occupied is obviously lower than that of the 3D terrain map, so that the 2.5D terrain map can be more efficiently and quickly retrieved and used.

[0099] The terrain obstacles in front of the fixed-wing aircraft are calculated by constructing a simulated radar, and the second estimated collision time and the second line-of-sight angle are obtained. The terrain obstacles do not need to be detected, identified and calculated on the physical level, but can be realized by an internal algorithm. Specifically, the second estimated collision time and the second line-of-sight angle are simulated and calculated based on the 2.5D terrain map and the flight parameters of the fixed-wing aircraft, the internal flight is simulated, and the fixed-wing aircraft is simulated to emit a radar signal to the terrain obstacle in the 2.5D terrain map, so as to simulate and calculate the second estimated collision time and the second line-of-sight angle. This method can accurately generate the second estimated collision time and the second line-of-sight angle internally, and is not affected by detection sensors, flight environment and data processing modules on the physical level, and is directly calculated internally.

[0100] In addition, the second collision time , the second azimuth angle and the second pitch angle corresponding to the flight obstacle are respectively calculated and generated based on the second estimated collision time and the second line-of-sight angle , the first azimuth angle and the first pitch angle , and the parameters of the first collision time , the first azimuth angle

[0101] and the first pitch angle remain consistent, so that the obstacle avoidance module can be directly identified and processed without the need for further processing. In the case where the 2.5D terrain map is known, the flight obstacles with unknown distances are unified in the obstacle avoidance framework, so as to avoid complex obstacle avoidance priority judgment.

[0102] The calculation step procedure and formula are as follows:

[0103]

[0104] The navigation control quantity Based on its own parameters, the first collision time , the first azimuth angle , the first pitch angle The second collision time , the second azimuth and the second pitch angle Calculation is performed. Where i refers to at least one of the possible flight obstacles or terrain obstacles, N is the total number of the flight obstacles, and M is the total number of the terrain obstacles.

[0105] The second threshold is a parameter set in advance for evaluating the first collision time Is it lower than the second threshold? Only when the first collision time Below the second threshold Only when the flight obstacle avoidance control is involved The first threshold A parameter set in advance for evaluating the second collision time Is it lower than the first threshold? Only when the second collision time Below the first threshold When the terrain obstacle avoidance control amount is involved The definition of specific parameters is based on the specific environment needs and safety requirements and is not limited here.

[0106] The navigation control amount Including flight obstacle avoidance control , terrain obstacle avoidance control amount and target point control amount , add the three components to get the navigation control amount , so that all relevant flight obstacles and terrain obstacles can be included in the obstacle avoidance decision and comprehensively considered to ensure the accuracy of the obstacle avoidance decision and the safety of the fixed-wing aircraft flight.

[0107] Among them, the flight obstacle avoidance control amount Mainly based on the first collision time of at least one of the flight obstacles , the first azimuth angle , the first pitch angle and its own flight speed Calculate the obstacle avoidance control amount and sum it up. Mainly based on the second collision time of at least one of the terrain obstacles the second azimuth angle and the second pitch angle and the self flight speed The obstacle avoidance control amount is calculated and summed up. The target point control amount is mainly calculated according to the self position information , the target point position information and the self flight speed . The flight obstacle avoidance control amount , the terrain obstacle avoidance control amount and the target point control amount all contain the angular velocity of the three-dimensional flight path in the horizontal direction and the angular velocity in the vertical direction. The obtained angular velocity of the three-dimensional flight path in the horizontal direction and the angular velocity in the vertical direction are delivered to the bottom controller, which calculates the corresponding attitude and thrust and distributes them to each actuator for execution.

[0108] In the S331, the 2.5D terrain map is obtained by pre-construction and / or by a SLAM model in real time. It should be noted that the 2.5D terrain map can be pre-constructed and saved in a fixed-wing aircraft in advance, and directly called in the flight process; of course, it can also be obtained in real time in the flight stage through the SLAM model. In this embodiment, the scheme of both existing at the same time is preferred. That is, the existing 2.5D terrain map is pre-stored in the fixed-wing aircraft, and in the flight process, it is obtained in real time based on the need through the SLAM model, ensuring the accuracy and integrity of the information while reducing the call of the SLAM model as much as possible, thereby reducing the working steps, shortening the response time, reducing the energy consumption and ensuring the flight safety.

[0109] The SLAM model is the abbreviation of Simultaneous Localization and Mapping, which means real-time positioning and map construction, that is, the fixed-wing aircraft can position itself according to the position and environment during movement, and construct the environment map at the same time. The main purpose of SLAM is to construct a map and estimate a position in an unknown environment, so as to guide the next action, such as navigation, obstacle avoidance, grabbing and other tasks. The construction of specific models and algorithms is not described here.

[0110] Please continue to refer to Figure 8 to Figure 11Since the fixed-wing aircraft may face some non-convex terrain (such as a valley, cul-de-sac, etc.) that cannot be directly passed during flight. Such non-convex terrain will cause a local optimal trap in the algorithm calculation process, resulting in the fixed-wing aircraft being unable to escape such obstacles and colliding. Therefore, the monocular obstacle avoidance method of the fixed-wing aircraft can also perform layered convex hull processing on the obtained 2.5D terrain map, specifically including the following steps:

[0111] S3311: Obtain a 2.5D original map, perform layered processing on the 2.5D map, and perform clustering processing on single-connected obstacles therein;

[0112] S3312: Perform convex hull processing on each layer of single-connected regions;

[0113] S3313: Output the 2.5D terrain map.

[0114] Wherein, the layered processing of the 2.5D map mainly includes obtaining the original 2.5D original map map ori , starting from the lowest point H_min, performing horizontal section processing with a slice resolution k (unit: m), until the slice height h p is greater than the highest point H_max of the original map. Take h(map ori )=h p The obtained map layeri is a 2D grid map, which can be regarded as a horizontal section of the 2.5D map in height h p . The part of the section in contact with the terrain can be regarded as the occupied part of the 2D grid map, and the part of the section not in contact with the terrain can be regarded as the unoccupied part of the 2D grid map. Due to the characteristics of the 2.5D map obstacles, the bottom layer of obstacles must contain the upper layer of obstacles, so it can be considered that the horizontal section of the original 2.5D map height between h p and h p +k can be represented by the horizontal section of height h p , which is a more conservative setting, which actually increases more impassable areas, but provides safety for the fixed-wing aircraft.

[0115] The clustering processing of the single-connected obstacles therein mainly uses the calsimpleconnect function to perform single-connected clustering, and the specific calculation formula is:

[0116] map layeri_sc =Calsimpleconnect ( map layeri );

[0117] The obstacle regions of each layer of the 2D grid map are single-connected clustered to generate a 2D grid map after single-connected clustering, as shown in FIG. 4. Figure 10 The single-connected clustering means that if there is a line segment connecting two obstacle points without passing through a non-obstacle region, the two obstacle points belong to a class. The single-connected clustering is performed because it is not desirable that the same convex hull contains non-connected obstacles, which would cause a large number of non-obstacle regions to be considered as obstacle regions, resulting in a smaller passable region and overly conservative flight obstacle avoidance.

[0118] The convex hull processing of each layer of the single-connected region is performed using mincovhull The minimum convex hull calculation is performed, and the specific formula is as follows:

[0119] map layeri_sc_cov =Mincovhull ( map layeri_sc );

[0120] The Quick Hull Algorithm is used to perform convex hull processing on each single-connected region, and the non-obstacle regions in the convex hull are filled to generate a 2D grid map after convex hull processing, as shown in FIG. 5. Figure 11

[0121] The obstacle regions of the 2D grid map after convex hull processing are assigned with heights to become a new 2.5D terrain map, as shown in FIG. 6. The height assignment is Figure 12 h p +k , that is: h ( map layeri_sc_cov ) =h+k After the processing of each layer is completed, the multiple layered 2.5D maps are added, and the height is the highest value in each layered 2.5D map. The calculation formula is as follows:

[0122] H ( map cov ) =h ( map layeri_sc_cov ∩ map cov &h ( map cov )​​<h map layeri_sc_cov

[0123] By preprocessing the 2.5D original map, the 2.5D original map is enveloped by a layered convex hull, thereby avoiding the problem of fixed-wing aircraft falling into local minimum. In a popular way, the entrance of the dead end in the terrain is closed, so as to avoid the fixed-wing aircraft from entering the dead end and being trapped in the local minimum due to the inability to cross and escape.

[0124] It should be further pointed out that the bottom control module is mainly based on the navigation control quantity to control the state change of the fixed-wing aircraft. The bottom control module mainly distributes the control to the mechanical structures such as rudders and motors provided in the fixed-wing aircraft by solving the specific flight attitude and thrust, so as to realize the change of the state of the fixed-wing aircraft. The navigation control quantity includes the angular rate of the three-dimensional track of the fixed-wing aircraft in the horizontal direction and the angular rate in the vertical direction, that is, based on the navigation control quantity adjust the working efficiency of the mechanical structures such as rudders and motors, so as to achieve the purpose of obstacle avoidance. Therefore, the flight speed of the fixed-wing aircraft generally does not change greatly, and the obstacle avoidance is realized by adjusting the angle, which has very good guarantee for the stability of the fixed-wing aircraft.

[0125] In addition, the monocular obstacle avoidance method of the fixed-wing aircraft in the present application is limited in the field of fixed-wing aircraft. The fixed-wing aircraft, which is also called the fixed-wing machine, is an aircraft heavier than air flying in the atmosphere, which generates forward thrust or pull force by a power device and generates lift by a fixed wing of the fuselage. Different from the rotary-wing aircraft or multi-rotor unmanned aerial vehicle, the fixed-wing aircraft obstacle avoidance has its particularity, and the obstacle avoidance is realized by angle adjustment, which can effectively ensure the flight safety of the fixed-wing aircraft. Therefore, the monocular obstacle avoidance method of the fixed-wing aircraft can safely, efficiently and low-costly avoid obstacles in the case of unknown distance information of the flight obstacle.

[0126] Compared with the prior art, the monocular obstacle avoidance method of the fixed-wing aircraft provided by the present application identifies the flight obstacle based on image processing and analysis, avoids obstacles by acquiring the first estimated collision time of the flight obstacle and the first line of sight angle, and combining the flight parameters of the fixed-wing aircraft. The monocular obstacle avoidance method of the fixed-wing aircraft can obtain rich environmental information, identify complex objects and dynamic flight obstacles, and the processing method has low cost without the need to install and use a depth camera, an infrared camera and a radar.

[0127] ​​In addition, the method does not need to obtain related depth information, has high calculation efficiency, can be calculated in real time only by using the first estimated collision time and the first line of sight angle, and can effectively respond when the flight obstacle approaches at high speed.

[0128] The fixed-wing aircraft monocular obstacle avoidance method also introduces the 2.5D terrain map, and calculates the terrain obstacles in front of the fixed-wing aircraft by constructing a simulated radar, so that the flight obstacles and the terrain obstacles are unified in the entire obstacle avoidance framework. There is no obstacle in data acquisition, identification and processing, which accelerates the real-time data summary of the obstacle avoidance module and generates the navigation control quantity.

[0129] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0130] The above-described embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A monocular obstacle avoidance method for a fixed-wing aircraft, characterized in that: The steps include: S1: providing an image acquisition module, and acquiring a monocular image through the image acquisition module, wherein the monocular image contains a flight obstacle; S2: Providing an image processing module, wherein the image processing module performs data processing on the monocular image acquired by the image acquisition module within a preset time interval to generate a first estimated collision time and a first sight angle corresponding to the flight obstacle; S3: Providing an obstacle avoidance module, wherein the obstacle avoidance module collects the first estimated collision time and the first sight angle generated by the image processing module in real time, and calculates and generates a navigation control variable in combination with flight parameters of the fixed-wing aircraft; S4: providing a bottom-level control module, wherein the bottom-level control module controls a state change of the fixed-wing aircraft based on the navigation control variable; Wherein, the S3 includes: S31: Based on the first estimated collision time, obtain a first collision time corresponding to at least one of the flight obstacles ; S32: Based on the first sight angle, obtain a first azimuth angle corresponding to at least one of the flight obstacles and the first pitch angle ; S33: Acquiring the flight parameters, including target point position information, fixed-wing aircraft position information, and fixed-wing aircraft speed information; S33 also includes: S331: Obtain a 2.5D terrain map, construct a simulated radar, calculate terrain obstacles in front of the fixed-wing aircraft, and obtain a second estimated collision time and a second line of sight angle; S332: Based on the second estimated collision time, obtain a second collision time corresponding to at least one of the terrain obstacles ; S333: Based on the second sight angle, obtain a second azimuth angle corresponding to at least one of the terrain obstacles and the second pitch angle ; S34: Calculate and generate the navigation control amount In the S34, the navigation control amount Including the angular rate of the fixed-wing aircraft's three-dimensional track in the horizontal direction and the angular rate in the vertical direction, the calculation formula is as follows: ,in, ,in, Wherein, N is the total number of the flight obstacles, and M is the total number of the terrain obstacles; is its own flying speed, is the flight obstacle avoidance control quantity, is the terrain obstacle avoidance control quantity, is the target point control amount, its own position information , target point location information ;in, i It refers to the existence of i said flight obstacles.

2. The fixed-wing aircraft monocular obstacle avoidance method according to claim 1, characterized in that: In S1, the image acquisition module includes at least three monocular cameras, which are respectively installed in front, on the left and on the right side of the fixed-wing aircraft. The horizontal shooting angle of at least three monocular cameras is not less than 225°, and the vertical shooting angle is not less than 30°.

3. The fixed-wing aircraft monocular obstacle avoidance method according to claim 1, characterized in that: The S2 includes: S21: Acquire a first image with an obstacle at time t1; S22: Acquire a second image having the flight obstacle at time t2; S23: Processing the first image and the second image to generate the first estimated collision time and the first sight angle based on a change in pixel area of ​​the flight obstacle in the image; Among them, in S23, data processing is performed on the first image and the second image, including using the YOLO algorithm and the Kalman filter algorithm to identify the image features of the flight obstacle and obtain the first estimated collision time.

4. The fixed-wing aircraft monocular obstacle avoidance method according to claim 3, characterized in that: Said S2 further comprises: S24: The state of the fixed-wing aircraft continues to change, and the image processing module smoothes the first estimated collision time and the first sight angle that have changed beyond a first threshold within a predetermined time, and transmits them to the obstacle avoidance module.

5. The fixed-wing aircraft monocular obstacle avoidance method according to claim 1, characterized in that: The step S34 also includes: determining the obstacle avoidance priority, controlling the obstacle avoidance trend, wherein the first collision time is set The second threshold, when the first collision time Below the second threshold, maintain the navigation control amount obstacle avoidance trend, reducing the influence weight of the target point position information.

6. The fixed-wing aircraft monocular obstacle avoidance method according to claim 1, characterized in that: In S331 , the 2.5D terrain map is pre-built and / or acquired in real time through a SLAM model.

7. The fixed-wing aircraft monocular obstacle avoidance method according to claim 6, characterized in that: Performing layered convex hull processing on the acquired 2.5D terrain map includes the following steps: S3311: Obtain the original 2.5D map, perform layering on the 2.5D map, and cluster the singly connected obstacles in it; S3312: Perform convex hull processing on each layer of simply connected regions; S3313: Output the 2.5D terrain map.

Images