Autonomous obstacle avoidance method and system for fixed-wing unmanned aerial vehicle

Abstract

The invention relates to the technical field of unmanned aerial vehicle obstacle avoidance, and provides a fixed-wing unmanned aerial vehicle autonomous obstacle avoidance method and system, and the method comprises the steps: extracting a preset route of a fixed-wing unmanned aerial vehicle, and collecting a left visual angle depth image, a right visual angle depth image, a fusion depth image and all piece element information of each collection moment in the flight process of the fixed-wing unmanned aerial vehicle; acquiring a cluster of the chip element information, judging whether the chip element information is a track approaching obstacle or not, acquiring a measurable change rate of the track approaching obstacle, acquiring attitude variables at all acquisition moments, and determining an orthogonal degree of the track approaching obstacle; edge images of the left visual angle depth image and the right visual angle depth image are determined, edges in all the edge images are matched, non-significant edges are screened, a to-be-selected track formed by the non-significant edges is determined, and single visual angle visibility is determined; and realizing autonomous obstacle avoidance of the fixed-wing unmanned aerial vehicle according to the single-view-angle visibility. According to the invention, the obstacle avoidance accuracy of the fixed-wing unmanned aerial vehicle in the flight process can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of obstacle avoidance for unmanned aerial vehicles (UAVs), and particularly to an autonomous obstacle avoidance method and system for fixed-wing UAVs. Background Art

[0002] A fixed-wing UAV is a type of UAV whose wings remain unchanged during flight and relies on the thrust generated by a power device and the lift generated by the wings for flight. Since a fixed-wing UAV depends on the wings to generate lift and maintains a relatively high speed during flight, it usually requires a larger turning radius to complete a turn, and the obstacle avoidance strategy is more dependent on advance prediction and planning.

[0003] Currently, generally, the SLAM lidar technology and binocular vision technology are used in combination. Based on the accurate distance information provided by the lidar and the rich scene understanding provided by the binocular vision, the accuracy and robustness of obstacle recognition during the flight of a fixed-wing UAV are improved. However, affected by the divergence density of pulsed lasers and the loss of detailed information in binocular vision, it is impossible to effectively identify and track insignificant obstacles encountered during the flight of a fixed-wing UAV, resulting in obstacle avoidance risks. Summary of the Invention

[0004] The present invention provides an autonomous obstacle avoidance method and system for fixed-wing UAVs to solve the problem of insufficient recognition accuracy of insignificant obstacles encountered during the flight of fixed-wing UAVs and the existence of obstacle avoidance risks. The specific technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present invention provides an autonomous obstacle avoidance method for a fixed-wing UAV, the method comprising the following steps:

[0006] Extract the preset flight path of the fixed-wing UAV, and collect the left-view depth image, right-view depth image, fused depth image, and all fragment information at each acquisition moment during the flight of the fixed-wing UAV. The fragment information contains different lidar point cloud data;

[0007] Cluster all the lidar point cloud data contained in the fragment information to obtain the clustering clusters of the fragment information. According to all the lidar point cloud data in the clustering clusters of the corresponding fragment information collected at two adjacent acquisition moments, determine whether the fragment information is a trajectory approaching obstacle. If so, obtain the measurable change rate of the trajectory approaching obstacle at each acquisition moment. According to the preset flight path of the fixed-wing UAV, obtain the attitude variables at all acquisition moments. According to the differences between the attitude variables of the trajectory approaching obstacle at all consecutive acquisition moments and the differences between the measurable change rates, determine the orthogonality of the trajectory approaching obstacle;

[0008] Determine the edge images of the left-view depth image and the right-view depth image according to the orthogonality of the track approaching the obstacle in the left-view depth image and the right-view depth image. Match the edges in all edge images, filter out insignificant edges, and determine the candidate tracks formed by the insignificant edges. Determine the single-view visibility according to the distortion variables in the X direction and the Y direction of all the insignificant edges corresponding to all candidate tracks;

[0009] Realize autonomous obstacle avoidance for a fixed-wing UAV according to the single-view visibility.

[0010] Furthermore, the method for judging whether the fragment information is a track approaching an obstacle according to all the lidar point cloud data within the clustering cluster of the corresponding fragment information collected at two adjacent acquisition times, and if so, obtaining the measurable change rate of the track approaching the obstacle at each acquisition time specifically includes:

[0011] Denote the mean value of the densities of all the lidar point cloud data within all the clustering clusters of the fragment information as the density mean value of the fragment information; denote the variance of all the lidar point cloud data within all the clustering clusters of the fragment information as the data variance of the fragment information; denote the product of the number of all the lidar point cloud data included in the fragment information and the density mean value of the fragment information as the first product of the fragment information, and denote the ratio of the first product of the fragment information to the data variance of the fragment information as the measurable degree of the fragment information;

[0012] Denote any acquisition time as the target acquisition time, and denote the previous adjacent acquisition time of the target acquisition time as the previous adjacent time. When the measurable degree of the fragment information collected at the target acquisition time is greater than the measurable degree of the corresponding fragment information collected at the previous adjacent time, denote the fragment information collected at the target acquisition time as a track approaching an obstacle; for each track approaching an obstacle, denote the normalized value of the difference between the measurable degree of the track approaching the obstacle at the target acquisition time and the measurable degree of the track approaching the obstacle at the previous adjacent time as the measurable change rate of the track approaching the obstacle at the target acquisition time;

[0013] When the measurable degree of the fragment information collected at the target acquisition time is less than or equal to the measurable degree of the corresponding fragment information collected at the previous adjacent time, the fragment information is not a track approaching an obstacle.

[0014] Furthermore, the method for obtaining the attitude variables at all acquisition times according to the preset flight path of the fixed-wing UAV specifically includes:

[0015] Obtain the roll angle, pitch angle, yaw angle and rotation direction of the fixed-wing UAV at each acquisition time according to the preset flight path of the fixed-wing UAV;

[0016] Arrange the roll angle, pitch angle, yaw angle, and rotation direction at the same acquisition moment in sequence to obtain the rotation vector at the same acquisition moment;

[0017] Denote the vector difference between the rotation vectors at the target acquisition moment and the previous adjacent moment as the attitude variable at the target acquisition moment.

[0018] Furthermore, the method for determining the orthogonality of the trajectory approaching the obstacle is as follows:

[0019] Denote the information entropy of the magnitudes of the attitude variables at all consecutive acquisition moments of the same trajectory approaching the obstacle as the attitude complexity of the same trajectory approaching the obstacle;

[0020] Denote the variance of the measurable change rates at all consecutive acquisition moments of the same trajectory approaching the obstacle as the obstacle complexity of the same trajectory approaching the obstacle;

[0021] Denote the normalized value of the ratio of the attitude complexity to the obstacle complexity of the same trajectory approaching the obstacle as the orthogonality of the same trajectory approaching the obstacle.

[0022] Furthermore, the method for determining the edge images of the left-view depth image and the right-view depth image is as follows:

[0023] Denote any left-view depth image or right-view depth image as the target depth image. Process the target depth image using the Otsu method to obtain the segmentation threshold x. Use the Canny edge detection algorithm to perform edge recognition on the target depth image respectively to obtain the edge image corresponding to the target depth image. Among them, the low threshold for the trajectory approaching obstacle i in the target depth image is set to The high threshold is set to ω i is the orthogonality of the trajectory approaching obstacle i in the target depth image.

[0024] Furthermore, the specific method for matching the edges in all edge images, screening out insignificant edges, and determining the candidate trajectories composed of insignificant edges includes:

[0025] Match the edges in the edge images at adjacent acquisition moments to obtain the matching edges. Denote the edges that are matching edges for three or more consecutive acquisition moments as significant edges, and denote the edges that are not significant edges as suspected insignificant edges;

[0026] Match all suspected insignificant edges at different acquisition moments, and denote the matching suspected insignificant edges as insignificant edges;

[0027] Based on the matched suspected insignificant edges, determine the optical flow of the insignificant edges, and record the optical flow of all insignificant edges as candidate tracks.

[0028] Further, the method for determining the single-view visibility according to the distortion variables in the X direction and the Y direction of all insignificant edges corresponding to all candidate tracks includes the following specific steps:

[0029] Use the distortion function to obtain all the distortion variables in the X direction and all the distortion variables in the Y direction of all insignificant edges;

[0030] Denote any insignificant edge corresponding to a candidate track as the target insignificant edge, and denote the ratio of the average value of the distortion variables in the X direction of the target insignificant edge to the average value of the distortion variables in the Y direction as the first ratio of the target insignificant edge; denote the insignificant edge adjacent to the target insignificant edge in the previous position in the candidate track as the previous adjacent insignificant edge, and denote the ratio of the average value of the distortion variables in the X direction of the previous adjacent insignificant edge to the average value of the distortion variables in the Y direction as the second ratio of the previous adjacent insignificant edge; denote the absolute value of the difference between the first ratio of the target insignificant edge and the second ratio of the previous adjacent insignificant edge as the distortion synchronization coefficient of the target insignificant edge.

[0031] Determine the single-view visibility according to the distortion synchronization coefficients of all insignificant edges of the candidate tracks screened from all left-view depth images and right-view depth images.

[0032] Further, the method for determining the single-view visibility according to the distortion synchronization coefficients of all insignificant edges of the candidate tracks screened from all left-view depth images and right-view depth images includes the following specific steps:

[0033] Denote the mean value of the distortion synchronization coefficients of all insignificant edges of the candidate tracks screened from all left-view depth images as the left-view distortion synchronization mean value;

[0034] Denote the mean value of the distortion synchronization coefficients of all insignificant edges of the candidate tracks screened from all right-view depth images as the right-view distortion synchronization mean value;

[0035] Denote the normalized value of the mean of the left-view distortion synchronization mean value and the right-view distortion synchronization mean value as the single-view visibility.

[0036] Further, the method for realizing autonomous obstacle avoidance of a fixed-wing UAV according to the single-view visibility includes the following specific steps:

[0037] When the single-view visibility is greater than the first preset threshold, mark the object corresponding to the insignificant edge as an insignificant obstacle, and synchronously mark the insignificant obstacle in the fused depth map;

[0038] Mark the unobvious obstacles in the fused depth map on the flight route navigation map, generate an obstacle avoidance path according to the flight route navigation map, and realize the autonomous obstacle avoidance of the fixed-wing UAV.

[0039] In a second aspect, an embodiment of the present invention further provides a fixed-wing UAV autonomous obstacle avoidance system, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of the method described in any one of the above are realized.

[0040] The beneficial effects of the present invention are:

[0041] According to all the lidar point cloud data included in the fragment information in this application, the fragment information corresponding to the obstacles that are getting closer to the fixed-wing UAV during the flight of the fixed-wing UAV is determined, that is, the approaching obstacles of the flight path are obtained, and the integrity and credibility of the information provided by the approaching obstacles of the flight path are evaluated. The measurable change rate of the approaching obstacles of the flight path is obtained. According to the attitude variables at all acquisition moments during the flight of the fixed-wing UAV, the possibility of the approaching obstacles of the flight path on the flight path of the fixed-wing UAV is evaluated, and the orthogonality of the approaching obstacles of the flight path is determined; Since the obstacle texture information of unobvious obstacles such as obstacles with small and variable reflection areas and dynamic obstacles is relatively fuzzy, in order to accurately avoid obstacles for the fixed-wing UAV, the obstacle texture information of these obvious obstacles is accurately extracted. First, according to the orthogonality of the approaching obstacles of the flight path in the target depth image, the low threshold and high threshold of the Canny edge detection algorithm for edge recognition of the approaching obstacles of the flight path are determined, the discrete edge lines on the approaching obstacles of the flight path are more accurately recognized, the flight path of the approaching obstacles of the flight path is more carefully captured, and the candidate flight paths formed by the unobvious edges are determined; Further, evaluate the synchronism of the distortion of the cameras of the binocular vision system, evaluate the possibility of random loss of information in the depth image of a single view corresponding to the candidate flight path, and further determine the single-view visibility. The single-view visibility is the possibility evaluation of unobvious obstacles appearing in the flight trajectory when the fixed-wing UAV is flying; Autonomous obstacle avoidance of the fixed-wing UAV is realized according to the single-view visibility, the problem that the fixed-wing UAV has insufficient recognition accuracy for unobvious obstacles encountered during flight and there is an obstacle avoidance risk is solved, and the accuracy of obstacle avoidance during the flight of the fixed-wing UAV is improved. Description of the Drawings

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 Schematic flowchart of the fixed-wing UAV autonomous obstacle avoidance method provided by an embodiment of the present invention;

[0044] Figure 2 Flowchart for obtaining measurable change rate provided by an embodiment of the present invention. Detailed implementation manners

[0045] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0046] Please refer to Figure 1 , which shows the flowchart of the fixed-wing UAV autonomous obstacle avoidance method provided by an embodiment of the present invention. The method includes the following steps:

[0047] Step S001, extract the preset flight path of the fixed-wing UAV, and collect the left-view depth image, right-view depth image, fused depth image, and all fragment information at each acquisition moment during the flight of the fixed-wing UAV. The fragment information contains different lidar point cloud data.

[0048] Extract the preset flight path of the fixed-wing UAV. Install a SLAM lidar system and a binocular vision system on the fixed-wing UAV. Start the fixed-wing UAV to fly according to the preset flight path. At each acquisition moment during the flight of the fixed-wing UAV, use the SLAM lidar system to collect all fragment information, and at the same time, use the binocular vision system to collect the left-view depth image, right-view depth image, and fused depth image.

[0049] Among them, the SLAM lidar system can emit laser pulses. After the laser pulses encounter obstacles and return, the SLAM lidar system receives the returned signals to obtain all fragment information around the position of the fixed-wing UAV. Each obstacle corresponds to one fragment information, and one fragment information contains multiple lidar point cloud data. Since the laser pulses have a certain divergence angle, the farther the distance from the fixed-wing UAV, the sparser the emission density of the lidar, and the smaller obstacles in all environments may not be covered in the obtained fragment information. For example, branches protruding above the tree crown, or small obstacles such as wires and vines.

[0050] Among them, the binocular vision system includes two cameras. The two cameras are set at different positions and are parallel to each other. They can respectively obtain the left-view depth image and the right-view depth image of the fixed-wing unmanned aerial vehicle (UAV). The binocular vision system can obtain the fused depth map by fusing the left-view depth image and the right-view depth image collected at the same moment. The left-view depth image, the right-view depth image, and the fused depth map are all depth maps. The position of the pixel point in the depth map represents the horizontal and vertical positions in space of the position of the obstacle corresponding to the pixel point. The pixel value of the pixel point is the distance between the position of the obstacle corresponding to the pixel point and the binocular vision system. Therefore, the left-view depth image, the right-view depth image, and the fused depth map can all provide the spatial information of the obstacles around the fixed-wing UAV. Further, the left-view depth image, the right-view depth image, and the fused depth map can provide information about the smaller obstacles in the environment. However, the obstacle information provided in these three depth maps is sticky. For example, the distances between the leaves at different positions of a tree and the fixed-wing UAV are different, and the edges of the leaves are irregular, making it difficult to determine the true boundaries of different leaves.

[0051] Therefore, according to the fragment information, the left-view depth image, the right-view depth image, and the fused depth map, the true boundaries of the small obstacles and the distance information between the obstacles and the UAV can be restored, improving the obstacle avoidance efficiency of the fixed-wing UAV and enhancing its flexibility.

[0052] Preferably, in an embodiment of the present application, when collecting the fragment information, the left-view depth image, the right-view depth image, and the fused depth map, data collection starts from the start of the fixed-wing UAV until the fixed-wing UAV completes landing and stops data collection. In this embodiment, the time interval between adjacent collection times is 0.01 seconds. In the actual application process, as other implementation manners, the implementer can determine the time interval between adjacent collection times according to the actual situation by himself / herself, and the present application does not make special restrictions.

[0053] So far, the preset flight path of the fixed-wing UAV, as well as the left-view depth image, the right-view depth image, the fused depth map, and all fragment information at each collection time during the flight of the fixed-wing UAV are obtained.

[0054] Step S002: Cluster all the lidar point cloud data contained in the fragment information to obtain the clustering clusters of the fragment information. Based on all the lidar point cloud data within the clustering clusters of the corresponding fragment information collected at two adjacent acquisition times, determine whether the fragment information is a trajectory approaching obstacle. If so, obtain the measurable change rate of the trajectory approaching obstacle at each acquisition time. Based on the preset flight path of the fixed-wing UAV, obtain the attitude variables at all acquisition times. Determine the orthogonality of the trajectory approaching obstacle based on the differences between the attitude variables at all consecutive acquisition times of the trajectory approaching obstacle and the differences between the measurable change rates.

[0055] Since there are multiple different lidar point cloud data in the fragment information, take the distance between the lidar point cloud data as the distance between two different lidar point cloud data, cluster all the lidar point cloud data contained in the same fragment information to obtain the clustering clusters of the same fragment information. At the same time, obtain the density of each lidar point cloud data within the clustering cluster.

[0056] It can be understood that each obstacle corresponds to a fragment information, and each reflecting surface of an obstacle corresponds to a clustering cluster of a fragment information.

[0057] Among them, in this embodiment, the DBSCAN clustering algorithm is used to cluster the lidar point cloud data. In the actual application process, on the basis of achieving the purpose of clustering, implementers can adopt other existing technologies such as the Mean Shift clustering algorithm, OPTICS clustering algorithm, Gaussian Mixture Models clustering algorithm, Spectral Clustering, etc. for clustering, and this application does not make special restrictions.

[0058] Denote the mean value of the densities of all lidar point cloud data within all clustering clusters of the fragment information as the density mean of the fragment information; denote the variance of all lidar point cloud data within all clustering clusters of the fragment information as the data variance of the fragment information; denote the product of the number of all lidar point cloud data contained in the fragment information and the density mean of the fragment information as the first product of the fragment information; denote the ratio of the first product of the fragment information to the data variance of the fragment information as the measurable degree of the fragment information.

[0059] Denote any acquisition time as the target acquisition time, and denote the previous adjacent acquisition time of the target acquisition time as the previous adjacent time. When the measurable degree of the fragment information collected at the target acquisition time is greater than the measurable degree of the corresponding fragment information collected at the previous adjacent time, denote the fragment information collected at the target acquisition time as a trajectory approaching obstacle.

[0060] When the measurability of the fragment information collected at the target acquisition moment is less than or equal to the measurability of the corresponding fragment information collected at the previous adjacent moment, the fragment information is not an obstacle approaching the flight path, and subsequent analysis of the fragment information that is not an obstacle approaching the flight path is no longer performed.

[0061] It can be understood that there is no previous adjacent acquisition moment for the first acquisition moment. Therefore, the fragment information collected at the first acquisition moment is not analyzed, that is, obstacles approaching the flight path are not screened from the fragment information collected at the first acquisition moment.

[0062] An obstacle approaching the flight path is the fragment information corresponding to an obstacle whose distance from the fixed-wing UAV becomes closer during the flight of the fixed-wing UAV.

[0063] The normalized value of the difference between the measurability of the obstacle approaching the flight path and the measurability at the previous adjacent acquisition moment corresponding to the acquisition moment of the obstacle approaching the flight path is denoted as the measurable change rate of the obstacle approaching the flight path.

[0064] It should be noted that in this embodiment, the Z-Score standard normalization method is used to calculate the normalized value. In the actual application process, the implementer can use other methods of existing technologies such as the maximum-minimum normalization method, the sigmoid function, etc. to calculate the normalized value, which is not limited here.

[0065] When the measurable change rate of the obstacle approaching the flight path is larger, at the acquisition moment corresponding to the obstacle approaching the flight path, the information provided by the obstacle approaching the flight path is more complete and credible.

[0066] The flowchart for obtaining the measurable change rate is as Figure 2 shown.

[0067] For obstacles with a small and variable reflection area or dynamic obstacles, the corresponding fragment information often cannot be completely captured by the SLAM lidar at a certain moment. When the fixed-wing UAV continues to fly, due to the change in the distance and divergence angle between the fixed-wing UAV and the obstacle, the corresponding fragment information may be completely captured by the SLAM lidar at a certain moment. Therefore, the flight path of the fragment information corresponding to the obstacle with a small and variable reflection area and the dynamic obstacle in time may be discontinuous, showing a discontinuous state.

[0068] According to the preset flight path of the fixed-wing UAV, obtain the roll angle, pitch angle, yaw angle, and rotation direction of the fixed-wing UAV at each acquisition moment, and arrange the roll angle, pitch angle, yaw angle, and rotation direction at the same acquisition moment in sequence to obtain the rotation vector at the same acquisition moment. Denote the vector difference between the rotation vectors at the target acquisition moment and the previous adjacent moment as the attitude variable at the target acquisition moment.

[0069] It can be understood that one acquisition moment corresponds to one attitude variable; one acquisition moment may correspond to multiple tracks approaching obstacles, and each track approaching an obstacle corresponds to a measurable change rate at one acquisition moment, and each track approaching an obstacle may correspond to different measurable change rates at different acquisition moments.

[0070] When the measurable change rates of all tracks approaching obstacles at the same acquisition moment are more stable and the attitude variables of the fixed-wing UAV at the same acquisition moment are more complex, there are more obstacles on the flight path of the fixed-wing UAV among the tracks approaching obstacles at this acquisition moment, and the information of the tracks approaching obstacles collected during the flight of the fixed-wing UAV is more real. The tracks approaching obstacles on the flight path of the fixed-wing UAV are recorded as tracks approaching obstacles with orthogonality.

[0071] Determine the orthogonality degree of the track approaching the obstacle according to the differences between the attitude variables of the track approaching the obstacle at all consecutive acquisition moments and the differences between the measurable change rates.

[0072] The information entropy of the modulus lengths of the attitude variables of the same track approaching an obstacle at all consecutive acquisition moments is denoted as the attitude complexity of the same track approaching an obstacle; the variance of the measurable change rates of the same track approaching an obstacle at all consecutive acquisition moments is denoted as the obstacle complexity of the same track approaching an obstacle; the normalized value of the ratio of the attitude complexity to the obstacle complexity of the same track approaching an obstacle is denoted as the orthogonality degree of the same track approaching an obstacle.

[0073] It should be noted that in this embodiment, the Z-Score standard normalization method is used to calculate the normalized value. In the actual application process, implementers can use other methods of existing technologies such as the maximum-minimum normalization method, sigmoid function, etc. to calculate the normalized value, which is not limited here.

[0074] The orthogonality degree of the track approaching an obstacle is used to evaluate the possibility of the track approaching an obstacle on the flight path of the fixed-wing UAV. When the possibility of the track approaching an obstacle on the flight path of the fixed-wing UAV is greater, the orthogonality degree of the track approaching an obstacle is greater.

[0075] So far, the orthogonality degrees of all tracks approaching obstacles are obtained.

[0076] Step S003, determine the edge images of the left-view depth image and the right-view depth image according to the orthogonality degrees of the tracks approaching obstacles in the left-view depth image and the right-view depth image, match the edges in all the edge images, screen out the insignificant edges and determine the candidate tracks formed by the insignificant edges, and determine the single-view visibility according to the distortion variables in the X direction and the Y direction of all the insignificant edges corresponding to all the candidate tracks.

[0077] A large amount of obstacle texture information is provided in the left-view depth image, the right-view depth image, and the fused depth map. However, the obstacle texture information of insignificant obstacles such as obstacles with small and variable reflection areas and dynamic obstacles is relatively blurred. In order to accurately avoid obstacles for a fixed-wing UAV, it is necessary to extract the obstacle texture information of these significant obstacles.

[0078] Denote any one of the left-view depth image or the right-view depth image as the target depth image. Process the target depth image using the Otsu method to obtain the segmentation threshold x. Use the Canny edge detection algorithm to perform edge recognition on the target depth image respectively to obtain the edge image corresponding to the target depth image. Among them, the low threshold for the track approaching obstacle i in the target depth image is set to The high threshold is set to ω i is the orthogonality of the track approaching obstacle i in the target depth image.

[0079] The low threshold and the high threshold are two thresholds when the Canny edge detection algorithm determines the edge, which are well-known definitions and will not be elaborated here.

[0080] The edge images of all left-view depth images and right-view depth images can be obtained in the same way.

[0081] According to the orthogonality of the track approaching the obstacle in the target depth image, determine the low threshold and the high threshold for the Canny edge detection algorithm to perform edge recognition on the track approaching the obstacle, which can more accurately identify the discrete edge lines on the track approaching the obstacle and more carefully capture the track of the track approaching the obstacle.

[0082] Use the NCC (Normalized Cross-Correlation) matching algorithm to match the edges in the edge images at adjacent acquisition times to obtain the matched edges. Denote the edges that are matched in three or more consecutive acquisition times as significant edges, and denote the edges that are not significant edges as suspected insignificant edges. Use the NCC matching algorithm to match all suspected insignificant edges at different acquisition times, and denote the matched suspected insignificant edges as insignificant edges.

[0083] Among them, the significant edges correspond to the edges of relatively significant obstacles identified during the flight of the fixed-wing robot.

[0084] According to the matched suspected insignificant edges, determine the optical flow of the insignificant edges, and denote the optical flows of all insignificant edges as candidate tracks.

[0085] So far, the candidate tracks that can be screened out from all left-view depth images and right-view depth images are obtained.

[0086] To avoid the influence of the distortion of the cameras in the binocular vision system on the obstacle avoidance calculation of the fixed-wing UAV, the distortion function is used to obtain all the distortion amounts in the X direction and all the distortion amounts in the Y direction of all insignificant edges, and the average value of the distortion amounts in the X direction and the average value of the distortion amounts in the Y direction are calculated respectively.

[0087] Among them, using the distortion function to obtain the sum of the distortion amounts in the X direction and the distortion amounts in the Y direction of all insignificant edges, and calculating the average value of the distortion amounts are all well-known technologies and will not be elaborated here.

[0088] According to the distortion amounts in the X direction and the distortion amounts in the Y direction of the insignificant edges, the distortion synchronization coefficient of the insignificant edges is determined.

[0089] Any insignificant edge corresponding to the candidate flight path is denoted as the target insignificant edge, and the ratio of the average value of the distortion amounts in the X direction of the target insignificant edge to the average value of the distortion amounts in the Y direction is denoted as the first ratio of the target insignificant edge; the insignificant edge adjacent to the target insignificant edge in the previous position in the candidate flight path is denoted as the previous adjacent insignificant edge, and the ratio of the average value of the distortion amounts in the X direction of the previous adjacent insignificant edge to the average value of the distortion amounts in the Y direction is denoted as the second ratio of the previous adjacent insignificant edge; the absolute value of the difference between the first ratio of the target insignificant edge and the second ratio of the previous adjacent insignificant edge is denoted as the distortion synchronization coefficient of the target insignificant edge.

[0090] The distortion synchronization coefficients of all the insignificant edges corresponding to the candidate flight path can be obtained by the same method. It can be understood that the first insignificant edge corresponding to the candidate flight path has no previous adjacent insignificant edge, so the first insignificant edge corresponding to the candidate flight path is not analyzed, that is, the distortion synchronization coefficient of the first insignificant edge corresponding to the candidate flight path is not obtained.

[0091] During the process of calculating the ratio, in order to avoid the situation where the denominator is zero, a preset value needs to be added to the denominator, and the value of the preset value in the embodiment is 0.01.

[0092] When the distortion in the X direction and the distortion in the Y direction of the adjacent insignificant edges corresponding to the candidate flight path are more asynchronous, the greater the possibility of random loss of information in the single-view depth image corresponding to the candidate flight path. At this time, the distortion synchronization coefficient of the insignificant edge is greater.

[0093] According to the distortion synchronization coefficients of all the insignificant edges of the candidate flight paths screened from all the left-view depth images and right-view depth images, the single-view visibility is determined.

[0094] The mean value of the distortion synchronization coefficients of all insignificant edges of the candidate flight tracks selected from all left-view depth images is denoted as the left-view distortion synchronization mean value; the mean value of the distortion synchronization coefficients of all insignificant edges of the candidate flight tracks selected from all right-view depth images is denoted as the right-view distortion synchronization mean value; the normalized value of the mean of the left-view distortion synchronization mean value and the right-view distortion synchronization mean value is denoted as the single-view visibility.

[0095] It should be noted that in this embodiment, the Z-Score standard normalization method is used to calculate the normalized value. In the actual application process, implementers can use other methods of existing technologies such as the maximum-minimum normalization method, sigmoid function, etc. to calculate the normalized value, which is not limited here.

[0096] When the single-view visibility is larger, when the fixed-wing UAV is flying, the possibility of an insignificant obstacle appearing in the flight trajectory is greater, and obstacle avoidance is more required.

[0097] Thus, the single-view visibility is obtained.

[0098] Step S004, implement autonomous obstacle avoidance for the fixed-wing UAV according to the single-view visibility.

[0099] When the single-view visibility is greater than the first preset threshold, the object corresponding to the insignificant edge is marked as an insignificant obstacle. And the insignificant obstacle is synchronously marked in the fused depth map.

[0100] Among them, the value of the first preset threshold in this embodiment is 0.6.

[0101] Mark the insignificant obstacles in the fused depth map on the flight route navigation map, generate an obstacle avoidance path according to the flight route navigation map, implement autonomous obstacle avoidance for the fixed-wing UAV, and ensure the flight safety of the fixed-wing UAV.

[0102] Thus, autonomous obstacle avoidance for the fixed-wing UAV is realized.

[0103] Based on the same inventive concept as the above method, an embodiment of the present invention also provides a fixed-wing UAV autonomous obstacle avoidance system, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above fixed-wing UAV autonomous obstacle avoidance methods.

[0104] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for autonomous obstacle avoidance of a fixed-wing UAV, characterized in that: The method comprises the following steps: Extract the preset route of the fixed-wing UAV, collect the left-view depth image, right-view depth image, fused depth map and all fragment information at each collection moment during the flight of the fixed-wing UAV, and the fragment information contains different laser radar point cloud data; Cluster all the laser radar point cloud data contained in the fragment information to obtain the clustering cluster of the fragment information, and judge whether the fragment information is a track approaching obstacle based on all the laser radar point cloud data in the clustering cluster of the corresponding fragment information collected at two adjacent collection moments. If so, obtain the measurable change rate of the track approaching obstacle at each collection moment, and obtain the attitude variables at all collection moments based on the preset route of the fixed-wing UAV. According to the difference between the attitude variables of the track approaching obstacle at all consecutive collection moments and the difference between the measurable change rates, determine the orthogonality of the track approaching obstacle; According to the degree of orthogonality of the tracks approaching obstacles in the left-view depth image and the right-view depth image, the edge images of the left-view depth image and the right-view depth image are determined, the edges in all edge images are matched, the insignificant edges are screened and the candidate tracks formed by the insignificant edges are determined, and the single-view visibility is determined according to the distortion amounts in the X direction and the Y direction of all the insignificant edges corresponding to all the candidate tracks; Autonomous obstacle avoidance for fixed-wing UAVs based on single-view visibility.

2. The autonomous obstacle avoidance method for a fixed-wing UAV according to claim 1, characterized in that: The method of judging whether the fragment information is a track approaching obstacle based on all the laser radar point cloud data in the cluster of the corresponding fragment information collected at two adjacent collection moments, and if so, obtaining the measurable change rate of the track approaching obstacle at each collection moment includes the following specific methods: The mean of the density of all the laser radar point cloud data in all the clusters of the fragment information is recorded as the density mean of the fragment information; the variance of all the laser radar point cloud data in all the clusters of the fragment information is recorded as the data variance of the fragment information; the product of the number of all the laser radar point cloud data contained in the fragment information and the density mean of the fragment information is recorded as the first product of the fragment information, and the ratio of the first product of the fragment information to the data variance of the fragment information is recorded as the measurability of the fragment information; Any collection time is recorded as the target collection time, and the collection time immediately before the target collection time is recorded as the previous adjacent time. When the measurability of the fragment information collected at the target collection time is greater than the measurability of the corresponding fragment information collected at the previous adjacent time, the fragment information collected at the target collection time is recorded as the track approaching obstacle. For each track approaching obstacle, the normalized value of the difference between the measurability of the track approaching obstacle at the target collection time and the measurability of the track approaching obstacle at the previous adjacent time is recorded as the measurable change rate of the track approaching obstacle at the target collection time. When the measurability of the fragment information collected at the target collection time is less than or equal to the measurability of the corresponding fragment information collected at the previous adjacent time, the fragment information is not a track approaching obstacle.

3. The fixed-wing UAV autonomous obstacle avoidance method according to claim 2, characterized in that: The specific method of obtaining the attitude variables at all collection moments according to the preset route of the fixed-wing UAV is as follows: According to the preset route of the fixed-wing UAV, the roll angle, pitch angle, yaw angle and rotation direction of the fixed-wing UAV at each collection moment are obtained; Arrange the roll angle, pitch angle, yaw angle and rotation direction at the same acquisition moment in sequence to obtain the rotation vector at the same acquisition moment; The vector difference between the rotation vector at the target acquisition moment and the previous adjacent moment is recorded as the posture variable at the target acquisition moment.

4. The autonomous obstacle avoidance method for a fixed-wing UAV according to claim 1, characterized in that: The method for determining the degree of orthogonality of the track approaching an obstacle is: The information entropy of the modulus of the attitude variables of the same track approaching the obstacle at all consecutive acquisition moments is recorded as the attitude complexity of the same track approaching the obstacle; The variance of the measurable change rate of the same track approaching the obstacle at all consecutive acquisition moments is recorded as the obstacle complexity of the same track approaching the obstacle; The normalized value of the ratio of the posture complexity of the same track approaching the obstacle to the obstacle complexity is recorded as the orthogonality degree of the same track approaching the obstacle.

5. The fixed-wing UAV autonomous obstacle avoidance method according to claim 1, characterized in that: The method for determining the edge images of the left-view depth image and the right-view depth image is as follows: Any left-view depth image or right-view depth image is recorded as the target depth image, and the target depth image is processed using the maximum inter-class variance method to obtain the division threshold x. The Canny edge detection algorithm is used to identify the edges of the target depth image and obtain the edge image corresponding to the target depth image. The low threshold of the track approaching obstacle i in the target depth image is set to The high threshold is set to ω i is the orthogonality of the track approaching obstacle i in the target depth image.

6. The autonomous obstacle avoidance method for a fixed-wing UAV according to claim 1, characterized in that: The specific method of matching the edges in all edge images, screening the insignificant edges and determining the candidate tracks formed by the insignificant edges is as follows: Match the edges in the edge images at adjacent acquisition moments to obtain matched edges, record the edges that are matched at more than or equal to 3 consecutive acquisition moments as significant edges, and record the edges that are not significant edges as suspected insignificant edges; Match all suspected insignificant edges at different acquisition times, and record the matched suspected insignificant edges as insignificant edges; According to the matched suspected insignificant edges, the optical flow of the insignificant edges is determined, and the optical flows of all insignificant edges are recorded as candidate tracks.

7. The autonomous obstacle avoidance method for a fixed-wing UAV according to claim 1, characterized in that: The method of determining the single-view visibility according to the distortion amounts in the X direction and the Y direction of all insignificant edges corresponding to all the selected tracks includes: Use the distortion function to obtain all distortion values ​​in the X direction and all distortion values ​​in the Y direction for all insignificant edges; Any insignificant edge corresponding to the candidate track is recorded as the target insignificant edge, and the ratio of the average value of the distortion amount of the target insignificant edge in the X direction to the average value of the distortion amount in the Y direction is recorded as the first ratio of the target insignificant edge; the previous adjacent insignificant edge of the target insignificant edge in the candidate track is recorded as the previous adjacent insignificant edge, and the ratio of the average value of the distortion amount of the previous adjacent insignificant edge in the X direction to the average value of the distortion amount in the Y direction is recorded as the second ratio of the previous adjacent insignificant edge; the absolute value of the difference between the first ratio of the target insignificant edge and the second ratio of the previous adjacent insignificant edge is recorded as the distortion synchronization coefficient of the target insignificant edge; The single-view visibility is determined according to the distortion synchronization coefficients of all insignificant edges of the candidate tracks screened out from all left-view depth images and right-view depth images.

8. The fixed-wing UAV autonomous obstacle avoidance method according to claim 7, characterized in that: The method of determining the single-view visibility according to the distortion synchronization coefficients of all insignificant edges of the selected track screened out from all left-view depth images and right-view depth images includes: The mean of the distortion synchronization coefficients of all the insignificant edges of the candidate tracks screened out from all left-view depth images is recorded as the left-view distortion synchronization mean; The mean value of the distortion synchronization coefficients of all insignificant edges of the candidate tracks screened out from all right-view depth images is recorded as the right-view distortion synchronization mean value; The normalized value of the mean of the left view distortion synchronization mean and the right view distortion synchronization mean is recorded as the single view visibility.

9. The fixed-wing UAV autonomous obstacle avoidance method according to claim 1, characterized in that: The specific method of realizing autonomous obstacle avoidance of a fixed-wing UAV based on single-view visibility is as follows: When the single-view visibility is greater than a first preset threshold, marking the object corresponding to the inconspicuous edge as an inconspicuous obstacle, and simultaneously marking the inconspicuous obstacle in the fused depth map; The unclear obstacles in the fused depth map are marked on the flight route navigation map, and an obstacle avoidance path is generated according to the flight route navigation map to realize autonomous obstacle avoidance of the fixed-wing UAV.

10. An autonomous obstacle avoidance system for a fixed-wing unmanned aerial vehicle, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

Images