Multi-sensor fusion-based unmanned aerial vehicle autonomous obstacle avoidance and path planning system and method

Abstract

The application discloses a multi-sensor fusion unmanned aerial vehicle autonomous obstacle avoidance and path planning system and method, and belongs to the technical field of unmanned aerial vehicle obstacle avoidance. The technical scheme points of the application comprise the following steps: acquiring dynamic obstacles and linear obstacles in a scanning area corresponding to a current time; performing normalization processing on the features of the dynamic obstacles and the linear obstacles, and correcting the features after the normalization processing to obtain obstacle scores; obtaining a current obstacle avoidance priority according to the obstacle scores; and updating a current path according to the obstacle avoidance priority to obtain a first path. The application corrects the reflectivity threshold value through the reflectivity detected by each sub-region, more accurately identifies the linear obstacles through the comprehensive texture feature value and the contour feature value, and calculates the obstacle avoidance priority according to the features of the obstacles. According to different priorities, different obstacle avoidance modes are executed, so that the original inspection path can be returned in time while the obstacle is avoided quickly.

Description

Technical Field

[0001] This invention relates to the field of drone obstacle avoidance technology, and more specifically to a multi-sensor fusion drone autonomous obstacle avoidance and path planning system and method. Background Technology

[0002] With the large-scale application of drone technology in fields such as power line inspection, geographic surveying and mapping, and emergency rescue, its operating environment is gradually changing from open field scenes to complex and obstacle-filled scenes. Among them, linear obstacles (such as high and low voltage lines and communication cables) have become one of the most likely sources of collision accidents due to their slender shape and easy confusion with the background.

[0003] Existing technologies typically use fixed thresholds or contour checks for identification, but these methods have a high false positive rate, resulting in varying perception accuracy for linear obstacles. Furthermore, existing technologies employ a single obstacle avoidance strategy, making it difficult to revert to the initial inspection path after obstacle avoidance. Therefore, existing technologies have shortcomings. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide a multi-sensor fusion-based autonomous obstacle avoidance and path planning system and method for unmanned aerial vehicles (UAVs). This system updates the reflectivity threshold by updating the background reflectivity and enhances the perception of linear obstacles by combining comprehensive texture feature values ​​and the contour feature values. It also determines the obstacle avoidance priority based on the characteristics of different obstacles, selects different obstacle avoidance strategies for different obstacle avoidance priorities, and ensures that the system can return to the initial inspection route as soon as possible after executing the obstacle avoidance steps.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] This invention provides a multi-sensor fusion-based autonomous obstacle avoidance and path planning method for unmanned aerial vehicles (UAVs), comprising:

[0007] Obtain dynamic and linear obstacles within the scanned area at the current moment;

[0008] The features of the dynamic obstacles and linear obstacles are normalized, and the normalized features are corrected to obtain the obstacle score.

[0009] The current obstacle avoidance priority is determined based on the obstacle score.

[0010] The current path is updated based on the obstacle avoidance priority to obtain the first path.

[0011] As a further improvement of the present invention, the scanning area includes multiple sub-regions, and the linear obstacles within the scanning area corresponding to the current moment are acquired, including:

[0012] Obtain the background reflectance and the object reflectance corresponding to each sub-region;

[0013] The reflectance threshold is corrected based on the background reflectance.

[0014] Based on the corrected reflectivity threshold and the object's reflectivity, linear obstacles are obtained.

[0015] As a further improvement of the present invention, the normalized features are modified to obtain the obstacle score, including:

[0016] Acquire the spectral image corresponding to the linear obstacle;

[0017] Texture uniformity, orientation consistency, and contrast are obtained from the spectral image;

[0018] Based on the texture uniformity, orientation consistency, and contrast, a comprehensive texture feature value is obtained;

[0019] Based on the comprehensive texture feature values ​​and visual image, the normalized features are corrected to obtain the obstacle score.

[0020] As a further improvement of the present invention, based on the comprehensive texture feature values ​​and the visual image, the normalized features are corrected to obtain the obstacle score, including:

[0021] Based on the comprehensive texture feature values, the reflectivity after normalization is corrected;

[0022] A linear contour is obtained based on the visual image;

[0023] Based on the fitting line error corresponding to the sub-line segment in the linear contour and the included angle corresponding to the sub-line segment, the contour feature value is obtained;

[0024] An obstacle score is obtained based on the contour feature values ​​and the corrected reflectivity.

[0025] As a further improvement of the present invention, the current path is updated according to the obstacle avoidance priority to obtain a first path, including:

[0026] Determine whether the characteristics of the dynamic obstacles and linear obstacles meet at least one preset condition; the preset condition includes feature conditions and multi-obstacle interaction conditions;

[0027] If so, based on the characteristics of the dynamic obstacles and linear obstacles, update the current path to obtain the second path;

[0028] The second path is updated according to the obstacle avoidance priority to obtain the first path.

[0029] As a further improvement of the present invention, the second path is updated according to the obstacle avoidance priority to obtain the first path, including:

[0030] If the current obstacle avoidance priority is high, the second path is updated based on the characteristics of the dynamic obstacle and the linear obstacle to obtain the first path.

[0031] If the current obstacle avoidance priority is medium or low, potential obstacle avoidance nodes are determined based on the positions of the dynamic obstacles and linear obstacles, and a first path is obtained based on the potential obstacle avoidance nodes and the RBF neural network model.

[0032] As a further improvement of the present invention, the second path is updated based on the characteristics of the dynamic obstacle and the linear obstacle to obtain the first path, including:

[0033] Based on the characteristics of the dynamic obstacles and linear obstacles, the opposite action vector is obtained;

[0034] The second path is updated based on the opposite action vector and the Bézier curve to obtain the first path.

[0035] As a further improvement of the present invention, a first path is obtained based on the potential obstacle avoidance nodes and the RBF neural network model, including:

[0036] A grid is generated based on the next adjacent inspection node, and the grid includes multiple spatial units;

[0037] The node spacing and initial control nodes are obtained based on the spatial unit and the RBF neural network model;

[0038] The control node sequence is obtained based on the climb rate between two adjacent nodes in the initial control node;

[0039] The control node sequence is updated based on the potential obstacle avoidance nodes to obtain the first path.

[0040] As a further improvement of the present invention, if the current obstacle avoidance priority is medium or low, and there are multiple drones, after obtaining the first path, the drone autonomous obstacle avoidance and path planning method further includes:

[0041] For each UAV, the corresponding policy set is obtained based on its updated control node sequence;

[0042] Based on the task completion rate, energy consumption, and conflict risk of each drone, the delivery matrix corresponding to the strategy set is obtained;

[0043] A strategy combination is obtained based on the delivery matrix and the Nash equilibrium strategy;

[0044] Update the first path according to the strategy combination.

[0045] This invention provides a multi-sensor fusion-based autonomous obstacle avoidance and path planning system for unmanned aerial vehicles (UAVs), comprising:

[0046] The acquisition module is used to acquire dynamic obstacles and linear obstacles within the scanning area at the current moment;

[0047] The correction module normalizes the features of the dynamic obstacles and linear obstacles, and corrects the normalized features to obtain the obstacle score.

[0048] A buffer module is used to obtain the current obstacle avoidance priority based on the obstacle score;

[0049] The path planning module is used to update the current path based on the obstacle avoidance priority.

[0050] This invention updates the reflectivity threshold by background reflectivity to determine whether there is an obstacle. Then, it modifies the characteristics of the obstacle by combining the texture feature value and the contour feature value to enhance the perception of linear obstacles, thereby obtaining a more accurate obstacle avoidance priority. Finally, for different obstacle avoidance priorities, different obstacle avoidance strategies are selected to update the current inspection path, and it is ensured that the initial inspection route can be returned as soon as possible after the obstacle avoidance is performed. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the method steps of the present invention;

[0052] Figure 2 This is a schematic diagram of the drone's scanning range;

[0053] Figure 3 This is a schematic diagram of the first path;

[0054] Figure 4 A schematic diagram of the steps to obtain the first path. Detailed Implementation

[0055] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof.

[0056] Identical parts are indicated by the same reference numerals. It should be noted that the terms "front," "rear," "left," "right," "up," and "down" used in the following description refer to directions in the accompanying drawings, while the terms "bottom surface," "top surface," "inner," and "outer" refer to directions toward or away from the geometric center of a specific part, respectively.

[0057] The term "and / or" in the following text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0058] like Figure 1 As shown, this application provides a multi-sensor fusion-based autonomous obstacle avoidance and path planning method for unmanned aerial vehicles (UAVs), including:

[0059] Obtain dynamic and linear obstacles within the scanned area at the current moment;

[0060] The features of dynamic obstacles and linear obstacles are normalized, and the normalized features are then corrected to obtain the obstacle score.

[0061] The current obstacle avoidance priority is determined based on the obstacle score;

[0062] The current path is updated based on obstacle avoidance priority to obtain the first path.

[0063] In this embodiment, the drone will conduct inspections according to an initial inspection route, which consists of multiple preset inspection nodes. If there is no obstacle avoidance required, the drone will arrive at the corresponding inspection node at each time point and then perform multi-angle image acquisition of the inspection object at that node. In this embodiment, each time an inspection node is reached, in addition to acquiring images or data of the inspection object, it is also necessary to collect data on the path between the current inspection node and the next adjacent inspection node through sensors.

[0064] Specifically, sensors include millimeter-wave radar, lidar, visible light vision sensors, and multispectral vision sensors, for example, such as Figure 2 As shown, when a drone needs to travel along path AB to the next adjacent inspection node, it needs to collect data on the area near path AB using sensors (such as...). Figure 2 Data within the sector range (in the image), due to Figure 2 Therefore, it is a two-dimensional image. Figure 2Only the vertical range is shown, but in practical applications, the horizontal range should also be included. Since different sensors have different scanning ranges and angles, after obtaining data from each sensor, only the data within the common field of view between the sensors is retained for subsequent analysis. The common field of view at the current moment is the corresponding scanning area. Specifically, millimeter-wave radar is used to measure the speed and direction of motion of dynamic obstacles, lidar is used to measure the distance between obstacles and the drone, as well as the reflectivity of obstacles, visual sensors are used to identify the type of obstacle (such as birds or other drones), and multispectral visual sensors are used to identify the texture and contours of obstacles.

[0065] Specifically, data is first collected by LiDAR, visible light vision sensor, and multispectral vision sensor. Initially, a preliminary linear obstacle region is obtained based on the image data collected by the visible light vision sensor. This embodiment does not limit the steps for identifying the preliminary linear obstacle region. For example, based on the grayscale values ​​of pixels in the image data, the region containing pixels with similar grayscale values ​​arranged in a line can be considered as the preliminary linear obstacle region. Then, combined with the data collected by LiDAR within this preliminary linear obstacle region, the reflectivity and point cloud density corresponding to the preliminary linear obstacle region can be obtained. Subsequently, millimeter-wave radar and visible light vision sensor collect data. Since the inspection object in this embodiment is fixed, as long as the millimeter-wave radar identifies an object in motion, it is considered a dynamic obstacle. Simultaneously, the millimeter-wave radar can measure the distance between the dynamic obstacle and the drone, and based on the data from the visible light vision sensor, the type of dynamic obstacle can be determined.

[0066] Based on the data collected by the sensors, it is possible to identify whether there are dynamic or linear obstacles when heading to the next adjacent inspection node. Then, the characteristics of the dynamic or linear obstacles are normalized to eliminate the influence between dimensions. Based on the normalized characteristics, the current obstacle avoidance priority can be determined, and an appropriate obstacle avoidance strategy can be selected according to the obstacle avoidance priority to update the current inspection path AB, thus obtaining a new path to the next adjacent inspection node, which is the first path.

[0067] This embodiment updates the reflectivity threshold by background reflectivity to determine whether there is an obstacle. Then, it modifies the characteristics of the obstacle by combining the texture feature value and the contour feature value to enhance the perception of linear obstacles, thereby obtaining a more accurate obstacle avoidance priority. Finally, for different obstacle avoidance priorities, different obstacle avoidance strategies are selected to update the current inspection path, and it is ensured that the initial inspection route can be returned as soon as possible after the obstacle avoidance is performed.

[0068] Furthermore, this embodiment provides a step for acquiring linear obstacles within the scanned area at the current moment, including:

[0069] Obtain the background reflectance and the object reflectance corresponding to each sub-region;

[0070] The reflectance threshold is adjusted based on the background reflectance.

[0071] Based on the corrected reflectivity threshold and the object's reflectivity, linear obstacles are obtained.

[0072] The scenario used in this embodiment is mainly power line inspection. The inspection objects include key components such as high-voltage lines and insulators. During the power line inspection, unexpected events may occur, such as high-voltage lines falling. If the inspection is still carried out according to the preset initial inspection route, it may cause the drone to collide with the high-voltage line or with birds during the inspection. Therefore, the drone needs to identify dynamic obstacles and linear obstacles during the inspection and avoid them in time. Although the drone can automatically execute the obstacle avoidance strategy, the drone's flight status and real-time data still need to be monitored manually. When the human detects that the obstacle avoidance strategy is unreasonable, the human will control the drone to avoid the obstacle. Otherwise, if it is reasonable, the drone will still avoid the obstacle on its own and continue the subsequent inspection work.

[0073] Furthermore, visible light vision sensors can only identify the color, shape, and texture of two-dimensional images. Since high-voltage lines and their reflections highly overlap in these features, after identifying the initial linear obstacle area based on the texture, it is impossible to directly distinguish whether the initial linear obstacle area is a high-voltage line or its reflection. However, this embodiment found that the material of high-voltage lines is mostly aluminum or steel-cored aluminum stranded wire, so its reflection of laser light is diffuse reflection. The reflection of a high-voltage line is formed when light shines on the high-voltage line and is reflected by a mirror into the sensor. Therefore, its reflectivity is higher than that of the high-voltage line itself. Based on this, this embodiment identifies linear obstacles by the reflectivity of each initial linear obstacle area.

[0074] Specifically, the reflectance collected by the lidar sensor is the reflectance of multiple sampling points. Existing technologies typically calculate the reflectance of a specific area by averaging the reflectance of each sampling point in that area, but this method has significant errors. For example, suppose the background of a high-voltage power line in an urban or suburban area includes metal billboards or snow-covered mountains. In this case, due to the high reflectance of the background and the unavoidable presence of some background areas within the initial linear obstacle area, the calculated average value of the initial linear obstacle area is high, leading the drone to believe that no linear obstacle exists. Therefore, this embodiment further divides the scanning area into multiple sub-regions, with each initial linear obstacle area comprising multiple sub-regions. For each sub-region, the reflectance of each sub-region can be obtained based on the reflectance of each sampling point it includes, serving as the object reflectance for that sub-region. Then, the background area surrounding the initial linear obstacle area is obtained based on image data collected by the visual sensor, and the background reflectance is obtained based on the average reflectance of each sampling point within the background area. This embodiment does not limit the size of the background area surrounding the initial linear obstacle area.

[0075] Next, a reflectivity threshold is obtained. The reflectivity threshold is the reflectivity that can effectively distinguish between high-voltage lines and their reflections under most conditions. This embodiment does not limit this, and those skilled in the art can set it themselves or determine it through experiments. For example, common scene images in power inspection scenarios (such as high-voltage lines with an open plain background or high-voltage lines with ordinary buildings as the background) can be obtained. Then, the distribution of background reflectivity and object reflectivity is statistically analyzed, and the reflectivity that can distinguish at least 70% or 80% is used as the reflectivity threshold.

[0076] After obtaining the current background reflectivity, the average background reflectivity of the scene image used to generate the reflectivity threshold is obtained, and the difference between the current background reflectivity and this average is calculated. If the difference is within a preset range, the reflectivity threshold is not adjusted, and objects in sub-regions with reflectivity less than the threshold are directly treated as linear obstacles. If the difference is outside the preset range, the current background reflectivity is increased or decreased proportionally based on the difference and the reflectivity threshold. A difference outside the preset range includes both differences greater than and less than the preset range. For example, when the current background reflectivity is greater than the preset range, the sub-region... The reflectivity of objects in the domain will be increased. If detection is still performed according to the reflectivity threshold, it is easy to miss detections. Therefore, it is necessary to increase the reflectivity threshold proportionally to obtain the corrected reflectivity threshold. Then, the reflectivity of objects in each sub-region of each preliminary linear obstacle region is obtained, and objects in the sub-regions that are less than the corrected reflectivity threshold are regarded as linear obstacles. When the current background reflectivity is less than the preset range, the reflectivity of objects in the sub-region will be reduced. If detection is performed according to the reflectivity threshold, it is easy to make false detections. Therefore, it is necessary to reduce the reflectivity threshold proportionally. This embodiment does not limit the specific value of the preset range.

[0077] Furthermore, since a preliminary linear obstacle region may include multiple background regions, such as a high-voltage power line whose background includes both sky and reflective metal, with the sky having lower reflectivity and the reflective metal having higher reflectivity, the current background reflectivity is calculated based on the average reflectivity of each sampling point within the background region. Therefore, the calculated current background reflectivity will be close to the reflectivity threshold. In this case, the sub-region with sky as its background is identified as a linear obstacle, while the sub-region with reflective metal as its background is not identified. To ensure the completeness of linear obstacle identification, images from a visible light vision sensor can be combined with contour recognition to supplement the unidentified regions, resulting in a complete linear obstacle. Alternatively, instead of using the average reflectivity of each sampling point within the background region of the entire preliminary linear obstacle region when calculating the current background reflectivity, the background region within each sub-region can be calculated separately, and then the background reflectivity corresponding to each sub-region can be calculated individually. The reflectivity threshold can be corrected, and the linear obstacle identification can be performed separately for each sub-region, ensuring the completeness of the linear obstacle identification.

[0078] This embodiment utilizes the difference in reflectivity between high-voltage lines and their reflections to identify objects when two-dimensional images are insufficient. Instead, it uses reflectivity data collected by LiDAR to improve recognition accuracy. Furthermore, considering the special case where the background region affects the reflectivity of objects, the reflectivity threshold is further adjusted to further improve the accuracy of linear obstacle recognition.

[0079] Furthermore, after identifying dynamic obstacles, a score corresponding to the dynamic obstacle needs to be obtained based on the distance and speed between the dynamic obstacle and the current inspection node. For example, different weights can be set for distance and speed, and the score can be obtained by weighting. The distance and speed have been normalized. Specifically, the distance needs to be reverse normalized, that is, the smaller the distance, the larger the normalized value, while the speed needs to be forward normalized. In addition, since there is a time difference between the time when the data is collected and the time when the UAV starts to act after updating the path, the dynamic obstacle is still moving during this period. Therefore, the distance moved by the dynamic obstacle during this period needs to be considered when calculating the distance. In this embodiment, it is assumed that there is at most one dynamic obstacle or linear obstacle.

[0080] Furthermore, this embodiment provides a step for correcting the normalized features to obtain an obstacle score, including:

[0081] Acquire the spectral image corresponding to the linear obstacle;

[0082] Texture uniformity, orientation consistency, and contrast are obtained from the spectral image;

[0083] The comprehensive texture feature value is obtained based on texture uniformity, orientation consistency, and contrast.

[0084] Based on the combined texture feature values ​​and visual image, the normalized features are corrected to obtain the obstacle score.

[0085] For dynamic obstacles, the features requiring normalization are distance and velocity; for linear obstacles, the features requiring normalization are reflectivity and point cloud density. Furthermore, while reflectivity can distinguish between high-voltage power lines and their reflections, it cannot distinguish linear obstacles with similarly low reflectivity (such as tree branches). Therefore, this embodiment further distinguishes them using texture and contour feature values.

[0086] Specifically, firstly, spectral images corresponding to linear obstacles are acquired using a multispectral vision sensor. Based on these spectral images, texture uniformity, orientation consistency, and contrast are obtained for each linear obstacle. When calculating texture uniformity, a gray-level co-occurrence matrix (GLCM) can be constructed. Then, the frequency of different gray-level value combinations occurring in a specific direction for adjacent pixels is counted. The frequencies are squared and summed to obtain the texture uniformity. When calculating orientation consistency, the gradient change direction of each pixel within the region of the linear obstacle is calculated. These directions are then divided into multiple intervals, and the number of pixels in each interval is counted. The ratio of the number of pixels in each interval to the total number of pixels is calculated, and the largest ratio is taken as the orientation consistency. Contrast can also be calculated using the GLCM. For example, pixel pairs with different gray-level values ​​can be counted, and the square of their gray-level difference multiplied by the sum of their corresponding frequencies is taken as the contrast. The specific calculation methods for these features are techniques that can be implemented by those skilled in the art, and will not be elaborated upon in this embodiment. Finally, the texture uniformity, orientation consistency, and contrast are weighted and averaged to obtain a comprehensive texture feature value. This embodiment does not limit the weights.

[0087] Furthermore, this embodiment provides a step for correcting the normalized features based on the comprehensive texture feature values ​​and the visual image to obtain an obstacle score, including:

[0088] The reflectance after normalization is corrected based on the comprehensive texture feature values;

[0089] Obtain linear contours from visual images;

[0090] Based on the fitting line error corresponding to the sub-line segment in the linear profile and the included angle corresponding to the sub-line segment, the profile feature value is obtained;

[0091] Obstacle scores are obtained based on contour feature values ​​and corrected reflectivity.

[0092] Specifically, the visual image corresponding to the identified linear obstacle can be determined based on the image acquired by the visible light vision sensor. Then, contour recognition is performed to obtain the linear contour corresponding to the linear obstacle. Next, for each linear contour, it is divided into multiple sub-segments, and the pixel at the center of each sub-segment is determined. Then, the fitted line of the linear contour is obtained using the least squares method and the pixel at the center. Next, for each pixel in the linear contour corresponding to the linear obstacle, its vertical distance to the fitted line is calculated, and the maximum vertical distance is selected. This maximum vertical distance is then normalized based on historical data to obtain the fitted line error. When calculating the included angle for each sub-segment, the included angle is obtained by connecting the pixel at the center of each sub-segment with that pixel as the vertex. The proportion of sub-segments with included angles less than a preset value is recorded as a continuity index. Finally, the continuity index and the fitted line error are weighted and averaged to obtain the contour feature value corresponding to the linear obstacle. This embodiment does not impose restrictions on the weights.

[0093] Next, the texture feature value, contour feature value, and normalized reflectance and point cloud density corresponding to each linear obstacle are weighted and averaged to obtain the score corresponding to each linear obstacle. This embodiment does not limit the specific value of the weights, but since low reflectance is the core physical characteristic of high-voltage lines, it can effectively distinguish high-voltage lines from reflections. Moreover, the reflectance data of LiDAR is less affected by the environment than visual features, and the physical basis for the distinction is more stable. The point cloud density of high-voltage lines is low and continuously distributed, while the point cloud density of interference objects such as tree branches is higher and more disordered. Thus, the point cloud data can be distinguished from the perspective of three-dimensional morphology, which is more reliable. Therefore, the weights corresponding to reflectance and point cloud density should be higher. Visual texture is easily affected by the lighting angle and image quality, and its reliability is weaker than that of reflectance and point cloud density. Contour extraction is easily affected by factors such as occlusion and image blurring. Therefore, the weights corresponding to texture feature value and contour feature value should be lower.

[0094] Furthermore, since collisions between high-voltage lines and drones can easily cause power accidents, while collisions between drones and tree branches at most result in mission interruption with lower risks, this embodiment focuses more on the identification of high-voltage lines. That is, when calculating the score for each linear obstacle, it is desirable for the score corresponding to high-voltage lines to be higher, so that the obstacle avoidance priority is higher when linear obstacles such as high-voltage lines appear, thereby enabling the execution of obstacle avoidance strategies and avoiding situations where obstacle avoidance is not timely or is not executed when the priority is low, which could lead to power accidents.

[0095] Therefore, in order to further increase the score of linear obstacles such as high-voltage lines, after calculating the comprehensive texture feature value, when the comprehensive texture feature value is greater than the preset threshold, it indicates that the linear obstacle is more likely to be a high-voltage line. Since reflectivity has the highest weight, the reflectivity after normalization is increased according to the proportion of comprehensive texture feature values ​​greater than the preset threshold. This increases the value of reflectivity and compensates for the weak detection signal of the lidar for low-reflectivity targets, thereby increasing the score. In this embodiment, the value of the preset threshold is not limited.

[0096] This embodiment independently collects the object reflectance for each sub-region and simultaneously acquires the background reflectance. Based on this, the reflectance threshold is dynamically adjusted, effectively avoiding the problem of missed detection when calculating the global average. Furthermore, when calculating the score of linear obstacles, the reflectance is corrected through texture features, which compensates for the weak detection signal of the lidar for low-reflectance targets. This allows linear obstacles that are closer to the high-voltage line, as determined by texture features, to have a higher score, facilitating the subsequent calculation of obstacle avoidance priority and the execution of obstacle avoidance strategies based on the obstacle avoidance priority, effectively avoiding collisions with high-voltage lines.

[0097] Furthermore, this embodiment provides a step of updating the current path according to obstacle avoidance priority to obtain a first path, including:

[0098] Determine whether the characteristics of dynamic obstacles and linear obstacles meet preset conditions; preset conditions include feature conditions and multi-obstacle interaction conditions;

[0099] If so, update the current path based on the characteristics of the dynamic obstacles and linear obstacles to obtain the second path;

[0100] The second path is updated based on obstacle avoidance priority to obtain the first path.

[0101] After obtaining the scores for dynamic obstacles and each linear obstacle, the system first needs to filter out obstacles that require avoidance. For example, for dynamic obstacles, the system predicts their movement path based on their speed and direction, then obtains the current inspection path. If the movement path intersects with the current inspection path, and the time required for the drone and the dynamic obstacle to reach the intersection is the same based on the drone's inspection speed and the speed of the dynamic obstacle, then the obstacle is considered a dynamic obstacle that requires avoidance. This time is called the collision time. For linear obstacles, the system determines whether their location is on the current inspection path. If so, they are considered linear obstacles that require avoidance. For each obstacle that requires avoidance, its priority is determined according to a preset priority standard. For example, the priority can be divided into three levels: high, medium, and low. Each level corresponds to a different score range. Therefore, the priority can be determined by the score range to which the score belongs. This embodiment does not limit the specific value of the score range.

[0102] Specifically, for high-level obstacles, it indicates that the dynamic obstacle may be moving at a high speed or be close to the drone. In this case, rapid obstacle avoidance is required. However, since the drone tends to be stationary when acquiring images at the current inspection node, to avoid drone jitter (such as sudden turns) caused by rapid execution of the obstacle avoidance strategy and to ensure flight stability and safety, this embodiment sets up a buffer mechanism. That is, when at least one preset condition is met, the current inspection path is updated to obtain a second path, and then the first path is obtained. Among them, for dynamic obstacles, the corresponding characteristic condition is that the speed is less than a preset speed or the distance is greater than a preset distance. For linear obstacles, the corresponding characteristic condition is that the distance is greater than a preset distance. If the distance between the dynamic obstacle and the linear obstacle is less than or equal to the preset distance or the speed of the dynamic obstacle is greater than or equal to the preset speed, although the characteristic conditions are met, there is not enough time to execute the buffer mechanism. In this case, the obstacle avoidance strategy must be executed quickly, without generating a second path, and the current inspection path is directly updated to obtain the first path.

[0103] For example, a pre-stored direction can be set for the drone to quickly avoid obstacles when the buffer mechanism is not executed. Specifically, the performance limit parameters of the drone can be obtained according to the actual situation of the drone equipment. For example, if a drone has a maximum tilt angle of 45° and a maximum ascent / descent speed of 5m / s, the pre-stored direction can be set to 45° to the left, 45° to the right, and ascending... Rice and Fall rice, and The values ​​are respectively equal to the maximum ascent speed or the maximum descent speed multiplied by the preset collision time. Since the collision time was not obtained when setting the pre-stored direction, the preset collision time can be equal to the preset distance divided by the preset speed. When the buffer mechanism is not executed, a pre-stored direction needs to be randomly selected to perform fast obstacle avoidance. However, the selected pre-stored direction should meet the requirement that after performing obstacle avoidance according to the pre-stored direction, there will be no collision with any obstacle from the position after the pre-stored direction is completed to the next adjacent inspection node. If no such pre-stored direction exists, the pre-stored direction can be selected multiple times and fast obstacle avoidance can be performed. Compared with the traditional path planning method, this embodiment can complete obstacle avoidance in a shorter time based on the pre-stored direction, improving obstacle avoidance speed and efficiency. The setting of the performance limit parameters is only an example, and this embodiment does not impose any restrictions on it.

[0104] Furthermore, if the obstacle to be avoided is a dynamic obstacle and the dynamic obstacle is of a high level, and the speed of the dynamic obstacle is less than the preset speed and the distance is greater than the preset distance, then rapid obstacle avoidance is required. At the same time, in order to avoid the drone shaking due to rapid execution of the obstacle avoidance strategy, a buffer mechanism needs to be implemented. Specifically, firstly, a safe distance is determined based on the intersection point obtained above. The safe distance is the minimum safe distance between the position of the UAV and the position of the intersection point after the buffer mechanism is executed within the collision time. This embodiment does not limit the specific value of the minimum safe distance. The purpose of setting the minimum safe distance is to provide fault tolerance for obstacle avoidance. The position that meets this safe distance should be a circle centered on the intersection point. Then, combined with the predicted path of the dynamic obstacle, a position not located on the predicted path is randomly selected on the circle. Then, the distance between this position and the current inspection node is determined. When the distance and collision time are known, the UAV can automatically determine its acceleration and speed according to its corresponding equipment parameters (such as maximum acceleration and maximum speed) to slowly accelerate to the position within the collision time. This embodiment does not limit the acceleration and speed, that is, the duration of the buffer mechanism is equal to the collision time. The UAV needs to gradually accelerate within the collision time. Compared with direct rapid obstacle avoidance, the buffer mechanism allows the UAV to accelerate at a slower speed, which increases the distance between the dynamic obstacle and the UAV while avoiding sudden acceleration that could cause the UAV to lose control.

[0105] If the obstacle to be avoided is a linear obstacle that meets the characteristic conditions, since linear obstacles are usually fixed, compared to dynamic obstacles, when there are only linear obstacles, only the turning angle and direction need to be adjusted when performing the buffer mechanism and subsequent obstacle avoidance actions, without adjusting the speed. For example, in order to maintain the stability of the drone, a small turning angle, such as 15°, can be preset. Then the drone will choose the turning direction automatically according to the position of the linear obstacle. For example, when the linear obstacle is directly in front, the drone can choose to turn up or down 15°. The turning direction should meet two conditions: first, it should be away from the direction of movement of the dynamic obstacle, and second, it should be close to the next adjacent inspection node. However, this embodiment does not limit the specific direction and angle.

[0106] If the obstacles requiring obstacle avoidance are dynamic obstacles and linear obstacles that meet the characteristic conditions, it is first necessary to determine whether the dynamic obstacles and linear obstacles meet the multi-obstacle interaction conditions. The multi-obstacle interaction conditions refer to the collision between the linear obstacle and the dynamic obstacle, and the predicted fall trajectory of the dynamic obstacle after the collision (e.g., the fall trajectory of a kite after colliding with a detached high-voltage line) intersects with the current inspection route. At the same time, the time difference between the dynamic obstacle and the linear obstacle reaching this intersection point is less than a preset time difference. This embodiment does not impose a restriction on the preset time difference; the purpose of setting the preset time difference is to provide fault tolerance. Then, the collision positions of the UAV with the linear obstacle and the dynamic obstacle are obtained without considering the multi-obstacle interaction conditions. Then, the occurrence times of the three collision positions are compared, and the above-mentioned buffering mechanism is executed sequentially for the three collision positions. The method of executing the buffering mechanism for the intersection of the fall trajectory and the current inspection route can refer to the buffering mechanism for dynamic obstacles. After each execution of the buffering mechanism, it is necessary to determine whether the buffering mechanism needs to be executed for subsequent obstacles based on the current position of the UAV. At the same time as executing the buffering mechanism, a signal or sound can be emitted to make the dynamic obstacle aware of the collision risk.

[0107] At this point, due to the buffer mechanism, the drone does not need to follow the current path AB to reach the next adjacent inspection node. Instead, it reaches the buffer node and then executes the obstacle avoidance strategy. The buffer node is the position reached after the buffer mechanism is completed. The inspection path at this time is from the current inspection node to the buffer node, and then from the buffer node to the next adjacent inspection node. This path is called the second path. Since the obstacle avoidance strategy has not yet been executed, the path from the buffer node to the next adjacent inspection node in the second path is undetermined.

[0108] This embodiment sets up a buffer mechanism to allow the drone to gradually adjust its speed and angle, avoiding the impact of sudden shaking on the drone's stability. However, the buffer mechanism is only a transitional phase. Its main purpose is to smoothly switch the drone from a normal inspection state to a state where it can quickly avoid obstacles by adjusting its speed and direction slightly. However, the speed and direction after buffering only provide better conditions for obstacle avoidance, but do not allow the drone to completely get away from obstacles. Even for fixed linear obstacles, there may be measurement errors due to environmental interference. If it is a dynamic obstacle, its speed and direction may change continuously. The speed and direction adjusted based on the current state during the buffer phase may no longer be able to guarantee safety due to the subsequent movement of the obstacle. Therefore, continuous obstacle avoidance is required to keep the drone away from obstacles and ensure flight safety.

[0109] Furthermore, this embodiment provides a step of updating the second path according to the obstacle avoidance priority to obtain the first path, including:

[0110] If the current obstacle avoidance priority is high, the second path is updated based on the characteristics of dynamic obstacles and linear obstacles to obtain the first path.

[0111] If the current obstacle avoidance priority is medium or low, potential obstacle avoidance nodes are determined based on the positions of dynamic and linear obstacles, and the first path is obtained based on the potential obstacle avoidance nodes and the RBF neural network model.

[0112] Furthermore, this embodiment provides a step of updating the second path based on the characteristics of dynamic obstacles and linear obstacles to obtain the first path, including:

[0113] Based on the characteristics of dynamic obstacles and linear obstacles, the opposite action vector is obtained;

[0114] The second path is updated based on the opposite action vector and the Bézier curve to obtain the first path.

[0115] For example, if both dynamic obstacles and linear obstacles exist simultaneously, the drone's position after the buffering mechanism is executed is first subtracted from the coordinates of the dynamic obstacle to obtain the threat vector of the dynamic obstacle. Then, the position of the linear obstacle is obtained, and its position relative to the drone is determined. For example, if it is directly above the drone, the drone's y-axis coordinates are subtracted from the average y-axis coordinates of the linear obstacle to obtain the coordinate difference. Thus, the threat vector of the linear obstacle is obtained as follows: If the obstacle is directly in front, subtract the average x-axis coordinates of the drone from the average x-axis coordinates of the linear obstacle to obtain the coordinate difference. Thus, the threat vector of the linear obstacle is obtained as follows: Then, the two threat vectors are weighted and fused and inverted to obtain the opposite action vector. In this embodiment, the weights are not restricted. If there are only dynamic obstacles or linear obstacles, the corresponding threat vectors are directly inverted as the opposite action vector.

[0116] Next, the first path is obtained based on the Bézier curve. Specifically, the position of the buffer node is first taken as the starting point. Then, the starting point is added to the normalized vector of the opposite movement vector and multiplied by the safety distance to obtain the control point. This embodiment does not limit the safety distance, which can be determined by those skilled in the art based on the speed and volume of the dynamic obstacle. Then, the adjacent next inspection node is taken as the ending point. Substituting the coordinates of the starting point, ending point, and control point into the Bézier curve, the path from the buffer node to the adjacent next inspection node can be obtained. Finally, this path is combined with the path from the current inspection node to the buffer node, and recorded as the first path. For example, as shown... Figure 3 As shown, A is the current inspection node, B is the inspection node corresponding to the next adjacent moment, C is the current position of the dynamic obstacle, the solid line segment is the linear obstacle, if traveling along the current inspection route AB, it will collide with the dynamic obstacle at point E, D is the buffer node, F is the control point, and ADFB is the first path.

[0117] This embodiment analyzes the threat vectors of dynamic and linear obstacles, then inverses them to obtain the opposite action vector. This vector points directly to the optimal direction away from all threatening obstacles, which can quickly provide clear and accurate directional guidance for the drone's obstacle avoidance actions, ensuring that the drone avoids the collision risk of the two types of obstacles from the root. Then, the obstacle avoidance trajectory is planned through Bézier curves, minimizing the impact of obstacle avoidance on the drone's own state while ensuring safety.

[0118] Furthermore, this embodiment provides a step for obtaining a first path based on potential obstacle avoidance nodes and an RBF neural network model, including:

[0119] A grid is generated based on the next adjacent inspection node, and the grid consists of multiple spatial cells;

[0120] The node spacing and initial control nodes are obtained based on the spatial unit and RBF neural network model;

[0121] The control node sequence is obtained based on the climb rate between two adjacent nodes in the initial control node sequence;

[0122] The first path is obtained by updating the control node sequence based on the potential obstacle avoidance nodes.

[0123] Specifically, when the obstacle avoidance priority is medium or low, there is more time to determine a more reasonable path. First, it is necessary to obtain the position of the dynamic obstacle after the buffering mechanism is executed. For example, the position of the dynamic obstacle can be predicted by Kalman filtering, and the covariance matrix obtained in the last iteration prediction according to Kalman filtering can be obtained. Then, two first potential obstacle avoidance nodes are obtained based on the covariance matrix.

[0124] Specifically, such as Figure 4 As shown, after extracting the covariance matrix, only the variables related to the x and y directions are extracted to obtain the two-dimensional submatrix:

[0125]

[0126] And based on the eigenvalues ​​of the two-dimensional submatrix and Obtain the major axis of the ellipse short axis and rotation angle Then, two first potential obstacle avoidance nodes are generated on the elliptical boundary. and These two first potential obstacle avoidance nodes correspond to the two ends of the major axis of the ellipse. The role of the potential obstacle avoidance nodes is to define the safe range for the UAV to travel. During obstacle avoidance, the UAV should stay away from the potential obstacle avoidance nodes. Compared with selecting all positions on the boundary of the ellipse as the first potential obstacle avoidance nodes, this embodiment selects nodes only in the critical direction of the ellipse (i.e., the two ends of the major axis). The direction of the major axis of the ellipse is usually consistent with the direction of movement of the dynamic obstacle, which is the risk direction most likely to collide. Therefore, selecting only two nodes can reduce the amount of computation while covering the risk direction most likely to collide.

[0127] Next, the starting coordinates and ending coordinates of the linear obstacle are obtained, and then the direction of the linear obstacle is obtained. Then, the vertical direction of the direction is determined. The direction perpendicular to the direction can be divided into two parts, namely the part closer to the drone and the part farther away from the drone. Then, along the direction of the linear obstacle, multiple second potential obstacle avoidance nodes are evenly set in the part closer to the drone. In this embodiment, the distance between these potential obstacle avoidance nodes is not limited, for example, it is set to 30cm or 50cm.

[0128] Next, the UAV needs to determine the grid between its current position and the adjacent next inspection node. The grid includes multiple spatial cells. The size of the grid is determined based on the current position, the UAV's scanning area, and the position of the adjacent next inspection node. This embodiment does not limit the specific size of the grid and spatial cells, but the grid should include the position of the adjacent next inspection node. Then, the task value matrix corresponding to the generated grid is used. Each element in the task value matrix corresponds to a grid cell. For any element, if it includes the position of the adjacent next inspection node, its corresponding value is 10, otherwise it is 5. Here, 10 and 5 are just examples to highlight the value of the adjacent next inspection node. This embodiment does not limit its specific value. Then, the mean value of all spatial cells in the grid is calculated and input into the trained RBF neural network model. The RBF neural network model will map the mean to the node spacing coefficient according to the Gaussian radial basis function. The node spacing coefficient is multiplied by the preset benchmark node spacing to obtain the node spacing. This embodiment does not limit the preset benchmark node spacing, for example, it is set to 50m.

[0129] Then, based on the node spacing, starting from the current position of the UAV and ending at the position of the next adjacent inspection node, multiple initial control nodes are generated along a straight line to obtain an initial control node sequence. The distance between the last two initial control nodes may be less than the node spacing. Then, the climb rate between two adjacent initial control nodes is checked. For two initial control nodes with a climb rate greater than the preset climb rate, initial control nodes are added evenly between them until the climb rate between any two adjacent initial control nodes is less than or equal to the preset climb rate. Finally, all initial control nodes are recorded as control nodes, and the sequence of control nodes is recorded as the control node sequence.

[0130] Next, the aforementioned potential obstacle avoidance nodes are acquired, including a first potential obstacle avoidance node and a second potential obstacle avoidance node. The control node sequence is then checked for conflicts with these potential obstacle avoidance nodes, specifically whether the control node's position exceeds the range defined by the potential obstacle avoidance nodes, thus increasing the likelihood of collisions with linear or dynamic obstacles. If no conflict occurs, a path is generated from the UAV's current position (buffer node) to the next adjacent inspection node based on the control node sequence and a Bézier curve. This path is then combined with the path from the current inspection node to the buffer node to obtain the first path. If a conflict occurs, each conflicting control node is acquired and replaced with the nearest potential obstacle avoidance node, resulting in an updated control node sequence. This updated control node sequence is then repeatedly verified based on the climb rate and conflict with obstacle avoidance nodes until the resulting control node sequence satisfies the preset climb rate and does not conflict with potential obstacle avoidance nodes. Finally, the first path is obtained based on the final control node sequence and the Bézier curve.

[0131] Furthermore, the aforementioned methods for autonomous obstacle avoidance and path planning for unmanned aerial vehicles (UAVs) also include:

[0132] For each UAV, the corresponding policy set is obtained based on its updated control node sequence;

[0133] Based on the mission completion rate, energy consumption, and conflict risk of each drone, a delivery matrix corresponding to the strategy set is obtained;

[0134] The strategy combination is obtained based on the delivery matrix and the Nash equilibrium strategy;

[0135] Update the first path based on the strategy combination.

[0136] Specifically, if there is only one drone, the inspection is performed according to the first path. If there are multiple drones, there may be conflicts between the first paths of the multiple drones, so further optimization is needed. Specifically, assuming there are two drones A and B, firstly, the conflict area between the two drones is determined, that is, the location where the two drones collide. Then, the position of this location in the control node sequence corresponding to drone A is determined, that is, between which two control nodes this location is located. Afterwards, multiple feasible paths between these two nodes are obtained, resulting in the policy set corresponding to drone A. For example, the policy set corresponding to drone A could be... Similarly, the strategy set corresponding to drone B can be obtained. Then, based on the communication mechanism between drones, the other party's strategy set is obtained. Then, from the perspective of drone A, a delivery matrix is ​​constructed. The delivery matrix contains the... Line number The elements in the column refer to the actions performed by drone A. Drone B executes At that time, A's comprehensive benefit is determined based on task completion, energy consumption, and conflict risk, in order to calculate the... Line number Taking the elements of a column as an example, task completion can be divided into two levels: complete coverage and no coverage. When executing... Afterwards, when drone A approaches the next adjacent inspection node, it is in a complete coverage level; conversely, when it moves away, it is in a non-coverage level. Energy consumption can be determined based on turning angle and climb altitude. For example, low, medium, and high energy consumption levels can be determined based on turning angle and climb altitude. The larger the turning angle and the higher the climb altitude, the higher the energy consumption level. Conflict risk includes low, medium, and high risk levels. When drone A executes... Drone B executes The closer the two drones are, the higher their corresponding level. Then, different scores are set for each level in the determination of mission completion, energy consumption, and conflict risk. The score of the complete coverage level is higher than that of the uncovered level. The higher the energy consumption level, the lower the score. The higher the conflict risk level, the lower the score. The actual scores are statistically analyzed, weighted, and normalized to obtain the corresponding comprehensive benefit. This embodiment does not restrict the division of the score interval or the score value. Similarly, the delivery matrix corresponding to drone B can be obtained.

[0137] Next, iterative operations are performed based on the delivery matrix. First, the strictly disadvantaged strategy corresponding to drone A is eliminated. A strictly disadvantaged strategy is defined as follows: if there exists a strategy whose payoffs are all lower than another strategy, then the latter is a strictly disadvantaged strategy and can be directly eliminated. For example, first compare... , and The benefits, if compared with When comparing, ,but ,at this time and Neither of them meets the definition of a strictly disadvantageous strategy. When comparing, , , , This represents the element in the first row and first column of the delivery matrix corresponding to drone A, and so on. To satisfy the definition of a strictly disadvantageous strategy, Remove from the strategy set to obtain Similarly, we need to eliminate the strictly disadvantageous strategy corresponding to drone B. Assuming that after elimination we get... The delivery matrix is ​​updated based on the updated policy set, and then a policy is selected from each of the two updated policy sets based on Nash equilibrium. Nash equilibrium is defined as follows: if the policy combination... Satisfy: For drone A, yes The optimal choice for drone B is... yes The optimal path is a Nash equilibrium. The first path is then updated based on the selected strategy combination to avoid collisions between the two drones. If no such strategy combination exists, a hybrid strategy is used, meaning drone A needs to choose a path with probability... and choose One of them, drone B needs to be probabilistically... and choose one of the, The value of should guarantee and The expected returns are equal, and similarly, The value of should guarantee and The expected returns are equal, and then the two drones make a strategy selection based on probability and update the first path.

[0138] Furthermore, as the drone travels to the next adjacent inspection node, it is still necessary to monitor the position of dynamic obstacles in real time. If the position deviates from the previously predicted position and the deviation exceeds a preset deviation value, the first path needs to be updated in real time. Specifically, firstly, a prediction is made based on the drone's current position to obtain a predicted position sequence. Then, according to a preset objective function and constraints, a control output sequence is obtained. For example, the objective function... for:

[0139]

[0140] in, This indicates the number of elements in the predicted position sequence. Represents the first position in the predicted position sequence One location, This represents the first path obtained based on the aforementioned Nash equilibrium strategy. express arrive The square of the Euclidean distance, express The corresponding moment is the position of the dynamic obstacle. This represents the minimum safety threshold for dynamic obstacles and drones. for and The distance between them Indicates to The control commands, their dimensions and The same, where each element represents The distance that the corresponding element in the table needs to be moved. express The sum of squares of all elements in the sequence. Constraints are used to ensure that the distance between the corrected position of the UAV and the corresponding dynamic obstacle is greater than a minimum safety threshold. After obtaining the control output sequence, the UAV position is corrected only according to the first element of the sequence, and then the above real-time detection and correction steps are repeated in subsequent time steps.

[0141] This embodiment first predicts the position of dynamic obstacles using Kalman filtering, and generates potential obstacle avoidance nodes based on the positions of dynamic obstacles and linear obstacles. Then, it generates a control node sequence based on the RBF neural network model and the potential obstacle avoidance nodes, and generates a first path based on the control node sequence and Bézier curves, ensuring that the UAV can smoothly complete the inspection while avoiding obstacles. In addition, this embodiment selects the optimal strategy combination based on Nash equilibrium iteration for the case of multiple UAVs colliding, and corrects the first path in real time according to the position deviation of dynamic obstacles during the UAV's flight, ensuring the real-time performance of obstacle avoidance.

[0142] Furthermore, embodiments of this application provide a multi-sensor fusion-based autonomous obstacle avoidance and path planning system for unmanned aerial vehicles, including:

[0143] The acquisition module is used to acquire dynamic obstacles and linear obstacles within the scanning area at the current moment;

[0144] The correction module normalizes the features of dynamic obstacles and linear obstacles, and corrects the normalized features to obtain the obstacle score.

[0145] The buffer module is used to determine the current obstacle avoidance priority based on the obstacle score;

[0146] The path planning module is used to update the current path based on obstacle avoidance priority.

[0147] This application provides a multi-sensor fusion-based autonomous obstacle avoidance and path planning system and method for unmanned aerial vehicles (UAVs). The system updates the reflectivity threshold by background reflectivity to determine whether obstacles exist. Then, it modifies the characteristics of obstacles by integrating texture feature values ​​and contour feature values ​​to enhance the perception of linear obstacles, thereby obtaining a more accurate obstacle avoidance priority. Finally, for different obstacle avoidance priorities, different obstacle avoidance strategies are selected to update the current inspection path, and it is ensured that the system can return to the initial inspection route as soon as possible after obstacle avoidance.

[0148] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0149] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0150] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0151] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for multi-sensor fusion based autonomous obstacle avoidance and path planning of a UAV, the method comprising: The method comprises the following steps: acquiring dynamic obstacles and linear obstacles in a scanning area corresponding to a current time; normalizing features of the dynamic obstacles and the linear obstacles, and correcting the normalized features to obtain obstacle scores; updating a current path according to the obstacle scores to obtain a first path; wherein the step of correcting the normalized features to obtain the obstacle scores comprises the following steps: acquiring a spectral image corresponding to the linear obstacles; obtaining texture uniformity, direction consistency and contrast according to the spectral image; obtaining a comprehensive texture feature value according to the texture uniformity, the direction consistency and the contrast; correcting the normalized features according to the comprehensive texture feature value and a visual image to obtain the obstacle scores; wherein the step of correcting the normalized features according to the comprehensive texture feature value and the visual image to obtain the obstacle scores comprises the following steps: correcting the normalized reflectivity according to the comprehensive texture feature value; obtaining a linear profile according to the visual image; obtaining a profile feature value according to a fitting straight line error corresponding to a sub-line segment in the linear profile and an included angle corresponding to the sub-line segment; obtaining the obstacle scores according to the profile feature value and the corrected reflectivity. The scanning area comprises a plurality of sub-areas, and the step of acquiring the linear obstacles in the scanning area corresponding to the current time comprises the following steps:

2. The multi-sensor fusion based autonomous obstacle avoidance and path planning method for UAV according to claim 1, wherein, acquiring a background reflectivity and an object reflectivity corresponding to each sub-area; correcting a reflectivity threshold according to the background reflectivity; obtaining the linear obstacles according to the corrected reflectivity threshold and the object reflectivity. The step of updating the current path according to the obstacle scores to obtain the first path comprises the following steps:

3. The multi-sensor fusion based autonomous obstacle avoidance and path planning method for UAV according to claim 1, wherein, determining whether the features of the dynamic obstacles and the linear obstacles satisfy preset conditions; the preset conditions comprise feature conditions and multi-obstacle interaction conditions; if yes, updating the current path according to the features of the dynamic obstacles and the linear obstacles to obtain a second path; updating the second path according to the obstacle scores to obtain the first path. The step of updating the second path according to the obstacle scores to obtain the first path comprises the following steps:

4. The multi-sensor fusion based autonomous obstacle avoidance and path planning method for UAV according to claim 3, wherein, if the current obstacle priority is a high level, updating the second path according to the features of the dynamic obstacles and the linear obstacles to obtain the first path; if the current obstacle priority is a medium level or a low level, determining a potential obstacle avoidance node according to the positions of the dynamic obstacles and the linear obstacles, and obtaining the first path according to the potential obstacle avoidance node and a RBF neural network model. The step of updating the second path according to the features of the dynamic obstacles and the linear obstacles to obtain the first path comprises the following steps:

5. The multi-sensor fusion based autonomous obstacle avoidance and path planning method for UAV according to claim 4, wherein, obtaining an opposite action vector according to the features of the dynamic obstacles and the linear obstacles; updating the second path according to the opposite action vector and a Bezier curve to obtain the first path. The step of obtaining the first path according to the potential obstacle avoidance node and the RBF neural network model comprises the following steps:

6. The multi-sensor fusion based autonomous obstacle avoidance and path planning method for UAV according to claim 4, wherein, generating a grid according to an adjacent next inspection node; the grid comprises a plurality of space units; obtaining a node distance and an initial control node according to the space units and the RBF neural network model; ​ According to the climbing rate between two adjacent nodes in the initial control node, a control node sequence is obtained; According to the potential obstacle avoidance node, the control node sequence is updated to obtain the first path.

7. The multi-sensor fusion based autonomous obstacle avoidance and path planning method for UAV according to claim 6, wherein, If the current obstacle avoidance priority is medium and low, and there are multiple unmanned aerial vehicles, after obtaining the first path, the unmanned aerial vehicle autonomous obstacle avoidance and path planning method further comprises: For each unmanned aerial vehicle, a corresponding strategy set is obtained according to the updated control node sequence corresponding thereto; According to the task completion degree, energy consumption and conflict risk of each unmanned aerial vehicle, a delivery matrix corresponding to the strategy set is obtained; According to the delivery matrix and Nash equilibrium strategy, a strategy combination is obtained; The first path is updated according to the strategy combination.

8. A multi-sensor fusion unmanned aerial vehicle autonomous obstacle avoidance and path planning system, characterized in that, Comprise: The acquisition module is used for acquiring dynamic obstacles and linear obstacles in the scanning area corresponding to the current time; The correction module normalizes the features of the dynamic obstacles and linear obstacles, and corrects the normalized features to obtain obstacle scores; The buffer module is used for obtaining the current obstacle avoidance priority according to the obstacle scores; The path planning module is used for updating the current path according to the obstacle avoidance priority; Wherein, the correction of the normalized features to obtain the obstacle scores comprises: Obtain the spectral image corresponding to the linear obstacle; According to the spectral image, the texture uniformity, the direction consistency and the contrast are obtained; According to the texture uniformity, the direction consistency and the contrast, a comprehensive texture feature value is obtained; According to the comprehensive texture feature value and the visual image, the normalized features are corrected to obtain the obstacle scores; Wherein, according to the comprehensive texture feature value and the visual image, the normalized features are corrected to obtain the obstacle scores, comprising: According to the comprehensive texture feature value, the normalized reflectivity is corrected; According to the visual image, a linear profile is obtained; According to the fitting straight line error corresponding to the sub-line segment in the linear profile and the included angle corresponding to the sub-line segment, a profile feature value is obtained; According to the profile feature value and the corrected reflectivity, the obstacle scores are obtained.

Images