Unmanned aerial vehicle system in satellite signal rejection area

Abstract

The invention relates to the technical field of unmanned aerial vehicles, in particular to an unmanned aerial vehicle system in a satellite signal rejection area. Comprising a state evaluation module for evaluating the state of the unmanned aerial vehicle based on multi-source data; the mapping module constructs a multi-level map based on multi-source data; the environment sensing module is used for acquiring a multi-level map and identifying obstacles in the environment through a neural network model; the path planning module obtains a global path based on a global planner; obtaining a local path based on an improved time domain elastic band planner; combining the global path and the local path to obtain an unmanned aerial vehicle line trajectory; the control module obtains data of the environment sensing module and data of the path planning module and is used for controlling flight of the unmanned aerial vehicle. The limitation of a traditional navigation mode is broken through, and technical support is provided for reliable operation of the unmanned aerial vehicle in an extremely narrow and complex environment through sensor data fusion, multi-level map construction, systematic path planning and the like.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and more particularly to a UAV system for satellite signal denial areas. Background Technology

[0002] With the rapid development of drone technology, its application in fields such as industrial inspection, emergency rescue, and infrastructure construction is becoming increasingly widespread. Especially in complex industrial settings such as mine shafts, high-speed railway hollow piers, and bridge box girders, drones, with their unique mobility, can replace manual labor in completing inspection and survey tasks in hazardous environments. However, these typical operating environments often feature complex spatial structures and severe or even complete satellite signal attenuation, posing serious challenges to the autonomous navigation and obstacle avoidance capabilities of drones.

[0003] In satellite-denied environments, drones that traditionally rely on global navigation satellite systems face the risk of positioning failure. Especially in the aforementioned confined spaces, not only are satellite signals physically blocked, but multipath effects and electromagnetic interference can further worsen positioning conditions. Furthermore, these environments typically feature poor lighting, repetitive visual textures, and complex spatial structures, making single-vision-based navigation methods prone to cumulative errors or even complete failure. Simultaneously, due to extremely limited space, the safety margin between the drone and obstacles is minimal; any delay in perception or decision-making can lead to a collision.

[0004] To address the aforementioned issues, there is an urgent need to develop a drone system solution capable of precise positioning, intelligent obstacle avoidance, and fully autonomous operation even in the absence of satellite signals. Summary of the Invention

[0005] In view of the above problems, this disclosure provides an unmanned aerial vehicle (UAV) system for satellite signal denial areas that overcomes or at least partially solves the above problems. The purpose is to break through the limitations of traditional navigation methods and provide technical support for the reliable operation of UAVs in extremely narrow and complex environments through sensor data fusion, multi-level map construction and systematic path planning, thereby promoting the application of UAVs in satellite signal denial areas.

[0006] The objective of this invention can be achieved through the following technical solutions: The present invention provides an unmanned aerial vehicle (UAV) system for satellite signal denial areas, comprising: The system includes a status assessment module, a mapping module, an environmental perception module, a path planning module, and a control module. The status assessment module assesses the drone's status based on multi-source data. The mapping module constructs a multi-layered map based on multi-source data; The environmental perception module acquires multi-layer maps and identifies obstacles in the environment through a neural network model; The path planning module obtains a global path based on a global planner; obtains a local path based on an improved time-domain elastic band planner; and obtains the UAV trajectory by combining the global path and the local path. The control module acquires data from the environmental perception module and the path planning module to control the flight of the drone.

[0007] Furthermore, the state assessment module acquires multi-source data, including lidar data, camera images, and IMU data; Based on IMU data, the pre-integration error is obtained through the IMU pre-integration model; Based on camera images, the landmark points are transformed from the world coordinate system to the camera coordinate system to obtain the reprojection error; Based on the point cloud formed by lidar data, the lidar point cloud matching error is obtained for each lidar point to the nearest plane in the target point cloud.

[0008] Furthermore, based on the covariance matrix corresponding to the pre-integration error, the weighted least squares function of the pre-integration error is obtained; Based on the covariance matrix corresponding to the reprojection error, the weighted least squares function of the reprojection error is obtained; Based on the covariance matrix corresponding to the lidar point cloud matching error, the weighted least squares function of the lidar point cloud matching error is obtained. The optimal UAV state is evaluated to minimize the sum of the weighted least squares functions corresponding to the pre-integration error, reprojection error, and lidar point cloud matching error.

[0009] For IMU data processing, an IMU pre-integration model is used. IMU pre-integration calculates the relative motion between two adjacent keyframes, avoiding repeated propagation of IMU data during optimization. Pre-integrated measurements include position increments. Speed ​​increment and rotation increment , representing the pre-integral changes in position, velocity, and attitude from time i to time j, respectively. These are the relative constraints connecting the motion between two keyframes, defined over the time interval. The pre-integration error is obtained by directly integrating the raw angular velocity and acceleration measurements from the IMU; therefore, the pre-integration error is defined as: ; in, , representing the position increment error, where The pre-integrated measurement value of the UAV's position status; , representing the speed increment error, where The pre-integrated measurement value of the UAV's speed state; , representing the rotational increment error, The pre-integrated measurement value of the UAV's attitude state; This indicates the accelerometer bias error. This represents the zero bias vector of the accelerometer at time i. This represents the zero-bias vector of the accelerometer at time j; This indicates the accelerometer bias error. Let represent the zero bias vector of the gyroscope at time i. Let represent the zero bias vector of the gyroscope at time j.

[0010] For a camera image containing visual landmarks, the coordinates of each visual landmark in the camera coordinate system are: ,in, It is a rotation matrix, representing the rotation from the world coordinate system to the camera coordinate system. It is a translation vector, representing the translation from the world coordinate system to the camera coordinate system; It represents the three-dimensional position of a landmark in the world coordinate system.

[0011] Then project it onto the normalized plane: ; represents the two-dimensional coordinates of the landmark on the camera's normalized plane; Then, the reprojection error is expressed as: ;in, This is a camera projection model, responsible for normalizing coordinates. Converted to pixel coordinates using camera intrinsic parameters , This represents the two-dimensional pixel coordinates of the landmark point as actually observed in the image.

[0012] For point clouds of LiDAR data, the LiDAR point cloud matching error is expressed as: ; in, This indicates that in the target point cloud, such as the point cloud of the previous frame, the source point... The normal vector of the local plane containing the corresponding nearest neighbor; T is the transformation matrix; This represents the i-th 3D point in the source point cloud, which is the point cloud of the current frame. Indicates that in the target point cloud, with The three-dimensional position of the corresponding nearest neighbor.

[0013] The UAV status is assessed by minimizing the pre-integration error, reprojection error, and lidar point cloud matching error. Represented as ; in, The goal of the optimization problem is to find an optimal system state X that minimizes the subsequent cost function. This represents the square of the Mahalanobis distance, i.e., the weighted least squares function; The inverse of the covariance matrix corresponding to the error represents the uncertainty of our measurements from different sensors. The greater the uncertainty, the smaller the weight.

[0014] Furthermore, the multi-layered map includes a grid map and an Euclidean map; Based on the multi-source LiDAR data, the occupancy probability of the grid is obtained through the Bayesian update formula, and the occupancy probability of all grids is collected to obtain the occupied grid map. Based on the occupied grid map, the signed distance from each grid to the nearest obstacle is calculated to obtain the Euclidean map.

[0015] Furthermore, the step of obtaining the occupancy probability of a grid cell based on the multi-source data using a Bayesian update formula, and then aggregating the occupancy probabilities of all grid cells to obtain an occupied grid cell map, includes: Set the prior probability of occupying a grid map: for each grid initial probability ; Constructing an inverse sensor model: Obtaining the laser beam measurement value from each of the lidar data. ; For each grid cell on the ray path , Calculate the distance function ;in, For grid The world coordinates of the center point This refers to the sensor's position in the current world coordinate system. Calculate relative position ;in, The distance from the sensor to the obstacle being measured; Therefore, the inverse sensor model is: ;in, This indicates the probability of the measurement endpoint being occupied. Represents the free space probability along the ray path; The value of the leading edge region representing the measurement endpoint; The occupancy probability of the grid is obtained using the Bayesian update formula: ; in, This is the prior probability; This represents the probability that grid i is occupied given 1 to t observations. This represents the probability that grid i is occupied given observations from 1 to t-1. Update the occupied grid map: By updating the occupied probability of the grid, an updated occupied grid map is obtained.

[0016] Furthermore, based on the occupied grid map, the signed distance from each grid cell to the nearest obstacle is calculated to obtain the Euclidean map, including: For grid Its signed distance is represented as ; in Represents a grid any point in the middle To the boundary of the obstacle The distance; ; in, It is the boundary of the obstacle. The point on; express and The Euclidean distance between them; Based on the constructed occupied grid map, the signed distance from each grid cell to the nearest obstacle is calculated, and the obtained signed distances are used to update the corresponding grid cells in the grid map. Inside, we obtained the Euclidean map.

[0017] Further gradient calculations are performed, including the gradient of the signed distance, pointing in the direction of the nearest obstacle; The formula is expressed as: ; Where i, j, and k are any points The three-dimensional coordinates To occupy the resolution of the raster map; if points In free space, then It is a positive number, and this value is the safe distance between the drone and the nearest obstacle.

[0018] If point It is right on the boundary of the obstacle, then =0.

[0019] If point Inside the obstacle, this is a situation that theoretically shouldn't happen, but mathematically it's defined, so the distance is defined as a negative number, which makes... =0.

[0020] Furthermore, the acquisition of multi-layer maps and the identification of obstacles in the environment through a neural network model include: The process involves acquiring an occupied grid map of a multi-layered map, and then outputting multi-task prediction results through an encoder-decoder network architecture. The map height, width, and channel data of the occupied grid map are used as inputs. The map channel data includes the occupancy probability and the gradient of the signed distance. The multi-task prediction results include obstacle semantic segmentation, obstacle distance estimation, boundary clarity, and accessibility score.

[0021] Furthermore, the encoder includes shallow feature extraction, spatial downsampling, mid-level feature extraction, deep feature extraction, and global context extraction; The decoder includes initial upsampling and skip connections, secondary upsampling and skip connections, and tertiary upsampling and skip connections.

[0022] The input G of the encoder-decoder network is represented as: map height data H × map width data W × map channel data C.

[0023] Furthermore, the local path obtained based on the improved time-domain flexible band planner includes: A pre-route trajectory is constructed based on an improved temporal elastic band planner, which is represented as a series of pose sequences. Then, the optimal UAV pose sequence is solved based on obstacle cost, dynamic constraint cost, time optimality cost, and global path consistency cost to obtain the local path.

[0024] Furthermore, the pose sequence , ;in, The coordinates of the drone's position; Indicates the yaw angle; Represents a timestamp; Construct the optimization equation: ;in For the cost of obstacles; This refers to the velocity constraint cost within the dynamic constraint cost; This refers to the acceleration constraint cost within the dynamic constraint cost; This comes at the cost of time optimality. This is due to the cost of global path consistency; in, ; Express posture To the obstacle The signed distance; This is expressed as the attenuation coefficient; The cost of speed constraint is expressed as ; The cost of acceleration constraints is expressed as ; The time optimality cost is expressed as ; The cost of global path consistency is expressed as: ; Express posture To global path The closest distance.

[0025] The technical solution proposed in this application can bring the following beneficial effects: 1. This invention achieves reliable perception in extreme environments such as complete darkness and weak texture by tightly coupling LiDAR data and camera image data. Compared with traditional methods, it can not only detect the geometric existence of obstacles, but also identify their semantic categories, enabling UAVs to make more forward-looking decisions, identify narrow passages in advance and adjust their flight attitude, significantly improving the passability and safety in complex structures such as mines and box girders. 2. This invention constructs a multi-layered map system. The multi-layered map architecture ensures that the UAV can maintain centimeter-level positioning accuracy even in environments without satellite signals. Other modules can extract maps from different layers, improving information processing efficiency and solving the problem of insufficient adaptability of a single map representation in complex environments. 3. This invention provides a complete technical solution for fully autonomous operation of UAVs in extremely narrow environments without satellite signals through multi-level environmental perception, precise map construction, and intelligent path planning, significantly improving the application value and reliability of UAVs in complex industrial scenarios.

[0026] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, specific embodiments of this disclosure are described below. Attached Figure Description

[0027] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this disclosure. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1This is a schematic diagram of the structure of an unmanned aerial vehicle system in a satellite signal denial area provided in the embodiments of this specification; Figure 2 This is a flowchart of the steps for evaluating the optimal UAV state provided in the embodiments of this specification; Figure 3 This is a flowchart illustrating the steps for constructing an occupied grid map as provided in the embodiments of this specification; Figure 4 This is a flowchart of the path planning method provided in the embodiments of this specification. Detailed Implementation

[0028] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. The technical solutions provided by various embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0029] Figure 1 This is a schematic diagram of the structure of an unmanned aerial vehicle system in a satellite signal denial area, provided for one or more embodiments of this specification.

[0030] like Figure 1 As shown, this application embodiment provides a drone system for a satellite signal denied area. The drone system for a satellite signal denied area includes: The system comprises: a status assessment module S101, a mapping module S102, an environmental perception module S103, a path planning module S104, and a control module S105. The status assessment module assesses the drone's status based on multi-source data. The mapping module constructs a multi-layered map based on multi-source data; The environmental perception module acquires multi-layer maps and identifies obstacles in the environment through a neural network model; The path planning module obtains a global path based on a global planner; obtains a local path based on an improved time-domain elastic band planner; and obtains the UAV trajectory by combining the global path and the local path. The control module acquires data from the environmental perception module and the path planning module to control the flight of the drone.

[0031] Furthermore, the state assessment module acquires multi-source data, including lidar data, camera images, and IMU data; The specific steps include: S21. Based on IMU data, obtain the pre-integration error through the IMU pre-integration model; S22. Based on the camera images, the landmark points are transformed from the world coordinate system to the camera coordinate system to obtain the reprojection error; S23. Based on the point cloud formed by lidar data, the lidar point cloud matching error is obtained for each lidar point to the nearest plane in the target point cloud.

[0032] S24. Based on the covariance matrix corresponding to the pre-integration error, obtain the weighted least squares function of the pre-integration error; S25. Based on the covariance matrix corresponding to the reprojection error, obtain the weighted least squares function of the reprojection error; S26. Based on the covariance matrix corresponding to the lidar point cloud matching error, obtain the weighted least squares function of the lidar point cloud matching error; S27. Evaluate the optimal UAV state so that the sum of the weighted least squares functions corresponding to the pre-integration error, reprojection error, and lidar point cloud matching error is minimized.

[0033] In areas where satellite signals are denied, GPS signals cannot be obtained, and drones need to rely on their own sensors for positioning. The tightly coupled laser-vision-inertial odometry method (LVI-SLAM) can fuse data from lidar, vision, and IMU. It uses high-frequency data from the IMU to compensate for the low-frequency updates of the laser and vision, while using absolute observations from the laser and vision to correct IMU drift. This fusion method can still maintain robustness under extreme conditions such as rapid movement, texture loss, or changes in lighting.

[0034] First, construct the UAV state vector, which includes the UAV's own state and the state of the landmarks perceived in the environment, represented as: ; in, Represents: the three-dimensional position vector of the UAV in the world coordinate system; The three-dimensional velocity vector of the UAV in the world coordinate system; , the unit quaternion of the UAV attitude (orientation); it belongs to the special orthogonal group SO(3) and is used to describe three-dimensional rotation; The zero bias vector of the accelerometer is an error term used to correct for the slow-changing bias inherent in accelerometer measurements. The zero bias vector of the gyroscope is also an error term used to correct the inherent, slowly changing bias in the gyroscope's measurements. , where i is the three-dimensional position vector of the i-th environmental landmark in the world coordinate system. The landmark is derived from the image features of the camera or the specific point cloud features of the lidar. T is the transformation matrix.

[0035] For IMU data processing, an IMU pre-integration model is used. IMU pre-integration calculates the relative motion between two adjacent keyframes, avoiding repeated propagation of IMU data during optimization. Pre-integrated measurements include position increments. Speed ​​increment and rotation increment , representing the pre-integral changes in position, velocity, and attitude from time i to time j, respectively. These are the relative constraints connecting the motion between two keyframes, defined over the time interval. The pre-integration error is obtained by directly integrating the raw angular velocity and acceleration measurements from the IMU; therefore, the pre-integration error is defined as: ; in, , representing the position increment error, where The pre-integrated measurement value of the UAV's position status; , representing the speed increment error, where The pre-integrated measurement value of the UAV's speed state; , representing the rotational increment error, The pre-integrated measurement value of the UAV's attitude state; This indicates the accelerometer bias error. This represents the zero bias vector of the accelerometer at time i. This represents the zero-bias vector of the accelerometer at time j; This indicates the accelerometer bias error. Let represent the zero bias vector of the gyroscope at time i. This represents the zero bias vector of the gyroscope at time j; T is the transformation matrix; in, ; ; ; Let represent the rotation matrix corresponding to the body posture at time i; This represents the position state of the drone at time i; This represents the position and state of the UAV at time j; This represents the speed state of the drone at time i; This represents the velocity state of the drone at time j; This represents the attitude state of the UAV at time i; This represents the attitude state of the UAV at time j; This represents the time interval from time i to time j; g represents the gravitational acceleration vector in the world coordinate system; This represents the quaternion multiplication operator.

[0036] IMU pre-integration technology pre-integrates frequent IMU measurements into relative motion constraints between two keyframes. This way, in subsequent optimization, when the initial state... , , When adjusted, there is no need to repeat the integral calculation, which greatly improves the calculation efficiency.

[0037] For a camera image containing visual landmarks, the coordinates of each visual landmark in the camera coordinate system are: ,in, It is a rotation matrix, representing the rotation from the world coordinate system to the camera coordinate system. It is a translation vector, representing the translation from the world coordinate system to the camera coordinate system; Represents the three-dimensional position of a landmark in the world coordinate system; Then project it onto the normalized plane: ; represents the two-dimensional coordinates of the landmark on the camera's normalized plane; Then, the reprojection error is expressed as: ;in, This is a camera projection model, responsible for normalizing coordinates. Converted to pixel coordinates using camera intrinsic parameters , These are the actual two-dimensional pixel coordinates of the landmark point as observed in the image; By minimizing reprojection error, the drone pose and landmark positions can be accurately corrected.

[0038] For point clouds of LiDAR data, the LiDAR point cloud matching error is expressed as: ; in, This indicates that in the target point cloud, such as the point cloud of the previous frame, the source point... The normal vector of the local plane containing the corresponding nearest neighbor; T is the transformation matrix; This represents the i-th 3D point in the source point cloud, which is the point cloud of the current frame. Indicates that in the target point cloud, with The three-dimensional position of the corresponding nearest neighbor; By minimizing the matching error of the lidar point cloud, the current frame point cloud can be fitted onto the surface of the global map, thereby obtaining high-precision pose estimation.

[0039] The UAV status is assessed by minimizing the pre-integration error, reprojection error, and lidar point cloud matching error. Represented as ; in, The goal of the optimization problem is to find an optimal system state X that minimizes the subsequent cost function. This represents the square of the Mahalanobis distance, i.e., the weighted least squares function; The inverse of the covariance matrix corresponding to the error represents the uncertainty of our measurements from different sensors. The greater the uncertainty, the smaller the weight.

[0040] Tight coupling can fully utilize the advantages of different sensors: IMU provides short-term accurate motion prediction, while vision and laser provide absolute observation, reducing drift and improving positioning accuracy and robustness.

[0041] Furthermore, the multi-layered map includes a grid map and an Euclidean map; Based on the multi-source LiDAR data, the occupancy probability of the grid is obtained through the Bayesian update formula, and the occupancy probability of all grids is collected to obtain the occupied grid map. Based on the occupied grid map, the signed distance from each grid to the nearest obstacle is calculated to obtain the Euclidean map.

[0042] Furthermore, the step of obtaining the occupancy probability of a grid cell based on the multi-source data using a Bayesian update formula, and then aggregating the occupancy probabilities of all grid cells to obtain an occupied grid cell map, includes: S31. Set the prior probability of occupying a grid map: for each grid initial probability ; S32. Construct an inverse sensor model: Obtain the laser beam measurement value from each of the lidar data. ; To represent the straight-line distance from the sensor origin to the obstacle surface, The horizontal angle represents the azimuth angle of the laser beam in the horizontal plane. The pitch angle; Set the measurement endpoint as , ; The position of the obstacle in the current world coordinate system; via coordinates and The coordinates can be calculated using the Euclidean distance formula. The Euclidean distance formula is existing technology and will not be elaborated here.

[0043] For each grid cell on the ray path , Calculate the distance function ;in, For grid The world coordinates of the center point This refers to the sensor's position in the current world coordinate system. and Calculation method reference ; in, This represents the Euclidean distance from the sensor origin to the center of the grid.

[0044] Calculate relative position ; represents the ratio of the distance from the sensor to the grid to the distance from the sensor to the obstacle being measured; where, The distance from the sensor to the obstacle being measured; Inverse sensor model: ;in, This represents the probability of occupancy at the measurement endpoint, and its value range is... ; This represents the free space probability along the ray path, and its value range is... ; The value representing the leading edge region at the measurement endpoint is set by a technician; for example, it is 0.1. S33. Obtain the occupancy probability of the grid using the Bayesian update formula: ; This is the prior probability, usually set to 0.5; This represents the probability that grid i is occupied given 1 to t observations. Similarly, This represents the probability that grid i is occupied given observations from 1 to t-1.

[0045] S34. Update the occupied grid map: By updating the occupation probability of the grid, an updated occupied grid map is obtained.

[0046] Occupied grid maps are primarily used for obstacle avoidance and traversable area analysis, providing the basic geometry of the environment.

[0047] Furthermore, based on the occupied grid map, the signed distance from each grid cell to the nearest obstacle is calculated to obtain the Euclidean map, including: For grid Its signed distance is represented as ; in Represents a grid any point in the middle To the boundary of the obstacle The distance; It defines the shortest distance from any point in space to the entire set of obstacles. It is calculated by finding the point on the surface of the obstacle that is closest to the given point and then measuring the straight-line distance between them. in, It is the boundary of the obstacle. The point on; express and The Euclidean distance between them; Based on the constructed occupied grid map, the signed distance from each grid cell to the nearest obstacle is calculated to obtain the Euclidean map.

[0048] The signed distance is defined as follows: for each grid cell, the signed distance represents the distance from the grid cell to the nearest obstacle, with positive indicating free space and negative indicating occupied area.

[0049] Further gradient calculations are performed, including the gradient of the signed distance, pointing in the direction of the nearest obstacle; The formula is expressed as: ; Where i, j, and k are any points The three-dimensional coordinates To occupy the resolution of the raster map; if points In free space, then It is a positive number, and this value is the safe distance between the drone and the nearest obstacle.

[0050] If point It is right on the boundary of the obstacle, then = 0.

[0051] If point Inside the obstacle, this is a situation that theoretically shouldn't happen, but mathematically it's defined, so the distance is defined as a negative number, which makes... = 0.

[0052] Value to route planning: Security: The planner can directly require all waypoints to meet preset conditions; for example, satisfying... This ensures that the drone maintains a safe distance of at least 0.5 meters from any obstacle.

[0053] Efficiency: Gradient The direction in which the distance increases the fastest points directly away from the obstacle, providing the optimal escape direction for the local path.

[0054] Furthermore, the acquisition of multi-layer maps and the identification of obstacles in the environment through a neural network model include: The process involves acquiring an occupied grid map of a multi-layered map, and then outputting multi-task prediction results through an encoder-decoder network architecture. The map height, width, and channel data of the occupied grid map are used as inputs. The map channel data includes the occupancy probability and the gradient of the signed distance. The multi-task prediction results include obstacle semantic segmentation, obstacle distance estimation, boundary clarity, and accessibility score.

[0055] Furthermore, the encoder includes shallow feature extraction, spatial downsampling, mid-level feature extraction, deep feature extraction, and global context extraction; The decoder includes initial upsampling and skip connections, secondary upsampling and skip connections, and tertiary upsampling and skip connections.

[0056] The input G of the encoder-decoder network is represented as: map height data H × map width data W × map channel data C.

[0057] Map height data represents the number of grid cells in the vertical direction of the raster map; map width data represents the number of grid cells in the width direction of the raster map. The encoder includes shallow feature extraction, spatial downsampling, mid-level feature extraction, deep feature extraction, and global context extraction, specifically: Level 1 performs shallow feature extraction. ; in, This is the first layer feature, with dimensions H×W×32; Indicates the convolution operation; The weights are 3×3 convolutional kernels with a size of 3×3×C×32, which are learned through training. The bias vector is obtained through training. This is the ReLU activation function.

[0058] Level 2 performs spatial downsampling. ; This is the second layer feature, with dimensions of (H / 2)×(W / 2)×64; This represents 2×2 max pooling with a step size of 2; The weights are 3×3 convolutional kernels with a size of 3×3×32×64. The bias vector is obtained through training.

[0059] Level 3 performs mid-level feature extraction. ; This is the third layer feature, with dimensions of (H / 4)×(W / 4)×128; The weights are 3×3 convolutional kernels with dimensions of 3×3×64×128. The bias vector is obtained through training.

[0060] Level 4 performs deep feature extraction. ; This is the fourth layer feature, with dimensions of (H / 8)×(W / 8)×256; The weights are 3×3 convolutional kernels with dimensions of 3×3×128×256. The bias vector is obtained through training.

[0061] Level 5 performs global context extraction. ; The bottleneck layer feature has a size of (H / 8)×(W / 8)×256. The weights are 1×1 convolutional kernels with a size of 1×1×256×256; The bias vector is obtained through training.

[0062] The decoder includes first upsampling and skip connections, second upsampling and skip connections, and third upsampling and skip connections, specifically: Level 6 performs the first upsampling and skip connections. ; This is the sixth layer feature, with dimensions of (H / 4)×(W / 4)×128; The weights are 3×3 convolutional kernels with a size of 3×3×(256+128)×128. This indicates an output dimension concatenation operation; This indicates bilinear interpolation upsampling, amplified by 2 times.

[0063] Level 7 performs secondary upsampling and skip connections. ; This is the seventh layer feature, with dimensions of (H / 2)×(W / 2)×64; The weights are 3×3 convolution kernels with a size of 3×3×(128+64)×64.

[0064] Level 8 undergoes three upsampling operations and skip connections. ; This is the eighth layer feature, with dimensions of H×W×32; The weights are 3×3 convolution kernels with a size of 3×3×(64+32)×32.

[0065] Multi-task prediction results include obstacle semantic segmentation, obstacle distance estimation, boundary clarity, and accessibility score; Among them, level 9-1 performs obstacle semantic segmentation. ; The semantic segmentation result is represented by a size of H×W×3, which represents the probability of each grid cell belonging to the background, static obstacle, or dynamic obstacle. The weights are 1×1 convolutional kernels with dimensions of 1×1×32×3.

[0066] Level 9-2 performs obstacle distance estimation. ; This represents the distance estimation result, with dimensions H×W×1; ReLU indicates that the distance is guaranteed to be non-negative; The weights are 1×1 convolutional kernels with a size of 1×1×32×1.

[0067] Level 9-3 performs boundary sharpness estimation. ; This indicates the sharpness of the boundary, with dimensions of H×W×1. A larger value indicates a sharper boundary. sigmoid means compressing the output to the range [0,1]. The weights are 1×1 convolutional kernels with a size of 1×1×32×1.

[0068] Accessibility is assessed at levels 9-4. ; This represents the passability score, with dimensions H×W×1. A larger value indicates that it is easier to pass. The weights are 1×1 convolutional kernels with a size of 1×1×32×1.

[0069] The environmental perception module uses neural networks to achieve accurate obstacle recognition, classification, and boundary localization, providing structured environmental information for path planning. The multi-task prediction results of the environmental perception module are fed back to the occupied grid map to verify the occupancy probability of the occupied grid map.

[0070] Path planning methods based on multi-layer maps include: S41. Obtain multi-layer map data and drone status; S42. Obtain the global path based on the global planner; S43. Based on the improved time-domain elastic band planner, a local path is obtained; S44. By combining the global path and the local path, the UAV's trajectory is obtained.

[0071] Furthermore, the local path obtained based on the improved time-domain flexible band planner includes: A pre-route trajectory is constructed based on an improved temporal elastic band planner, which is represented as a series of pose sequences. Then, the optimal UAV pose sequence is solved based on obstacle cost, dynamic constraint cost, time optimality cost, and global path consistency cost to obtain the local path.

[0072] Furthermore, the pose sequence is represented as: , ; in, The coordinates of the drone's position; Indicates the yaw angle; Represents a timestamp; Construct the optimization equation: ;in For the cost of obstacles; This refers to the velocity constraint cost within the dynamic constraint cost; This refers to the acceleration constraint cost within the dynamic constraint cost; This comes at the cost of time optimality. This is due to the cost of global path consistency; Weighting coefficient , , , , Adjustments need to be made based on the actual scenario. For example, in areas with dense obstacles, increase... In areas where rapid passage is required, increase... .

[0073] in, ; Express posture To the obstacle The signed distance, obtained from the ESDF map, reflects the distance from the current location to the nearest obstacle; but The smaller the value, the exponentially higher the cost, thus ensuring safety; It is expressed as the attenuation coefficient, which is set according to the size of the drone and safety requirements. It is usually taken as 1 / 3 of the safe distance and is used to control the sensitivity of the cost to changes with distance. The cost of speed constraint is expressed as ; The cost of acceleration constraints is expressed as ; The time optimality cost is expressed as ; The cost of global path consistency is expressed as: ; to ensure that the local trajectory does not deviate too far from the global path; in, , The instantaneous motion is calculated by the difference of trajectory points. , The maximum speed and acceleration of the drone are determined by its physical characteristics.

[0074] Express posture To global path The closest distance.

[0075] The global path is obtained based on the global planner; the global planner uses the A* algorithm, denoted as: ; The actual path length is obtained through cumulative calculation using Dijkstra's algorithm. Euclidean distance estimation guides the search direction; a global path is obtained through a global planner. Global path planning using the A* algorithm is an existing technology and will not be described in detail in this embodiment.

[0076] The global path generates a coarse path based on prior information, providing macroscopic navigation direction; the local path generates a safe and feasible local trajectory by processing sensor data in real time. The improved TEB algorithm achieves both safety and dynamic feasibility and time optimality through multi-objective optimization.

[0077] This invention achieves reliable perception in extreme environments such as complete darkness and weak texture by tightly coupling lidar data and camera image data. Compared with traditional methods, it can not only detect the geometric existence of obstacles, but also identify their semantic categories, enabling UAVs to make more forward-looking decisions, identify narrow passages in advance and adjust their flight attitude, significantly improving passability and safety in complex structures such as mines and box girders.

[0078] This invention constructs a multi-layered map system. The multi-layered map architecture ensures that the UAV can maintain centimeter-level positioning accuracy even in environments without satellite signals. Other modules can extract maps from different layers, improving information processing efficiency and solving the problem of insufficient adaptability of a single map representation in complex environments.

[0079] This invention provides a complete technical solution for fully autonomous operation of UAVs in extremely narrow environments without satellite signals through multi-level environmental perception, precise map building, and intelligent path planning, significantly improving the application value and reliability of UAVs in complex industrial scenarios.

[0080] This embodiment can divide the UAV system in the satellite signal denial area into functional modules according to the above method example. For example, each module can correspond to a separate function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0081] When using integrated units, an unmanned aerial vehicle (UAV) system in a satellite-denied area can include a processing module and a storage module. The processing module is used to control and manage the actions of the UAV system in the satellite-denied area. The storage module is used to support the execution of mutual program code and data by the UAV system in the satellite-denied area.

[0082] The processing module may be a processor or a controller, which can implement or execute various exemplary logic blocks, modules, and circuits as disclosed in this application. The processor may also be a combination of computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and microprocessors, etc., and the storage module may be a memory.

[0083] This embodiment also provides a computer-readable storage medium (including but not limited to disk storage, CD-ROM, optical storage, etc.) storing computer program code. When the computer program code is run on a computer, the computer executes the above-mentioned related method steps to realize the UAV system in the satellite signal denial area provided in the above embodiment.

[0084] This embodiment also provides a computer program product. When the computer program product is run on a computer, it causes the computer to perform the aforementioned steps to realize the UAV system for satellite signal denial areas provided in the above embodiment. The beneficial effects of the above embodiments can be found in the corresponding methods described above, and will not be repeated here.

[0085] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0086] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. In the description of this disclosure, it should be understood that if terms such as "upper," "lower," "front," "rear," "left," and "right" are used to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, they are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the indicated position or element must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this disclosure.

[0087] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0088] The above are merely embodiments of this disclosure and are not intended to limit the scope of this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of the claims of this disclosure.

Claims

1. A drone system for a satellite signal denial zone, characterized in that, include: The system includes a status assessment module, a mapping module, an environmental perception module, a path planning module, and a control module. The status assessment module assesses the drone's status based on multi-source data. The mapping module constructs a multi-layered map based on multi-source data; The environmental perception module acquires multi-layer maps and identifies obstacles in the environment through a neural network model; The path planning module obtains a global path based on a global planner; obtains a local path based on an improved time-domain elastic band planner; and obtains the UAV trajectory by combining the global path and the local path. The control module acquires data from the environmental perception module and the path planning module to control the flight of the drone.

2. A drone system for a satellite signal denial zone according to claim 1, wherein, The state assessment module acquires multi-source data, including lidar data, camera images, and IMU data. Based on IMU data, the pre-integration error is obtained through the IMU pre-integration model; Based on camera images, the landmark points are transformed from the world coordinate system to the camera coordinate system to obtain the reprojection error; Based on the point cloud formed by lidar data, the lidar point cloud matching error is obtained for each lidar point to the nearest plane in the target point cloud.

3. A drone system for a satellite signal denial zone according to claim 2, wherein, Based on the covariance matrix corresponding to the pre-integration error, the weighted least squares function of the pre-integration error is obtained; Based on the covariance matrix corresponding to the reprojection error, the weighted least squares function of the reprojection error is obtained; Based on the covariance matrix corresponding to the lidar point cloud matching error, the weighted least squares function of the lidar point cloud matching error is obtained. The optimal UAV state is evaluated to minimize the sum of the weighted least squares functions corresponding to the pre-integration error, reprojection error, and lidar point cloud matching error.

4. A drone system for a satellite signal denial zone according to claim 3, wherein, The multi-layered map includes a grid map and an Euclidean map; Based on the multi-source LiDAR data, the occupancy probability of the grid is obtained through the Bayesian update formula, and the occupancy probability of all grids is collected to obtain the occupied grid map. Based on the occupied grid map, the signed distance from each grid to the nearest obstacle is calculated to obtain the Euclidean map.

5. A drone system for a satellite signal denial zone according to claim 4, wherein, The process of obtaining the occupancy probability of a grid cell based on the multi-source data using a Bayesian update formula, and then aggregating the occupancy probabilities of all grid cells to obtain an occupied grid cell map, includes: Set the occupancy grid map prior probabilities: for each grid , the initial probability ; constructing an inverse sensor model: obtaining laser beam measurements in each of the lidar data ; For each grid on the ray path , Computing a distance function ; wherein, is the world coordinate of the center point of the grid ; and is the position of the sensor in the current world coordinate system Computing relative position ; wherein, is the distance of the sensor to the measuring obstacle; Then, the inverse sensor model is ; where, denotes the occupancy probability at the measurement endpoint; denotes the free space probability along the ray path; denotes the front region value of the measurement endpoint; The occupancy probability of the grid is obtained using the Bayesian update formula: ； wherein is the prior probability; P (i | 1 : t) represents the probability that cell i is occupied given the first t observations, P (i | 1 : t-1) represents the probability that cell i is occupied given the first t-1 observations; Update the occupied grid map: By updating the occupied probability of the grid, an updated occupied grid map is obtained.

6. The UAV system of a satellite signal denial zone according to claim 4, wherein, The process of calculating the signed distance from each grid cell to the nearest obstacle based on the occupied grid map to obtain the Euclidean map includes: For the grid with signed distance representation ; wherein representing a grid any point in distance to obstacle boundary ; ； wherein, is a point on the obstacle boundary ; denotes and the Euclidean distance between Based on the constructed occupied grid map, the signed distance from each grid cell to the nearest obstacle is calculated to obtain the Euclidean map.

7. The drone system of a satellite signal denial zone according to claim 5, wherein, The process of acquiring a multi-layered map and identifying obstacles in the environment using a neural network model includes: The occupancy grid map of the multi-level map is acquired, and a multi-task prediction result is output through an encoder-decoder network architecture; map height data, map width data and map channel data of the occupancy grid map are taken as inputs; the map channel data includes occupancy probability and gradient of signed distance; the multi-task prediction result includes obstacle semantic segmentation, obstacle distance estimation, boundary definition and passability score.

8. A drone system for a satellite signal denial zone according to claim 7, wherein, The encoder includes shallow feature extraction, spatial down-sampling, middle feature extraction, deep feature extraction and global context extraction. The decoder includes first up-sampling and jump connection, second up-sampling and jump connection and third up-sampling and jump connection. The input G of the encoder-decoder network is represented as: map height data H x map width data W x map channel data C.

9. The drone system of a satellite signal denial zone according to claim 7, wherein, The local path is obtained based on the improved time-domain elastic band planner, and the local path includes: The pre-route trajectory is constructed based on the improved time-domain elastic band planner, and the pre-route trajectory is represented as a series of pose sequences; then, optimal UAV pose sequences are solved based on obstacle cost, dynamics constraint cost, time optimality cost and global path consistency cost, and the local path is obtained.

10. The UAV system of a satellite signal denial zone according to claim 9, wherein, the sequence of poses , ; wherein, is a drone position coordinate; represents a yaw angle; represents a time stamp; An optimization equation is constructed: ; where is an obstacle cost; is a velocity constraint cost in the kinematic constraint cost; is an acceleration constraint cost in the kinematic constraint cost; is a time optimality cost; is a global path consistency cost; wherein ; represents a pose to an obstacle signed distance; is represented as an attenuation coefficient; The speed constraint cost is represented as ; The acceleration constraint cost is expressed as ; The time optimality cost is expressed as ; The global path consistency cost is expressed as: ; represents the closest distance of the pose to the global path .

Images