Active micro-maneuvering-based SLAM methods, UAVs, media, and application products
By identifying uncertain areas of both static and dynamic states and performing micro-maneuvers to filter dynamic targets, the problem of distinguishing dynamic feature points in UAV SLAM is solved, improving positioning accuracy and mapping reliability, reducing computational consumption and image blurring, and ensuring flight stability.
Patent Information
- Application Number
- CN202611282782.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-24
- Publication Date
- 2026-11-17
AI Technical Summary
Existing UAV SLAM methods struggle to effectively distinguish between dynamic feature points and static scene feature points when dealing with dynamic targets whose direction of motion is similar to that of the UAV or whose speed is slow. This leads to increased pose accumulation error and local map distortion.
By acquiring the current frame image, inertial measurement data, and SLAM state information, dynamic and static uncertain regions are identified, and multiple candidate micro-maneuvers are executed to amplify the motion differences of feature points. The dynamic target region is screened using reprojection residuals and theoretical optical flow vectors, dynamic target feature points are eliminated, and static feature points are added.
It improves the recognizability of dynamic and static features, reduces pose accumulation error, enhances the positioning accuracy and mapping reliability of the SLAM system, reduces computational consumption and image blurring and distortion, and ensures flight stability and safety.
Smart Images

Figure CN123111231A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing, and more particularly to a SLAM method based on active micro-maneuvering, unmanned aerial vehicles, media, and program products. Background Technology
[0002] With the rapid development of drone technology, its application in complex spatial scenarios such as low-altitude logistics, environmental inspection, and disaster relief is becoming increasingly widespread. In these applications, vision-based simultaneous localization and mapping (Visual SLAM) technology is the core foundation for achieving fully autonomous navigation and environmental perception for drones. Visual SLAM technology can rely solely on the camera equipment on the system to scan the surrounding environment while moving in an unknown environment, thereby constructing a three-dimensional global coordinate map of the surrounding environment and simultaneously calculating the drone's real-time position and attitude in space, providing crucial underlying data support for the drone's autonomous flight decisions.
[0003] In related technologies, drones typically fly along a preset route to collect images. Based on the changes in the observed positions of each feature point between consecutive image frames, their motion consistency index is calculated. Feature points whose motion consistency index deviates from the static scene model are identified as dynamic target feature points and filtered out. The remaining feature points are used for pose estimation and map construction.
[0004] However, as the types of dynamic targets in the operating environment increase and their motion forms become more complex, the SLAM methods of related technologies, when dealing with dynamic targets whose motion direction is similar to that of the UAV, or dynamic targets with slower motion speeds, show that the changes in the observed position of such dynamic targets between consecutive image frames are similar to those of static scenes. The motion consistency index of the corresponding feature points falls into the normal fluctuation range of static scenes, and participates in pose estimation and map construction together with the feature points of static scenes, resulting in an increase in pose accumulation error and local map distortion. Summary of the Invention
[0005] This application provides a SLAM method, UAV, medium, and program product based on active micro-maneuvering, which addresses how to improve the distinguishability of dynamic target feature points with similar motion directions or slower motion speeds to static scene feature points, thereby reducing pose accumulation errors and local map distortion caused by such dynamic target feature points.
[0006] In a first aspect, this application provides a SLAM method based on active micro-maneuvering, applied to unmanned aerial vehicles (UAVs), the method comprising:
[0007] Acquire the current frame image, the UAV's inertial measurement data, and SLAM state information, including the UAV's current pose and local position. Figure 3 Tracking status of dimensional points and feature points;
[0008] Based on the current pose of the drone, the local area Figure 3 The feature points are projected onto the current frame image, and the dynamic and static uncertain regions in the current frame image are determined based on the reprojection residual obtained from the projection and the tracking status of the feature points.
[0009] When the number of feature points in the dynamic and static uncertainty region exceeds a preset threshold, the dynamic gain score of each feature point in the dynamic and static uncertainty region is predicted after the UAV performs multiple candidate micro-maneuvers. Based on the dynamic gain score, the target micro-maneuver is determined from the multiple candidate micro-maneuvers. The candidate micro-maneuvers include translational micro-maneuvers, yaw micro-maneuvers, and altitude micro-maneuvers. The dynamic gain score characterizes the degree of motion difference of feature points in the dynamic and static uncertainty region.
[0010] Under preset flight constraints, the UAV is controlled to perform micro-maneuvers on the target, and target frame images and target inertial measurement data are acquired within the execution time window of the target micro-maneuvers.
[0011] Based on the target inertial measurement data, predict the theoretical observation position of feature points in the target frame image within the dynamic and static uncertainty region, and determine the dynamic target region from the dynamic and static uncertainty region based on the response residual of the actual observation position of the feature points relative to the theoretical observation position.
[0012] Remove the SLAM feature points corresponding to the dynamic target region from the dynamic and static uncertain regions, and update the SLAM state information based on the remaining SLAM feature points.
[0013] By employing the aforementioned technical solution, the UAV actively amplifies the motion differences of feature points within uncertain regions through target micro-maneuvers, thereby separating dynamic targets with motion similar to the environment from the observation response residuals. This method improves the discriminability of dynamic and static features, reduces pose accumulation errors caused by erroneous features, and directly enhances the local localization and global mapping accuracy of its own SLAM system.
[0014] In some embodiments, the step of determining the dynamic and static uncertain regions in the current frame image based on the reprojection residual obtained from the projection and the feature point tracking state specifically includes:
[0015] Based on the current pose of the drone, the local area Figure 3 The feature points are projected onto the current frame image, and the directed line segment pointing from the reprojected pixel position of each feature point to the actual observed pixel position is determined as the reprojection residual vector of the feature point.
[0016] Based on the inertial measurement data, predict the theoretical optical flow vector of each feature point in the current frame image under a purely static scene;
[0017] Feature points whose angle between the reprojection residual vector and the theoretical optical flow vector is greater than a preset angle threshold, and whose number of consecutive tracking frames in the feature point tracking state is greater than a preset frame threshold, are identified as suspicious feature points.
[0018] Density clustering is performed on the suspicious feature points, and the resulting clusters of feature points are expanded outward into their pixel neighborhoods according to a preset ratio to obtain the dynamic and static uncertain region.
[0019] By employing the aforementioned technical solution, the UAV initially screens for anomalies based on the difference between the reprojection residual and the theoretical optical flow, and then uses density clustering and expansion operations to reasonably enclose potential dynamic targets. This allows subsequent maneuver strategies to focus only on highly suspicious and uncertain regions, ensuring a high success rate in capturing dynamic targets while reducing the frequency of blind detection and overall computational costs.
[0020] In some embodiments, the step of predicting the dynamic gain score of each feature point in the dynamic-static uncertainty region after the UAV performs multiple candidate micro-maneuvers specifically includes:
[0021] Based on the maneuver parameters corresponding to each candidate micro-maneuver and the current pose of the UAV, predict the camera's predicted pose after the UAV performs each candidate micro-maneuver.
[0022] Based on the camera predicted pose and the local area Figure 3 The depth of each feature point in the dimensional point is used to predict the optical flow vector of each feature point in the purely static scene within the dynamic and static uncertainty region.
[0023] Based on the component amplitude of the predicted optical flow vector of each feature point in the direction perpendicular to the reprojection residual vector, the motion difference indication value of each feature point is obtained.
[0024] The sum of the motion difference indication values of all the feature points within the dynamic and static uncertainty region is determined as the dynamic gain score of the candidate micro-maneuver.
[0025] By employing the above technical solution, the UAV predicts the optical flow distribution based on candidate maneuvers and extracts the optical flow component in the vertical residual direction as a difference index. This method quantifies the ability of different maneuvers to break the current motion ambiguity of the system, so as to select observation angles that are more likely to expose dynamic targets and ensure that better separation of dynamic and static targets can be achieved with less flight cost.
[0026] In some embodiments, the step of controlling the UAV to perform the target micro-maneuver under preset flight constraints specifically includes:
[0027] Obtain the standard control quantity of the mission trajectory currently being performed by the UAV within the execution time window of the target micro-maneuver, and solve the maneuver parameters corresponding to the target micro-maneuver into incremental control quantities relative to the standard control quantity;
[0028] When the composite control quantity obtained by superimposing the incremental control quantity and the standard control quantity satisfies the preset flight constraint, the UAV is controlled to perform the target micro-maneuver according to the composite control quantity.
[0029] Within the execution time window, the planned inertial measurement data of the UAV is predicted based on the synthetic control quantity, and the actual collected target inertial measurement data is compared with the planned inertial measurement data to obtain the execution disturbance deviation;
[0030] When the execution disturbance deviation is less than a preset disturbance threshold, the dynamic target region is determined based on the target frame image and the target inertial measurement data;
[0031] When the execution disturbance deviation is greater than or equal to the preset disturbance threshold, the target frame image acquired within the current execution time window is discarded.
[0032] By adopting the above technical solution, the UAV incrementally adds micro-maneuvers to the original control, while still ensuring the execution of the main flight mission. Simultaneously, a deviation monitoring mechanism is introduced to promptly discard distorted data caused by external interference such as wind. This improves the stability and safety of the UAV's autonomous flight control and also ensures that the extraction process of dynamic target response residuals has sufficiently high numerical reliability.
[0033] In some embodiments, the step of determining the dynamic target region from the dynamic-static uncertainty region based on the response residual of the actual observed position of the feature point relative to the theoretical observed position specifically includes:
[0034] The directed line segment pointing from the actual observation position of each feature point within the dynamic and static uncertainty region to the corresponding theoretical observation position is determined as the response residual vector of the feature point;
[0035] Feature points whose magnitude of the response residual vector is greater than a preset residual threshold are identified as candidate dynamic feature points;
[0036] The candidate dynamic feature points are jointly clustered based on the directional similarity of the response residual vector and the pixel position proximity to obtain multiple clusters. The candidate dynamic feature points with different response residual vector directions correspond to different clusters.
[0037] Each pixel region covered by a cluster is defined as a dynamic target region.
[0038] By employing the aforementioned technical solution, the UAV extracts residual abnormal points and performs joint clustering using directional similarity and spatial proximity. This method aggregates discrete feature points with local motion coordination into physical entities, thereby dividing them into several independent dynamic regions, improving the fine-grained recognition and environmental scene segmentation accuracy in complex scenarios with multiple moving entities operating concurrently.
[0039] In some embodiments, the step of acquiring target frame images and target inertial measurement data within the execution time window of the target micro-maneuver specifically includes:
[0040] After the target micro-maneuver is started, the cumulative displacement of the UAV is monitored in real time, and the time interval from the start of the maneuver to the moment when the cumulative displacement reaches the preset observation parallax threshold is determined as the execution time window;
[0041] Within the execution time window, multiple frames of intermediate image sequences and intermediate inertial measurement data synchronized with the intermediate image sequences are continuously acquired;
[0042] Based on the attitude change parameters corresponding to the intermediate inertial measurement data, the intermediate image with the attitude change parameters less than a preset anti-shake threshold and the smallest attitude change parameters is extracted from the intermediate image sequence to obtain the target frame image;
[0043] All inertial measurement data generated within the execution time window are determined as the target inertial measurement data.
[0044] By adopting the above technical solution, the UAV defines the image acquisition window based on displacement, ensuring that the image sequence has the visual parallax required for processing. Simultaneously, image selection and calibration based on anti-shake thresholds reduce image streaking or blurring caused by maneuvering. This results in higher texture clarity for the target frame itself, effectively reducing photometric observation noise in the active image acquisition process.
[0045] In some embodiments, after the steps of removing SLAM feature points corresponding to the dynamic target region from the dynamic-static uncertainty region and updating the SLAM state information based on the remaining SLAM feature points, the method further includes:
[0046] The adjacent pixel bands within a preset range outside the dynamic target area are defined as the static supplementary area;
[0047] The number of remaining SLAM feature points is evaluated, and the target number of supplementary feature points corresponding to the remaining SLAM feature points is determined according to a preset tracking number threshold.
[0048] Within the static supplementation region, new feature points are extracted based on the target quantity of the supplementation feature points to obtain at least one static supplementation feature point;
[0049] The static supplementary feature points are added to the feature point tracking queue in the SLAM state information, and the feature point tracking queue is used for SLAM state updates in subsequent frames.
[0050] By adopting the above technical solution, after dynamic features are removed, the UAV actively mines and supplements new feature points from the outer static pixel band. This operation enables the system's tracking queue to quickly obtain high-quality feature backfilling, enriching the three-dimensional topology of visual tracking and reducing the probability of visual odometry interruption or local trajectory drift induced by a sudden decrease in tracking features.
[0051] Secondly, this application provides a drone, the drone comprising: one or more processors and a memory;
[0052] The memory is coupled to the one or more processors. The memory is used to store computer program code, which includes computer instructions. The one or more processors call the computer instructions to enable the UAV to implement the SLAM method based on active micro-maneuvering provided in the above embodiments, which will not be described in detail here.
[0053] Thirdly, this application provides a computer-readable storage medium including instructions that, when executed on a drone, enable the drone to implement a SLAM method based on active micro-maneuvering provided in the above embodiments, which will not be elaborated here.
[0054] Fourthly, this application provides a computer program product, including a computer program that, when run on a drone, enables the drone to implement a SLAM method based on active micro-maneuvering provided in the above embodiments, which will not be elaborated here.
[0055] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:
[0056] 1. By predicting the optical flow distribution of different candidate micro-maneuvers in uncertain regions and extracting the optical flow component perpendicular to the residual direction, the maneuver effectiveness is quantitatively evaluated, thereby selectively identifying the optimal micro-maneuver. This method can actively amplify the real motion differences between dynamic and static features with minimal flight cost, accelerating the exposure of highly deceptive dynamic targets that move in the same direction as or at a slower pace as the UAV, thus improving the natural identifiability of dynamic and static targets from the underlying observation data level.
[0057] 2. First, a cross-dimensional comparison is made between the reprojection residual and the theoretical optical flow. Density clustering and outward expansion operations are then used to directly converge the detection field of view to high-suspicion uncertainty regions, ensuring capture rate while reducing computational consumption caused by global detection. After stimulating the response residual, further joint clustering of anomalous feature points is performed based on directional similarity and spatial proximity. This process materializes and aggregates discrete feature points with local cooperative motion characteristics, improving the fine-grained scene stripping in multi-moving target concurrent environments and making the definition of each independent dynamic region more precise.
[0058] 3. Micro-maneuvering is flexibly superimposed onto standard control in an incremental manner, and disturbance deviation monitoring and parallax-based anti-shake screening are introduced. This reduces image streaking and observation noise caused by external wind disturbances and forced maneuvers, ensuring the clarity of target observation images and the smoothness and safety of flight. In addition, for the tracking gap caused by the removal of dynamic features, the UAV can mine features in the continuous static pixel band outside the dynamic area to fill in the gap, enriching the topology of SLAM feature tracking and reducing the risk of trajectory drift and tracking interruption caused by a sudden reduction in effective features. Attached Figure Description
[0059] Figure 1 This is a flowchart illustrating a SLAM method based on active micro-maneuvering in an embodiment of this application;
[0060] Figure 2 This is another flowchart illustrating a SLAM method based on active micro-maneuvering in an embodiment of this application;
[0061] Figure 3 This is a schematic diagram of a process for an unmanned aerial vehicle (UAV) to identify a dynamic target region in the current frame image in an embodiment of this application.
[0062] Figure 4 This is a schematic diagram of the physical device structure of a drone in an embodiment of this application. Detailed Implementation
[0063] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.
[0064] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.
[0065] During drone operations such as low-altitude logistics, environmental inspection, and disaster relief, visual SLAM systems need to continuously identify and remove dynamic target feature points in the scene to ensure the accuracy of pose estimation and map building. However, when the direction of motion of a dynamic target is very similar to the direction of motion of the drone itself, or when the speed of the dynamic target is extremely slow, the displacement changes of feature points of such targets between consecutive image frames are very similar to the static background. Traditional motion consistency detection methods are unable to effectively distinguish them from static feature points, resulting in these "deceptive" dynamic feature points being mixed into the pose estimation and map building process, causing problems such as increased pose accumulation error and local map distortion.
[0066] To address the aforementioned issues, this application provides a SLAM method based on active micro-maneuvering. During normal flight, the UAV first identifies uncertain regions with questionable dynamic and static attributes from the current sensing data. Then, it actively plans and executes a small attitude maneuver, utilizing the resulting change in observation perspective to actively amplify the motion differences between dynamic and static feature points. Next, based on the response residuals before and after the maneuver, the dynamic target region is accurately located. Finally, the target is removed from the SLAM tracking queue and replaced with static feature points. This achieves the active exposure and precise elimination of difficult-to-detect dynamic targets without interrupting the main flight mission, improving the positioning accuracy and mapping reliability of the SLAM system. Please refer to [link / reference] for details. Figure 1 This is a flowchart illustrating a SLAM method based on active micro-maneuvering in an embodiment of this application.
[0067] S101. Acquire the current frame image, the UAV's inertial measurement data, and SLAM status information.
[0068] The current frame image refers to a complete digital image captured by the camera equipment onboard the UAV at the current moment; inertial measurement data refers to the data measured and output in real time by the inertial measurement unit (IMU) onboard the UAV, including the linear acceleration and angular velocity of the UAV in three axes, reflecting the UAV's own motion state in a short period of time; SLAM state information refers to the comprehensive description of the UAV's own state and the surrounding environment state currently maintained by the SLAM system, where the UAV's current pose refers to the UAV's current three-dimensional position and three-dimensional attitude (orientation) in the world coordinate system, locally... Figure 33D points refer to map landmarks with three-dimensional spatial coordinates that have been reconstructed by the SLAM system near the UAV's recent flight path. These 3D points correspond to the observable feature points in the image. Feature point tracking status refers to the historical record of each feature point that the SLAM system is currently tracking, including information such as the number of frames that each feature point has been continuously and stably tracked.
[0069] Specifically, the drone's camera continuously acquires images at a fixed frame rate. When a new frame arrives, the system synchronously reads the IMU measurement data aligned with the timestamp of that frame and retrieves the latest maintained current pose and local position data from the SLAM system's internal state cache. Figure 3 The set of dimensional points and the tracking status record of all active feature points.
[0070] Optionally, the UAV uses the camera frame trigger as a time base to record the timestamp of the current frame image; extracts the inertial measurement data segment closest to the current frame timestamp from the IMU data buffer queue; and reads the latest pose, 3D point cloud, and feature tracking records written up to the completion of the previous frame processing from the SLAM state management module. It is understood that other synchronization mechanisms can also be used to achieve the aligned acquisition of multi-sensor data (e.g., a hardware-triggered timestamp synchronization scheme), which is not limited here.
[0071] S102, Based on the current pose, the local area... Figure 3 The dimensional points are projected onto the current frame image, and the dynamic and static uncertain regions in the current frame image are determined based on the reprojection residuals obtained from the projection and the feature point tracking status.
[0072] Among them, reprojection residual refers to the reprojection of local residuals. Figure 3 The difference between the theoretical pixel position obtained by projecting the current pose of the UAV onto the current frame image and the actual observed pixel position of the feature point in the current frame image is represented by the direction and magnitude of the difference, respectively, reflecting the direction and degree of deviation of the feature point from the static projection expectation. The dynamic-static uncertain region refers to the image region in the current frame image that is determined to have questionable dynamic-static attributes after preliminary analysis and cannot be clearly classified as static or dynamic based solely on the current frame data. The feature points in this region need to be further identified through subsequent active micro-maneuvers.
[0073] Specifically, the drone, based on its current pose and camera intrinsic parameters, will locally... Figure 3Each feature point is projected back into the pixel coordinate system of the current frame along the ray direction to obtain the theoretical projection position of each tracked feature point. This theoretical projection position is then compared with the actual pixel position detected by the feature tracking algorithm in the current frame image to obtain the reprojection residual, which reflects the consistency of the static projection of the feature point. In the ideal case where only static map points are present and the UAV pose estimation is accurate, the reprojection residual should be mainly caused by observation noise, with a small amplitude and random direction. However, when the entity corresponding to a feature point has real motion in the scene, its residual will show a significant directional deviation along the object's motion direction. The UAV further combines information such as the number of consecutive tracking frames in the feature point tracking state to filter out feature points whose residuals show directional anomalies from those that have been tracked stably for a long time and should not suddenly produce large static projection deviations. These scattered suspicious feature points are then aggregated and expanded into several continuous image sub-regions through spatial aggregation operations, serving as the dynamic and static uncertain regions output in this step.
[0074] In some embodiments, the drone can utilize its current pose and camera projection model to locally Figure 3 Each feature point is projected onto the pixel plane of the current frame, and the directed line segment formed by the reprojected pixel position of each feature point pointing to its actual observed pixel position is denoted as the reprojection residual vector of that feature point. Next, the UAV predicts the theoretical optical flow vector that each feature point should present in the current frame under the assumption of a purely static scene based on the short-term pose increment of the camera obtained by IMU integration. The angle between the theoretical optical flow vector and the reprojection residual vector is compared, and feature points with an angle greater than a preset angle threshold and a consecutive tracking frame count greater than a preset frame count threshold are marked as suspicious feature points. Finally, the UAV performs density clustering on the pixel plane for the suspicious feature points, and expands the pixel neighborhood of the clustered feature point clusters outward according to a preset ratio to encompass the possible complete outline of the target, thereby obtaining the dynamic and static uncertain region to be output in this step. It is understood that other methods, such as prior region removal based on semantic segmentation masks or uncertainty region selection based on optical flow field divergence anomalies, can also be used to determine the dynamic and static uncertain region; this application embodiment does not limit this approach.
[0075] In other specific implementations, the UAV can further use the theoretical optical flow vector calculated from the inertial measurement data as a reference benchmark for static motion. The directional difference between the reprojection residual vector and the theoretical optical flow vector, as well as the number of consecutive tracking frames in the feature point tracking state, are used as screening conditions. In essence, the theoretical optical flow vector still comes from the inertial measurement data's representation of the UAV's own pose changes.
[0076] S103. When the number of feature points in the dynamic and static uncertainty area exceeds the preset number threshold, predict the dynamic gain score of each feature point in the dynamic and static uncertainty area after the UAV performs multiple candidate micro-maneuvers, and determine the target micro-maneuver from multiple candidate micro-maneuvers based on the dynamic gain score.
[0077] It should be noted that the "micro-maneuver" in the embodiments of this application refers to a short-cycle maneuver with an amplitude not exceeding a preset amplitude threshold and a duration not exceeding a preset duration threshold, which achieves a balance between flight cost and perception gain: on the one hand, the maneuver amplitude is small enough not to significantly deviate from the original mission trajectory; on the other hand, the maneuver amplitude is sufficient to introduce a solvable observation parallax within the execution time window.
[0078] Among them, candidate micro-maneuvers are used to represent various small-amplitude, short-period, and superimposed maneuver schemes prepared by the UAV to actively stimulate the motion differences between feature points in a dynamic and static uncertain region. These schemes include, but are not limited to, translational micro-maneuvers, yaw micro-maneuvers, and altitude micro-maneuvers. Translational micro-maneuvers refer to the UAV applying a small lateral displacement in the horizontal direction; yaw micro-maneuvers refer to the UAV applying a small yaw angle around its vertical axis; and altitude micro-maneuvers refer to the UAV applying a small vertical displacement in the vertical direction. The dynamic gain score is used to represent a comprehensive quantitative measure of the degree of motion difference that all feature points in the dynamic and static uncertain region can exhibit under the assumption of a certain candidate micro-maneuver. The higher the score, the more beneficial the candidate micro-maneuver is to expose truly dynamic feature points from the static background. The preset number threshold is a lower limit on the number of feature points set to avoid high-cost micro-maneuver identification of regions containing only a few suspicious feature points.
[0079] Specifically, after confirming that the number of feature points in the current frame's dynamic-static uncertainty region exceeds a preset threshold, the UAV retrieves candidate micro-maneuvers from a predefined micro-maneuver library, including translation, yaw, and altitude. For each candidate micro-maneuver, kinematic prediction is performed without actual execution; that is, the subtle changes in camera perspective under the assumption of the candidate micro-maneuver are deduced. Based on this, the pixel displacement pattern of each feature point in the dynamic-static uncertainty region under a purely static scene assumption is predicted. Then, the UAV uses the difference between this displacement pattern and the reprojection residual direction obtained in step S102 as a metric to calculate the sum of the "additional motion differences" that each candidate micro-maneuver can cause in the region, and uses this sum as the dynamic gain score of the candidate micro-maneuver. In some embodiments, the UAV can determine the candidate micro-maneuver corresponding to the maximum dynamic gain score as the target micro-maneuver. Of course, in other embodiments, the UAV can also determine the candidate micro-maneuver with a dynamic gain score greater than a preset maneuver activation threshold as the target micro-maneuver. The preset maneuver activation threshold can be set by relevant technical personnel and actual needs, and is not limited here.
[0080] Understandably, different directions and types of micro-maneuvers have varying capabilities in revealing the same dynamic target: when the direction of the viewpoint change caused by the micro-maneuver happens to be orthogonal to the existing suspicious movement direction of feature points within the uncertain region, the feature points of the real dynamic target will deviate most significantly from the static prediction under the new viewpoint, thus being most helpful in distinguishing it from the static background in subsequent steps. Based on this, the UAV selects the micro-maneuver with the highest dynamic gain score from all candidate micro-maneuvers as the target micro-maneuver, thereby obtaining the best dynamic and static identification benefits with the least flight cost.
[0081] In some embodiments, the UAV can compare the number of feature points in the dynamic and static uncertain region obtained in step S102 with a preset threshold. If the threshold is not exceeded, the original mission trajectory is maintained and the active micro-maneuver process is skipped. If the threshold is exceeded, the candidate micro-maneuver evaluation process is entered. Then, the UAV constructs corresponding maneuver parameters for each candidate micro-maneuver according to a preset maneuver parameter table (such as the displacement direction and amplitude of translational micro-maneuver, the yaw direction and yaw angle of yaw micro-maneuver, and the rise and fall direction and rise and fall amplitude of altitude micro-maneuver). Based on the current pose, the predicted pose of the UAV body after each candidate micro-maneuver is calculated, and then the corresponding camera predicted pose is obtained by combining the body-camera extrinsic parameters. Next, the UAV calculates the predicted pose of the camera body and the local area based on the predicted pose of the camera body and the local area. Figure 3The depth information corresponding to each feature point in the dimensional point is used to infer the predicted optical flow vector of each feature point in the image after the candidate micro-maneuver using the static projection geometric model. The amplitude of the component of the predicted optical flow vector in the direction perpendicular to the reprojection residual vector obtained in step S102 is used as the motion difference indication value of the feature point. Finally, the UAV sums the motion difference indication values of all feature points in the dynamic and static uncertainty area and uses the summation result as the dynamic gain score of the corresponding candidate micro-maneuver. Finally, the scores of all candidate micro-maneuvers are sorted, and the candidate micro-maneuver with the highest score is selected as the target micro-maneuver and output to the subsequent flight control execution and image acquisition stages.
[0082] It should be noted that after selecting the one with the highest dynamic gain score, the UAV will compare it with a preset maneuver activation threshold. If the highest dynamic gain score is still less than or equal to the preset maneuver activation threshold, it indicates that the feature points in the current environment are essentially a purely static background, or that the current limited field of view does not have the physical basis for performing an exposure maneuver. In this case, the UAV will cancel the current active micro-maneuver and maintain the original normal SLAM update, and will restart the active micro-maneuver decision-making process from step S101 after the next frame image arrives. The preset maneuver activation threshold can be calibrated by those skilled in the art based on the empirical distribution of dynamic gain scores in static scenes.
[0083] S104. Under preset flight constraints, control the UAV to perform target micro-maneuvers and collect target frame images and target inertial measurement data within the execution time window of the target micro-maneuvers.
[0084] Among them, the preset flight constraints represent a set of control boundary conditions that the UAV must meet to ensure its own flight safety and the continuity of the main mission, such as the maximum attitude angular rate, maximum linear acceleration, maximum altitude deviation, maximum horizontal deviation relative to the mission trajectory, and minimum safe distance from obstacles; the execution time window refers to the time interval from when the UAV starts to perform the target micro-maneuver until the change in perspective caused by the target micro-maneuver is sufficient to provide effective observation parallax for subsequent dynamic and static identification; the target frame image refers to a frame of visual image acquired by the UAV within the execution time window, used for difference comparison with the current frame image to expose the dynamic target; the target inertial measurement data refers to the set of all inertial measurement samples that are time-synchronized with the execution time window to which the target frame image belongs and are output in real time by the IMU.
[0085] Specifically, after identifying a target micro-maneuver, the UAV does not directly overwrite the original flight control commands with independent maneuver commands. Instead, it first obtains the standard control quantity that should be issued within the execution time window of the current mission trajectory as a basis, then calculates the maneuver parameters of the target micro-maneuver into incremental control quantities relative to the standard control quantity. The incremental control quantities and the standard control quantities are then superimposed to form a composite control quantity. This superposition mechanism allows the micro-maneuver to act on the UAV in a "flexible addition" rather than a "mission interruption" manner, thus generating the required visual disturbance while continuing to advance the original mission trajectory.
[0086] Before issuing the synthetic control input, the UAV must compare it with the preset flight constraints item by item. The control input will only be executed if all boundary conditions are met; otherwise, the micro-maneuver will be abandoned to avoid flight risks. After the synthetic control input is issued, the UAV's onboard camera continuously acquires images within the execution time window, and the IMU synchronously outputs inertial measurement data. At the same time, the UAV will continuously monitor the deviation between the actual IMU output and the theoretical planned value during the maneuver to determine whether the maneuver has been severely distorted due to external wind disturbances or other factors.
[0087] In some embodiments, the UAV's flight control module queries the mission management module for the standard control quantity sequence that should be issued within the execution time window corresponding to the target micro-maneuver of the current mission trajectory. The micro-maneuver calculation module calculates the incremental control quantity relative to the standard control quantity based on the maneuver parameters of the target micro-maneuver, and then superimposes the two at timestamps to obtain the synthetic control quantity sequence. Before issuing the synthetic control quantity to the motor loop, the safety verification module compares the synthetic control quantity item by item with the preset flight constraints (including maximum attitude angular rate, maximum linear acceleration, maximum deviation from the mission trajectory, etc.). If all are satisfied, the motor is driven to execute the target micro-maneuver according to the synthetic control quantity. If any one exceeds the limit, the micro-maneuver is abandoned and the original mission trajectory is restored. Within the execution time window, the UAV instructs the onboard camera to continuously output images at the original working frame rate and the IMU to continuously output inertial measurement samples at the original sampling frequency. The data caching module timestamps and archives these two data streams.
[0088] After the execution time window ends, the disturbance monitoring module predicts the planned inertial measurement data within the window based on the synthetic control quantity and compares it point by point with the actual collected target inertial measurement data to obtain the execution disturbance deviation. When the deviation is less than the preset disturbance threshold, the collected target frame image and target inertial measurement data are used in step S105; otherwise, the target frame image collected within the current execution time window is discarded and the process is restarted in the next frame. In addition, the execution time window of the UAV can be defined by monitoring the cumulative displacement from the start of the self-maneuver in real time and closing the window when the cumulative displacement reaches the preset observation parallax threshold. Within the window, multiple intermediate images and synchronous intermediate inertial measurement data are continuously collected. Then, based on the attitude change parameters reflected by the intermediate inertial measurement data, an intermediate image with attitude change parameters less than the preset anti-shake threshold is selected as the target frame image, and all inertial measurement samples within the window are used as target inertial measurement data.
[0089] S105. Based on the target inertial measurement data, predict the theoretical observation position of feature points in the target frame image within the dynamic and static uncertainty region, and determine the dynamic target region from the dynamic and static uncertainty region based on the response residual of the actual observation position of the feature points relative to the theoretical observation position.
[0090] Among them, the theoretical observation position is used to represent the pixel position that these feature points should theoretically appear in the target frame image, which is calculated based on the pose change reflected by the target inertial measurement data within the execution time window of the UAV, assuming that the feature points in the dynamic and static uncertainty region are all static backgrounds; the response residual is used to represent the difference between the actual observation position and the corresponding theoretical observation position of each feature point in the target frame image within the dynamic and static uncertainty region, which reflects the direction and degree of deviation of the feature point from the static prediction under active micro-maneuvering excitation; the dynamic target region refers to the image sub-region that is accurately locked after the response residual significantly exceeds the static noise level and that actually has real motion.
[0091] Specifically, the UAV integrates and calculates the relative pose change of the camera during the target's micro-maneuvering based on the target's inertial measurement data. It then uses the current pose as a reference pose and superimposes the relative pose change to obtain the calculated pose at the target frame time. This process involves integrating the local features corresponding to each feature point within the dynamic and static uncertainty region. Figure 3The theoretical observation position of a feature point is obtained by projecting it onto the target frame image along the calculated pose, assuming a static orientation. Simultaneously, the UAV uses a feature point tracking algorithm to obtain the actual observation positions of these feature points on the target frame image, and the difference between the two is used to obtain the response residual. If the feature point is indeed a static background, the response residual should only be caused by observation noise and pose estimation noise, with a small amplitude and random direction. If the object corresponding to the feature point is moving in the actual scene, even if its direction of motion in the current frame was previously masked by its motion in the same direction as the UAV's own motion, it will show a significant directional deviation under the new perspective introduced by the micro-maneuver, and thus the amplitude of the response residual will significantly exceed the static noise level.
[0092] Based on this, the drone filters out feature points with abnormal response residuals from uncertain dynamic and static regions. Then, based on the synergy between these abnormal feature points in the residual direction and image spatial location, they are further aggregated into several independent dynamic target regions. The reason for adopting a joint clustering strategy of directional similarity and positional proximity is that multiple independently moving entities may exist simultaneously in real-world scenarios. Simply relying on pixel positional proximity may incorrectly merge multiple targets with different motion directions. However, joint directional similarity can ensure that feature points within the same cluster truly belong to the same moving entity, thereby improving the fine-grainedness of dynamic target recognition in complex scenes.
[0093] In some embodiments, the UAV can pre-integrate the target inertial measurement data to obtain the relative pose change of the camera from the current frame to the target frame, and then associate the local features associated with each feature point in the dynamic and static uncertainty region. Figure 3 The corresponding theoretical observation position is obtained by projecting the relative pose change and camera intrinsic parameters onto the target frame image. The UAV uses optical flow tracking or descriptor matching to find the actual observation position of these feature points on the target frame image, and determines the directed line segment pointing from the actual observation position of each feature point to the corresponding theoretical observation position as the response residual vector of the feature point. The UAV compares the magnitude of the response residual vector with a preset residual threshold, and determines the feature points with a magnitude greater than the threshold as candidate dynamic feature points, thereby initially eliminating small residual interference caused by noise. The UAV performs joint clustering on all candidate dynamic feature points according to the directional similarity of the response residual vector (e.g., the cosine value of the directional angle is higher than the set similarity threshold) and the pixel position proximity (e.g., the pixel distance is lower than the set proximity threshold). Candidate dynamic feature points with similar motion directions and spatial proximity are aggregated into several clusters, and the pixel area covered by each cluster is determined as an independent dynamic target area.
[0094] S106. Remove the SLAM feature points corresponding to the dynamic target region from the dynamic and static uncertain regions, and update the SLAM state information based on the remaining SLAM feature points.
[0095] Here, SLAM feature points represent the set of all active feature points that the SLAM system is tracking in the current frame and that are participating in pose estimation and map fusion. Each SLAM feature point is associated with a local feature point. Figure 3 Information such as dimension point number, pixel observation sequence, and tracking status.
[0096] Specifically, after determining the dynamic target region in the current frame image, the UAV will traverse all feature points originally maintained in the SLAM feature point tracking queue within the uncertain dynamic and static regions, and then retrieve the feature points that fall into the dynamic target region along with their associated local features. Figure 3 The dimensional point number is removed from the pose optimization input set of this frame; for the remaining SLAM feature points that do not fall into the dynamic target region and are confirmed as credible static, the UAV recalculates its current pose based on the geometric constraints between its actual observation position in the current frame and the associated 3D points, using standard visual SLAM pose optimization methods, and performs local SLAM optimization. Figure 3 The coordinates of the dimensional points are synchronously updated with the feature point tracking status records. Finally, the updated UAV pose and local position are updated. Figure 3 The tracking states of dimensional points and feature points are written back into the SLAM state information, serving as the input basis for the next frame's SLAM processing.
[0097] It is understandable that after removing dynamic feature points, a complete state update needs to be performed on the remaining features. This is because the current pose of the UAV and the current frame image serve both real-time use for autonomous navigation and control and as the basis for reprojection and screening of suspicious features in the next frame. Only by continuously maintaining a high-quality SLAM state can the benefits of removing dynamic targets be accumulated stably over time, thereby truly achieving the goal of suppressing pose accumulation errors and reducing local map distortion.
[0098] The following provides a more detailed description of the process of the method provided in this implementation. Please refer to [link / reference]. Figure 2 This is another flowchart illustrating a SLAM method based on active micro-maneuvering in an embodiment of this application.
[0099] S201. After acquiring the current frame image, the UAV's inertial measurement data, and SLAM state information, the local position is determined based on the UAV's current pose. Figure 3 The feature points are projected onto the current frame image to obtain the reprojection residual vector of the feature points.
[0100] Specifically, the UAV, based on the current pose maintained in the SLAM state information and the pre-calibrated camera projection geometry model, performs local mapping. Figure 3Perspective projection calculations are performed on each 3D point to obtain the theoretical projected pixel position of each point on the pixel plane of the current frame; then, the UAV acquires the same position on the current frame image based on the front-end feature tracking results. Figure 3 The actual observed pixel position of the corresponding feature point is defined by the dimensional point. Finally, the UAV constructs a two-dimensional directed line segment by taking the theoretical projection position of each feature point as the starting point and the actual observation position as the ending point, and denotes this directed line segment as the reprojection residual vector of the feature point.
[0101] In some embodiments, the UAV can extract the rotation matrix and translation vector corresponding to the current pose from the SLAM state information, and load the camera intrinsic parameter matrix to construct a complete perspective projection model; traversing the local area Figure 3 For each 3D point, a rigid body transformation from the world coordinate system to the camera coordinate system is performed sequentially, followed by perspective division on the normalized plane and intrinsic parameter mapping to the pixel plane to obtain the reprojected pixel position of the corresponding feature point. The front-end feature tracking module (such as the KLT optical flow tracker or a descriptor-based matcher) is called to detect the actual observed pixel position of each tracked feature point in the current frame image. A two-dimensional directed line segment is constructed with the reprojected pixel position of each feature point as the starting point and the actual observed pixel position as the ending point, and the line segment is stored in the feature point reprojection residual vector cache.
[0102] S202. Based on inertial measurement data, predict the theoretical optical flow vector of each feature point in the current frame image under a purely static scene.
[0103] The theoretical optical flow vector refers to the pixel displacement vector that each feature point should present between adjacent frames, calculated based on the drone's own motion information, under the premise that all entities corresponding to all feature points in the current frame image belong to the static background. Its direction and magnitude are determined only by the drone's own perspective change.
[0104] Specifically, based on the triaxial acceleration and triaxial angular velocity contained in the inertial measurement data, the UAV pre-integrates and calculates the short-term pose increment between adjacent image frames to obtain the relative pose transformation reflecting the change in its own viewpoint; then, it combines the local features associated with each feature point... Figure 3 By combining the depth information of the dimensional points with the camera projection geometry, and under the assumption of a purely static scene, the theoretical pixel displacement direction and magnitude that each feature point should present in the current frame image are inferred, thus obtaining the theoretical optical flow vector. If the actual scene is indeed purely static, the actual inter-frame pixel displacement of the feature point should closely match the theoretical optical flow vector; if the entity corresponding to a feature point has real motion, its actual displacement will deviate significantly from the theoretical optical flow vector in direction.
[0105] In some embodiments, the UAV can extract all IMU samples from the previous frame to the current frame from the inertial measurement data buffer, and pre-integrate them to obtain the UAV's own rotation and translation increments within this interval; the pose increments in the UAV's body coordinate system are converted to the camera coordinate system using body-camera extrinsic parameters to obtain the corresponding short-time camera pose increments; for each feature point in the tracking, based on its associated local... Figure 3 The depth value of the 3D point in the previous frame and the aforementioned camera pose increment are used to first perform a rigid body transformation on the 3D point along the pose increment, and then project it back to the pixel plane of the current frame to obtain its theoretical new pixel position under the pure static assumption; the direction and length of the line connecting the theoretical new pixel position and the actual observation position in the previous frame are recorded as the theoretical optical flow vector of the feature point and stored in the cache.
[0106] S203. Feature points whose angle between the reprojection residual vector and the theoretical optical flow vector is greater than a preset angle threshold and whose number of consecutive tracking frames in the feature point tracking state is greater than a preset frame number threshold are identified as suspicious feature points.
[0107] Among them, the preset angle threshold is a lower limit of the included angle set in advance to determine that the direction of the reprojection residual vector is significantly inconsistent with the direction of the theoretical optical flow vector. Only feature points with included angles exceeding this threshold are considered to have significantly deviated from the direction of the two vectors. The preset frame number threshold is a lower limit of the number of consecutive tracking frames set in advance to exclude misjudgments caused by newly emerging feature points that have not yet been tracked stably.
[0108] Specifically, for each feature point being tracked in the current frame image, the UAV extracts its reprojection residual vector and theoretical optical flow vector, and calculates the directional angle between them; simultaneously, it reads the number of consecutive tracking frames for that feature point from the feature point tracking status. The UAV jointly judges each feature point based on two conditions: "the angle is greater than a preset angle threshold" and "the number of consecutive tracking frames is greater than a preset frame threshold." Only feature points that meet both conditions are marked as suspicious feature points. The reason for using joint judgment instead of relying solely on the angle condition is that newly emerging and unstable feature points often exhibit short-term directional anomalies due to initial positioning deviations. If only the angle is used as the judgment criterion, these feature points are very likely to be mistakenly identified. Feature points with a sufficient number of consecutive tracking frames mean that they have maintained stable geometric relationships over the past many frames. If their orientation suddenly becomes abnormal in this case, it is more likely to be caused by real motion rather than observation noise.
[0109] In some embodiments, the UAV may further require, based on the aforementioned angle criterion and continuous tracking frame count criterion, that the magnitude of the reprojection residual vector of the feature point be greater than a preset lower limit threshold. This is because when the magnitude of the reprojection residual vector itself is too small, its direction will be primarily dominated by image observation noise. In this case, subsequent vertical component calculations based on this direction will lose numerical stability. Therefore, using "the magnitude of the reprojection residual vector being greater than the preset lower limit threshold" as a necessary prerequisite for selecting suspicious feature points can effectively ensure the signal-to-noise ratio and robustness of subsequent dynamic gain scoring, avoiding the situation where the vector direction is dominated by observation noise due to excessively small residual magnitude, thus affecting subsequent vertical component calculations based on the reprojection residual vector direction. The preset lower limit threshold can be determined by the statistical quantile (such as the median or a specified quantile) of the reprojection residual magnitude of static feature points outside the dynamic / static uncertainty region. A typical value is 0.5 to 2.0 pixels, and this is not limited here.
[0110] It should be noted that the reprojection residual vector reflects the directional deviation of the current observed position of a feature point relative to its projected position on the static map, while the theoretical optical flow vector reflects the expected displacement direction of the same feature point under the assumption of a purely static scene caused by the UAV's own pose changes. When the feature point is indeed a static background, its reprojection residual is mainly caused by noise, has a random direction, and does not have a stable angular relationship with the theoretical optical flow direction. However, when the entity corresponding to the feature point has real motion similar to the camera's self-motion direction, its reprojection residual will continuously accumulate along the entity's motion direction and deviate from the static motion direction indicated by the theoretical optical flow, thus forming a stable and significant angle between the two vectors. Therefore, using whether this angle exceeds a preset angle threshold as a preliminary screening condition for suspicious feature points has a clear physical and statistical basis.
[0111] S204. Perform density clustering on suspicious feature points, and expand the pixel neighborhood of the clustered feature point clusters according to a preset ratio to obtain dynamic and static uncertain regions.
[0112] Density clustering refers to a class of algorithms (such as DBSCAN) that perform unsupervised aggregation and grouping based on the spatial density distribution of feature points on the pixel plane. It can identify clusters of feature points that are close to each other without presetting the number of clusters and automatically mark isolated outliers as noise. The preset ratio refers to the pixel neighborhood expansion ratio coefficient set so that the clustered feature point clusters can encompass the complete outline of the potential dynamic target.
[0113] Specifically, the UAV first performs density clustering on the two-dimensional coordinate distribution of all suspicious feature points on the pixel plane of the current frame, grouping suspicious feature points that are close in location and whose density meets the aggregation condition into a feature point cluster, while discarding isolated suspicious feature points that are far from any dense cluster as noise. Next, the UAV calculates the outer envelope region (such as the minimum bounding rectangle or bounding convex hull) of each feature point cluster as the initial region, and expands the corresponding pixel neighborhood of the initial region outward along the four sides according to a preset ratio to obtain the final region after appropriate expansion. The union of all the final regions is taken as the dynamic and static uncertain region output by this step.
[0114] In some embodiments, the UAV can take the two-dimensional pixel coordinates of all feature points in the suspicious feature point list as input, call a density clustering algorithm (such as DBSCAN) to perform clustering operations on them according to a preset neighborhood radius and minimum neighbor number, and obtain several feature point clusters and several isolated points marked as noise; discard the noise isolated points, and calculate the axis-aligned bounding box or convex hull of each feature point cluster as the initial envelope region; expand each initial envelope region outward along the corresponding pixel bandwidth according to a preset ratio (such as expanding the width and height of the initial region outward by 30%), and obtain the expanded envelope region; merge all the expanded envelope regions into a dynamic and static uncertain region, and register all tracking feature points located in the region into the uncertain region feature point list.
[0115] As a parameter configuration example of a typical implementation of this application, the recommended ranges for the above thresholds are as follows (for example only, not constituting a limitation): the preset angle threshold can be 15°~45° (typical value 30°); the preset frame number threshold can be 3~10 frames (typical value 5 frames); the neighborhood radius in the density clustering algorithm (such as DBSCAN) can be 1%~3% of the image's shorter side size, and the minimum number of neighbors can be 3~5; the preset ratio (outward expansion ratio) can be 20%~50% (typical value 30%); the threshold for the number of feature points in the dynamic and static uncertain region involved in step S205 can be 5~15. The above values are for implementation reference only, and the specific values should be calibrated by those skilled in the art in combination with sensor specifications, flight speed and scene complexity, and are not limited here.
[0116] S205. When the number of feature points in the dynamic and static uncertain region exceeds the preset number threshold, predict the camera pose after the UAV performs each candidate micro-maneuver based on the maneuver parameters corresponding to each candidate micro-maneuver and the current pose of the UAV.
[0117] Among them, the maneuver parameters refer to the set of key parameters required to fully characterize each candidate micro-maneuver, including but not limited to displacement direction, displacement amplitude, deflection direction, deflection angle, rise and fall direction, rise and fall amplitude, and maneuver duration; the camera predicted pose refers to the three-dimensional position and three-dimensional attitude of the onboard camera in the world coordinate system after the UAV performs a certain candidate micro-maneuver.
[0118] Specifically, for each candidate micro-maneuver in the preset candidate micro-maneuver library, the UAV retrieves the corresponding displacement direction, displacement amplitude, deflection angle, or rise and fall amplitude from the maneuver parameter table. Based on the current UAV pose maintained in the SLAM state information, the UAV performs forward kinematics calculations along the motion direction and amplitude described by the corresponding maneuver parameters to obtain the predicted pose of the UAV at the moment the candidate micro-maneuver is completed. Then, the UAV uses pre-calibrated body-camera extrinsic parameters to convert the predicted pose of the body into the corresponding predicted pose of the camera.
[0119] It should be noted that the reason for performing independent pose prediction for each candidate micro-maneuver is that micro-maneuvers of different directions and types will bring about drastically different viewpoint change patterns, which in turn will cause the feature points in the uncertain dynamic and static regions to exhibit drastically different pixel displacement distributions. Only by calculating the corresponding camera prediction pose for each candidate micro-maneuver can the potential contribution of each candidate micro-maneuver to the separation effect of dynamic and static targets be quantitatively evaluated in subsequent steps, so as to obtain the best dynamic and static recognition benefits with the minimum flight cost.
[0120] Optionally, the UAV can read the world coordinate system position and attitude quaternion corresponding to the current pose of the UAV from the SLAM state information, construct the corresponding pose increment (i.e. translation increment vector and rotation increment quaternion) according to the maneuver parameters of each candidate micro-maneuver, and superimpose the pose increment with the current pose according to the rigid body transformation law to obtain the predicted pose of the UAV body after the candidate micro-maneuver is executed.
[0121] S206. Based on the camera's predicted pose and local... Figure 3 The depth of each feature point in the dimensional point is used to predict the optical flow vector of each feature point in a purely static scene within a region of dynamic and static uncertainty.
[0122] Here, the predicted optical flow vector refers to the pixel displacement vector that each feature point should exhibit after performing candidate micro-maneuvers, calculated based on the camera's predicted pose, assuming that all entities corresponding to all feature points within the uncertain dynamic and static regions are static backgrounds; the feature point depth refers to the local depth associated with the feature point. Figure 3 The longitudinal distance of a point relative to the camera's optical center in the camera coordinate system reflects the distance of the feature point from the UAV's onboard camera.
[0123] Specifically, for each candidate micro-maneuver corresponding to the camera's predicted pose, the UAV calculates the predicted optical flow vector for each feature point within the dynamic-static uncertainty region under the assumption of that candidate micro-maneuver. The UAV first starts from the local... Figure 3 The UAV reads the 3D world coordinates associated with the feature point from the set of points and transforms the 3D point from the world coordinate system to the current frame camera coordinate system by combining the current pose of the UAV and the camera intrinsic parameters. The component of the transformed coordinates along the camera optical axis is taken as the depth of the 3D point in the current frame camera coordinate system. For newly entered feature points whose depth has not yet converged in the local map (such as when the number of co-view frames is lower than the preset minimum co-view threshold), they can be temporarily excluded from the calculation of dynamic gain score to avoid depth gross errors from polluting the evaluation results of candidate micro-maneuvers. Next, the UAV reprojects the 3D point onto the camera pixel plane after the maneuver according to the camera predicted pose corresponding to the candidate micro-maneuver, and obtains the predicted new pixel position of the feature point under the pure static assumption. Finally, the UAV constructs a two-dimensional directed line segment with the actual observed pixel position of the current frame as the starting point and the predicted new pixel position as the ending point and determines it as the predicted optical flow vector of the feature point.
[0124] S207. Based on the component amplitude of the predicted optical flow vector of each feature point in the direction perpendicular to the reprojection residual vector, obtain the motion difference indication value of each feature point.
[0125] Among them, the motion difference indicator value is a scalar value used to quantify the ability of a candidate micro-maneuver to expose the dynamic and static attributes of a single feature point. The larger the value, the more the candidate micro-maneuver can make the feature point deviate from its original suspicious motion direction under the new perspective, thus helping to separate the real dynamic target from the static background.
[0126] Specifically, for each feature point within the dynamic and static uncertainty region, the UAV extracts the reprojection residual vector obtained in step S201 and the predicted optical flow vector obtained in step S206. Then, using the direction of the reprojection residual vector as a reference, the UAV constructs a direction vector perpendicular to it and orthogonally decomposes the predicted optical flow vector along this vertical direction to obtain the component of the predicted optical flow vector in the direction perpendicular to the reprojection residual vector. Finally, the UAV takes the amplitude of this vertical component as the motion difference indication value of the feature point under the corresponding candidate micro-maneuver.
[0127] It is understandable that the component magnitude perpendicular to the reprojection residual vector direction is selected as the evaluation index because the reprojection residual vector itself already reflects the suspicious motion direction of the feature point in the current frame. If the predicted optical flow caused by the candidate micro-maneuver happens to be in the same direction, the newly added observation information cannot further separate the dynamic target from the static background in this direction. Conversely, if the predicted optical flow caused by the candidate micro-maneuver has a large component perpendicular to this direction, the real dynamic target will inevitably deviate from the static prediction path under the new perspective, so that its response residual exhibits a strong directional anomaly, which is convenient for it to be accurately locked in step S105.
[0128] In some embodiments, the UAV can traverse the predicted optical flow vector cache corresponding to each candidate micro-maneuver, extract the reprojection residual vector and the predicted optical flow vector for each feature point in the dynamic and static uncertainty region; normalize the reprojection residual vector to obtain its direction unit vector, and construct a vertical direction unit vector orthogonal to it by rotating the unit vector by 90 degrees; perform a dot product operation between the predicted optical flow vector and the vertical direction unit vector to obtain the scalar projection component of the predicted optical flow vector in the vertical direction, and take the absolute value of the projection component as the motion difference indicator value of the feature point; store the motion difference indicator value of each feature point and its feature point number together in the motion difference indicator value cache of the corresponding candidate micro-maneuver for summation in step S208.
[0129] S208. The sum of motion difference indication values of all feature points in the dynamic and static uncertainty region is determined as the dynamic gain score of the candidate micro-maneuver, and the target micro-maneuver is determined from multiple candidate micro-maneuvers based on the dynamic gain score.
[0130] Specifically, for each candidate micro-maneuver, the UAV performs an arithmetic summation of the motion difference indication values of all feature points within the dynamic and static uncertainty region recorded in the corresponding motion difference indication value cache, and uses the summation scalar value as the dynamic gain score of the candidate micro-maneuver; then, the UAV compares the dynamic gain scores of all candidate micro-maneuvers according to their numerical values, and determines the candidate micro-maneuver corresponding to the highest score as the target micro-maneuver.
[0131] It should be noted that the aforementioned "preset maneuver activation threshold" can be calibrated based on the historical statistical distribution of the sum of motion difference indication values of all feature points in the uncertain area between static and dynamic scenes. For example, the 95th percentile of the statistical distribution of this summation value in a purely static calibration scene can be taken as the preset maneuver activation threshold, thereby ensuring that active micro-maneuvers are not mistakenly triggered in static scenes while maintaining sufficient sensitivity to real dynamic targets.
[0132] S209. Obtain the standard control quantity of the UAV's current mission trajectory within the execution time window of the target micro-maneuver, calculate the maneuver parameters corresponding to the target micro-maneuver into incremental control quantities relative to the standard control quantity, and superimpose the incremental control quantity with the standard control quantity to obtain the composite control quantity.
[0133] Among them, the standard control quantity refers to the sequence of control commands that should have been issued to the motor loop according to the original mission plan within the execution time window of the mission trajectory currently being performed by the UAV and the target micro-maneuver. It reflects the original flight attitude and speed requirements of the UAV without introducing active micro-maneuver.
[0134] Specifically, the UAV first estimates an initial execution time window based on the maneuver parameters (including the upper limit of maneuver duration) corresponding to the target micro-maneuver, and queries the mission management module for the standard control quantity sequence that should be issued within the initial execution time window for the current mission trajectory. This sequence carries the original flight plan of the UAV without introducing active micro-maneuvers. The endpoint of the initial execution time window is the time corresponding to the upper limit of maneuver duration, which is used for pre-fetching synthetic control quantities and checking flight constraints. After the maneuver begins, the endpoint of the window is updated according to the actual time when the cumulative displacement of S301 reaches the preset observation parallax threshold, thus obtaining the final execution time window. Next, the UAV solves the maneuver parameters (including displacement direction, displacement amplitude, or deflection angle, etc.) corresponding to the target micro-maneuver into incremental control quantities relative to the standard control quantities according to the kinematic model of the target micro-maneuver. These incremental control quantities only carry the control deviation necessary for the viewpoint disturbance. Finally, the UAV superimposes the incremental control quantities and the standard control quantities point by point according to the timestamp to obtain the synthetic control quantity sequence.
[0135] It is understandable that the execution time window determined in this step is the final execution time window dynamically determined based on the cumulative displacement during the actual maneuver execution, which may end earlier than the estimated end of the initial execution time window in S209; at this time, the synthetic control quantity after the end of the window will automatically fall back to the standard control quantity and no longer affect the main flight mission.
[0136] S210. When the synthetic control quantity meets the preset flight constraints, the UAV is controlled to perform target micro-maneuvers according to the synthetic control quantity.
[0137] Specifically, before the UAV actually sends the synthetic control quantities to the motor loop, it first compares and verifies the sequence of synthetic control quantities with the preset flight constraints item by item. The verification includes, but is not limited to, whether the attitude angular rate exceeds the preset maximum attitude angular rate, whether the linear acceleration exceeds the preset maximum linear acceleration, whether the horizontal deviation relative to the mission trajectory exceeds the preset maximum horizontal deviation, and whether the closest distance to the obstacle is lower than the preset minimum safe distance. When the synthetic control quantities meet all the preset flight constraints, the UAV sends the synthetic control quantities to the motor loop sequentially according to the timestamp, driving the body to complete the target micro-maneuver along the trajectory described by the synthetic control quantities within the execution time window; at the same time, the UAV's onboard camera continuously acquires images at the original working frame rate, and the IMU continuously outputs inertial measurement data at the original sampling frequency, and the data cache module synchronously archives the two according to the timestamp.
[0138] Conversely, if the synthetic control quantity violates any preset flight constraint at a certain moment or in a certain dimension, the target micro-maneuver is judged to have a flight risk and is immediately abandoned. The UAV returns to the original mission trajectory and continues to fly according to the standard control quantity, and the active micro-maneuver decision process is re-triggered in the next frame.
[0139] Specifically, such as Figure 3 The diagram shown is a flowchart illustrating a process for an unmanned aerial vehicle (UAV) to identify a dynamic target region in the current frame image in an embodiment of this application.
[0140] S301. After the target micro-maneuver begins, the cumulative displacement of the UAV is monitored in real time, and the time interval from the start of the maneuver to the moment when the cumulative displacement reaches the preset observation parallax threshold is determined as the execution time window.
[0141] Among them, the cumulative displacement refers to the Euclidean space distance that the UAV has moved in the world coordinate system relative to the starting position of the maneuver since the moment the micro-maneuver of the target began, reflecting the degree of deviation of the UAV body from the starting position of the maneuver; the preset observation parallax threshold (also known as the preset observation baseline threshold) is the lower limit of the cumulative displacement of the UAV relative to the starting position of the maneuver, set to ensure that the target frame image has sufficient visual parallax relative to the current frame image at the end of the execution time window to support subsequent dynamic and static identification. Its value reflects the minimum baseline length used for triangulation between the target frame and the current frame.
[0142] Specifically, the UAV records the start time of the micro-maneuver as the starting point of the execution time window the instant the micro-maneuver begins, and uses the UAV's spatial position at this moment as the reference position for calculating the cumulative displacement. Then, during the continuous maneuver, the UAV reads the linear acceleration output by the IMU in real time and performs secondary integration calculation, or directly obtains the latest pose from the pose estimation module and performs differential calculation to obtain the cumulative displacement since the start of the maneuver. The UAV continuously compares the cumulative displacement with the preset observation parallax threshold. Once the cumulative displacement reaches or exceeds the preset observation parallax threshold for the first time, it records this moment as the end point of the execution time window, and thus the time interval between the start time and the end time is determined as the execution time window.
[0143] In some embodiments, the UAV can, at the instant the first instruction of the synthetic control quantity is issued, latch the current time of the system clock as the start time of the maneuver and record it in the start field of the execution time window cache. At the same time, the position coordinates of the UAV in the world coordinate system at this moment are latched as the reference coordinates for calculating the cumulative displacement. During the continuous execution of the target micro-maneuver, the UAV periodically reads the latest body position coordinates from the pose estimation module at a high frequency (e.g., equal to or higher than the IMU sampling frequency), and calculates the Euclidean distance between the current position and the reference position as the real-time cumulative displacement using the reference coordinates as the starting point. The real-time cumulative displacement is compared with a preset observation parallax threshold one by one. Once the comparison result reaches or exceeds the threshold for the first time, the current system clock time is latched in the end field of the execution time window cache. The time interval defined by the start time and the end time is written as the execution time window into the sampling control configuration of the data acquisition module for use by the image and inertial measurement data acquisition links corresponding to steps S302 and S303.
[0144] S302. Within the execution time window, continuously acquire multiple frames of intermediate image sequences and intermediate inertial measurement data synchronized with the intermediate image sequences; and extract the intermediate image with the smallest attitude change parameter that is less than the preset anti-shake threshold from the intermediate image sequence based on the attitude change parameter corresponding to the intermediate inertial measurement data, to obtain the target frame image.
[0145] Among them, attitude change parameters refer to the quantitative values of the degree of attitude change of the airborne camera at the moment of exposure of the corresponding intermediate image, as reflected by the intermediate inertial measurement data. They are usually characterized by the amplitude of angular velocity or the increment of attitude angle during the exposure period. The preset image stabilization threshold is the upper limit of attitude change parameters set to filter out intermediate images that are streaked or blurred due to violent maneuvers.
[0146] Specifically, within the execution time window, the UAV continuously acquires images using its onboard camera at the original frame rate, resulting in an intermediate image sequence containing multiple original images. Simultaneously, the IMU continuously outputs inertial measurement samples at the original sampling frequency, and the data synchronization module aligns each intermediate image with the IMU samples surrounding its exposure time according to the timestamp, forming time-synchronized intermediate inertial measurement data. Next, for each frame in the intermediate image sequence, the UAV extracts the angular velocity amplitude or attitude angle increment from the IMU samples surrounding its exposure time as the corresponding attitude change parameter for that frame. Finally, the UAV compares the attitude change parameters of each intermediate image with a preset anti-shake threshold, and extracts one frame from all intermediate images whose attitude change parameters are less than the preset anti-shake threshold as the target frame image.
[0147] It should be noted that if the attitude change parameters of all intermediate images in the intermediate image sequence are greater than or equal to the preset anti-shake threshold, it is considered that there is no qualified target frame image in the current execution time window. The UAV abandons all image data collected in the current window and waits for the next frame to re-trigger the active micro-maneuver decision process. The processing logic is the same as S306.
[0148] S303. All inertial measurement data generated within the execution time window are identified as target inertial measurement data.
[0149] Specifically, the UAV extracts all inertial measurement samples (including the complete time series of triaxial acceleration and triaxial angular velocity output by the IMU from the start time to the end time) from the intermediate inertial measurement buffer queue established in step S302, and uses them as target inertial measurement data for unified identification and archiving.
[0150] S304. Within the execution time window, predict the planned inertial measurement data of the UAV based on the synthetic control quantity, and compare the actual collected target inertial measurement data with the planned inertial measurement data to obtain the execution disturbance deviation.
[0151] Among them, the planned inertial measurement data refers to the sequence of inertial measurement samples that the UAV should theoretically present within the execution time window, calculated by forward simulation of the UAV dynamics model based on the synthetic control quantity output in step S209, reflecting the expected body response of the maneuver scheme under ideal, undisturbed conditions; the execution disturbance deviation refers to the deviation between the actual collected target inertial measurement data and the planned inertial measurement data, reflecting the degree to which the UAV deviates from the planned trajectory due to non-ideal factors such as external wind disturbance and mechanical vibration during actual maneuvering.
[0152] Specifically, on the one hand, the UAV uses the synthetic control sequence output in step S209 and a pre-calibrated UAV dynamics model to perform forward simulation calculations on the three-axis acceleration and three-axis angular velocity of the aircraft at each moment within the execution time window, obtaining planned inertial measurement data that corresponds one-to-one with the target inertial measurement data on the timestamp; on the other hand, the UAV compares the actual collected target inertial measurement data with the planned inertial measurement data point by point according to the timestamp (such as calculating the norm of the difference in acceleration and the difference in angular velocity between the two sequences at each sampling moment and accumulating or averaging them along the time dimension), to obtain the execution disturbance deviation that reflects the overall execution quality of this maneuver.
[0153] In some embodiments, the UAV can load a pre-calibrated UAV six-degree-of-freedom dynamic model (including parameters such as mass, inertial tensor, and aerodynamic coefficients), use the synthetic control quantity sequence output in step S209 as model input for forward numerical simulation in time-stamp order, and output the corresponding three-axis acceleration and three-axis angular velocity of the airframe at each simulation moment, and write them into the planned inertial measurement data cache according to the timestamp; read the actual inertial measurement sample sequence from the target inertial measurement data cache collected in step S303, and align it point-to-point with the planned inertial measurement data according to the same timestamp; calculate the difference norm between the actual acceleration and the planned acceleration, and the difference norm between the actual angular velocity and the planned angular velocity at each alignment moment, and take the average or weighted sum of the norms over the entire time period as the execution disturbance deviation of the target micro-maneuver; write the execution disturbance deviation value into the disturbance evaluation cache for threshold determination in step S305.
[0154] S305. Whether the execution disturbance deviation is less than the preset disturbance threshold.
[0155] The preset disturbance threshold is the upper limit of the execution disturbance deviation set to determine whether the target micro-maneuver data has sufficient numerical reliability. Only when the execution disturbance deviation is lower than the threshold is the target frame image and target inertial measurement data collected in this maneuver considered reliable and usable for subsequent dynamic target area determination process.
[0156] Specifically, the UAV compares the execution disturbance deviation value calculated in step S304 with the preset disturbance threshold. If the execution disturbance deviation is less than the preset disturbance threshold, it is determined that the actual motion of the UAV during the target micro-maneuver is consistent with the motion height planned by the synthetic control quantity and the external interference is within an acceptable range. Then, the target frame image extracted in step S302 and the target inertial measurement data collected in step S303 are transmitted to the response residual calculation process corresponding to step S307 for subsequent accurate determination of the dynamic target area.
[0157] Conversely, if the perturbation deviation is greater than or equal to the preset perturbation threshold, it is determined that the actual movement of the aircraft during the current micro-maneuver has significantly deviated from the planned trajectory, and the collected target frame images and target inertial measurement data can no longer accurately reflect the true response residual of the dynamic target. Then, it jumps to the data abandonment branch corresponding to step S306, discards the target frame images collected in the current window, and waits for the next frame to re-trigger the entire active micro-maneuver decision process.
[0158] As one specific implementation, the preset disturbance threshold can be determined based on the upper bound of the 3σ of the time-averaged values of the acceleration difference norm and the angular velocity difference norm measured by the UAV under calibrated flight conditions without external wind disturbance; those skilled in the art can also make adaptive adjustments according to the aerodynamic characteristics of the aircraft, load conditions and wind force level of the operating environment, which is not limited here.
[0159] S306. Abandon the target frame images acquired within the current execution time window.
[0160] Specifically, after determining that the target micro-maneuver data is no longer reliable, the UAV removes and discards the target frame image extracted in step S302 and the target inertial measurement data collected in step S303 from the active micro-maneuver processing pipeline of the current frame. At the same time, the UAV retains the SLAM state information before the execution of the target micro-maneuver as the reference input for the next frame SLAM processing. That is, it no longer performs dynamic target region determination and removal operations on feature points in the dynamic and static uncertain areas of the current frame, but continues to maintain pose estimation and local map updates using the original tracking queue to avoid the overall degradation of the SLAM state due to the distortion of single maneuver data. At the same time, the processing status of the target micro-maneuver is marked as "abandoned" and recorded in the historical maneuver status log for subsequent diagnosis and statistical analysis. Then, the UAV continues to fly on the original mission track and waits for the next frame image to arrive before triggering the complete active micro-maneuver decision process again from step S101.
[0161] S307. The directed line segment pointing from the actual observation position of each feature point in the dynamic and static uncertainty region to the corresponding theoretical observation position is determined as the response residual vector of the feature point; and the feature points whose magnitude of the response residual vector is greater than the preset residual threshold are determined as candidate dynamic feature points.
[0162] Specifically, the UAV first pre-integrates and calculates the relative pose change of the camera from the current frame to the target frame based on the target inertial measurement data. Using the current frame UAV pose maintained in the SLAM state information as the reference pose, the aforementioned relative pose change is added as an increment to the reference pose to obtain the calculated pose at the target frame. The "current frame pose + IMU relative increment" approach is used here instead of directly using the IMU-independent pose integration because independent IMU integration introduces long-term cumulative drift. This step only focuses on the relative pose change within a few hundred milliseconds between the current frame and the target frame; using the relative increment approach effectively avoids the interference of long-term IMU drift on the response residual calculation.
[0163] Then, the drone will associate each feature point within the uncertain dynamic and static region with the local geography. Figure 3 Under the static assumption, the dimensional point is projected onto the target frame image along the relative pose change to obtain the theoretical observed pixel position of each feature point. Then, the UAV uses a front-end feature tracking algorithm to detect the actual observed pixel position of each feature point in the dynamic and static uncertainty region on the target frame image, and constructs a two-dimensional directed line segment with the actual observed pixel position as the starting point and the theoretical observed pixel position as the ending point, and uses it as the response residual vector of the feature point. Finally, the UAV compares the magnitude of the response residual vector of each feature point with a preset residual threshold, and marks the feature points with magnitudes greater than the preset residual threshold as candidate dynamic feature points and writes them into the candidate dynamic feature point list.
[0164] It is understandable that the preset residual threshold can be a fixed value pre-calibrated by relevant technicians based on the actual camera noise level, or it can be adaptively determined based on the distribution of the residual amplitude of static feature point response outside the dynamic and static uncertainty area. For example, a specified quantile (such as the 95th quantile) of the residual amplitude of the static feature point response can be taken as the preset residual threshold, so that the threshold can be automatically adjusted with the environmental noise level. This is not limited here.
[0165] It should also be noted that the reprojection residual vector and the response residual vector in the embodiments of this application are defined in the same way, that is, both are defined with the theoretical projected pixel position of the corresponding feature point on the target image as the starting point and the actual observed pixel position as the ending point. When the opposite convention is used, it is only necessary to negate the corresponding vector in the operation of magnitude comparison and direction clustering. The operation result is equivalent to the method described in the embodiments of this application. Such equivalent transformations are all within the protection scope of this application.
[0166] S308. Perform joint clustering of candidate dynamic feature points based on the directional similarity of the response residual vector and the pixel position proximity to obtain multiple clusters; determine the pixel region covered by each cluster as a dynamic target region.
[0167] Among them, directional similarity refers to the quantitative measure of the consistency of the directions between the response residual vectors corresponding to any two candidate dynamic feature points, which is usually represented by the cosine value of the angle between the two vectors or the difference in the direction angle; pixel location proximity refers to the quantitative measure of the spatial distance between any two candidate dynamic feature points on the pixel plane of the target frame, which is usually represented by the Euclidean pixel distance between the two points; clustering refers to the set of feature points formed by aggregating several candidate dynamic feature points with similar residual directions and spatial proximity after joint clustering.
[0168] Specifically, the UAV introduces two dimensions, "directional similarity" and "pixel position proximity," as joint clustering criteria for all candidate dynamic feature points. For any two candidate dynamic feature points, it simultaneously examines whether the angle between their response residual vectors is less than a preset angle threshold (i.e., the directional similarity is high enough) and whether the Euclidean distance between their pixel positions is less than a preset proximity threshold (i.e., the pixel position proximity is high enough). Only when both conditions are met are the two points assigned to the same cluster. Correspondingly, if the angle between the response residual vectors of two candidate dynamic feature points is greater than or equal to the preset angle threshold, they are assigned to different clusters even if their pixel positions are adjacent, thus ensuring that the clusters corresponding to different moving entities are separable in terms of residual direction semantics. After complete clustering, several independent clusters are obtained, and the candidate dynamic feature points in each cluster have similar motion directions and adjacent spatial positions.
[0169] The UAV then calculates the outer envelope region (including but not limited to the minimum bounding rectangle or bounding convex hull, and the pixel connected region obtained by expanding the set of feature points in the cluster by a preset number of pixels based on morphological dilation operation) on the pixel plane of the target frame for each cluster, and determines the pixel range covered by the outer envelope region as an independent dynamic target region.
[0170] S309. Determine the adjacent pixel bands within a preset range outside the dynamic target area as the static supplementary area; evaluate the number of remaining SLAM feature points, and determine the target quantity of supplementary feature points corresponding to the remaining SLAM feature points based on the preset tracking quantity threshold.
[0171] The preset tracking number threshold refers to the minimum number of active feature points in the feature point tracking queue required by the SLAM system to ensure stable operation of the visual odometry; the supplementary feature point target number refers to the number of new feature points that need to be extracted in the static supplementary area in order to bring the remaining number of SLAM feature points back up to above the preset tracking number threshold.
[0172] Specifically, the UAV, on the one hand, expands the pixel width corresponding to a preset range outward along the outer contour of each dynamic target region output in step S308 to obtain a ring-shaped adjacent pixel band around the dynamic target region, and uses the union of the adjacent pixel bands corresponding to all dynamic target regions as the static supplementary region output in this step; on the other hand, the UAV counts the current number of remaining SLAM feature points in the feature point tracking queue in the SLAM status information after step S106 is completed, and compares this number with a preset tracking number threshold. Specifically, the UAV determines the difference between the preset tracking number threshold and the number of remaining SLAM feature points as the target number of supplementary feature points.
[0173] When the remaining number is lower than the threshold, the difference is the number of new feature points that need to be added. When the remaining number has reached or exceeded the threshold, the difference is zero or negative. At this time, the target number of supplementary feature points is set to zero to maintain the health of the queue. In this case, the UAV can skip the static supplementary feature point extraction and queuing operation in step S310 and directly participate in the SLAM state update of subsequent frames with the current tracking queue.
[0174] It should be noted that the adjacent pixel band within the preset outer range is selected as the static supplementary region because, under the Markov smoothness continuity assumption, after removing dynamic target entities, the physical space immediately surrounding them is likely to be a static background rich in high-frequency textures. Extracting supplementary feature points in this region can quickly repair the broken local 3D topological features around the target, preventing visual odometry degradation caused by imbalances in the spatial distribution of feature points.
[0175] S310. Extract new feature points based on the target quantity of supplementary feature points within the static supplementation region to obtain at least one static supplementary feature point; add the static supplementary feature point to the feature point tracking queue in the SLAM state information.
[0176] Specifically, within the static supplementation area defined in step S309, the UAV calls an image feature point extraction algorithm (such as FAST corner detector, ORB feature point detector, or Shi-Tomasi corner detector, etc.) to perform a new feature point extraction operation according to the target number of supplementation feature points determined in step S309, obtaining at least one new static supplementation feature point and its position coordinates on the pixel plane of the target frame; then, the UAV assigns a new feature point number to each static supplementation feature point and initializes its tracking status record (including the initial value of the number of consecutive tracking frames, the first appearance timestamp, feature descriptor, etc.), and writes it together with its position coordinates into the feature point tracking queue in the SLAM status information.
[0177] In some embodiments, the UAV can read the static supplementation region and the target quantity of supplementation feature points determined in step S309 from the feature supplementation configuration cache, and use the static supplementation region as an image mask to crop or mask the target frame image; in the static supplementation region behind the mask, an image feature point extraction algorithm (such as FAST, ORB or Shi-Tomasi corner detector) is called, and the top few corner points are selected as candidate feature points according to the feature response intensity from high to low; non-maximum suppression and minimum spacing screening are performed on the candidate feature points according to the target quantity of supplementation feature points to obtain several static supplementation feature points that are evenly distributed and have sufficient texture response; a new globally unique feature point number is assigned to each static supplementation feature point, its continuous tracking frame counter is initialized (e.g., set to the initial value 1), its first appearance timestamp and the target frame number are recorded, and its feature descriptor and pixel coordinates are packaged together and written to the tail of the feature point tracking queue in the SLAM status information so that it is included in the continuous tracking range by the front-end tracking algorithm when the next frame arrives.
[0178] The drone in this embodiment of the invention is an electronic device. Figure 4 A schematic diagram of the architecture of an electronic device suitable for implementing embodiments of the present invention is shown.
[0179] It should be noted that, Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.
[0180] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by instructions (computer programs), or by instructions (computer programs) controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. The electronic device of this embodiment includes a storage medium and a processor, wherein the storage medium stores multiple instructions that can be loaded by the processor to execute any step of the method provided in the embodiments of the present invention.
[0181] Specifically, the storage medium and the processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more signal lines. The storage medium stores computer-executable instructions that implement data access control methods, including at least one software functional module that can be stored in the storage medium in the form of software or firmware. The processor executes various functional applications and data processing by running the software program and module stored in the storage medium. The storage medium can be, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The storage medium stores the program, and the processor executes the program after receiving the execution instructions.
[0182] Furthermore, the software programs and modules within the aforementioned storage medium may also include an operating system, which may include various software components and / or drivers for managing system tasks (e.g., memory management, storage device control, power management, etc.) and can communicate with various hardware or software components to provide an operating environment for other software components. The processor may be an integrated circuit chip with signal processing capabilities. The aforementioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc., which can implement or execute the methods, steps, and logic flowcharts disclosed in this embodiment. The general-purpose processor may be a microprocessor or any conventional processor.
[0183] Since the instructions stored in the storage medium can execute the steps in any of the methods provided in the embodiments of the present invention, the beneficial effects of any of the methods provided in the embodiments of the present invention can be achieved, as detailed in the preceding embodiments, and will not be repeated here.
[0184] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A SLAM method based on active micro-maneuvering, applied to unmanned aerial vehicles (UAVs), characterized in that, The method includes: The system acquires the current frame image, the UAV's inertial measurement data, and SLAM state information, including the UAV's current pose, the tracking status of local map 3D points and feature points; Based on the current pose of the UAV, the local map 3D points are projected onto the current frame image, and the dynamic and static uncertain regions are determined in the current frame image based on the reprojection residual obtained from the projection and the feature point tracking status. When the number of feature points in the dynamic and static uncertainty region exceeds a preset threshold, the dynamic gain score of each feature point in the dynamic and static uncertainty region is predicted after the UAV performs multiple candidate micro-maneuvers. Based on the dynamic gain score, the target micro-maneuver is determined from the multiple candidate micro-maneuvers. The candidate micro-maneuvers include translational micro-maneuvers, yaw micro-maneuvers, and altitude micro-maneuvers. The dynamic gain score characterizes the degree of motion difference of feature points in the dynamic and static uncertainty region. Under preset flight constraints, the UAV is controlled to perform micro-maneuvers on the target, and target frame images and target inertial measurement data are acquired within the execution time window of the target micro-maneuvers. Based on the target inertial measurement data, predict the theoretical observation position of feature points in the target frame image within the dynamic and static uncertainty region, and determine the dynamic target region from the dynamic and static uncertainty region based on the response residual of the actual observation position of the feature points relative to the theoretical observation position. Remove the SLAM feature points corresponding to the dynamic target region from the dynamic and static uncertain regions, and update the SLAM state information based on the remaining SLAM feature points.
2. The method according to claim 1, characterized in that, The step of determining the dynamic and static uncertain regions in the current frame image based on the reprojection residual obtained from projection and the feature point tracking state specifically includes: Based on the current pose of the UAV, the local map 3D points are projected onto the current frame image, and the directed line segment pointing from the reprojected pixel position of each feature point to the actual observed pixel position is determined as the reprojection residual vector of the feature point. Based on the inertial measurement data, predict the theoretical optical flow vector of each feature point in the current frame image under a purely static scene; Feature points whose angle between the reprojection residual vector and the theoretical optical flow vector is greater than a preset angle threshold, and whose number of consecutive tracking frames in the feature point tracking state is greater than a preset frame threshold, are identified as suspicious feature points. Density clustering is performed on the suspicious feature points, and the resulting clusters of feature points are expanded outward into their pixel neighborhoods according to a preset ratio to obtain the dynamic and static uncertain region.
3. The method according to claim 2, characterized in that, The step of predicting the dynamic gain score of each feature point in the dynamic-static uncertainty region after the UAV performs multiple candidate micro-maneuvers specifically includes: Based on the maneuver parameters corresponding to each candidate micro-maneuver and the current pose of the UAV, predict the camera's predicted pose after the UAV performs each candidate micro-maneuver. Based on the camera's predicted pose and the depth of each feature point in the local map's 3D points, predict the optical flow vector of each feature point in the dynamic-static uncertainty region under a purely static scene. Based on the component amplitude of the predicted optical flow vector of each feature point in the direction perpendicular to the reprojection residual vector, the motion difference indication value of each feature point is obtained. The sum of the motion difference indication values of all the feature points within the dynamic and static uncertainty region is determined as the dynamic gain score of the candidate micro-maneuver.
4. The method according to claim 3, characterized in that, The step of controlling the UAV to perform the target micro-maneuver under preset flight constraints specifically includes: Obtain the standard control quantity of the mission trajectory currently being performed by the UAV within the execution time window of the target micro-maneuver, and solve the maneuver parameters corresponding to the target micro-maneuver into incremental control quantities relative to the standard control quantity; When the composite control quantity obtained by superimposing the incremental control quantity and the standard control quantity satisfies the preset flight constraint, the UAV is controlled to perform the target micro-maneuver according to the composite control quantity. Within the execution time window, the planned inertial measurement data of the UAV is predicted based on the synthetic control quantity, and the actual collected target inertial measurement data is compared with the planned inertial measurement data to obtain the execution disturbance deviation; When the execution disturbance deviation is less than a preset disturbance threshold, the dynamic target region is determined based on the target frame image and the target inertial measurement data; When the execution disturbance deviation is greater than or equal to the preset disturbance threshold, the target frame image acquired within the current execution time window is discarded.
5. The method according to claim 1, characterized in that, The step of determining the dynamic target region from the dynamic and static uncertainty region based on the response residual of the actual observed position of the feature point relative to the theoretical observed position specifically includes: The directed line segment pointing from the actual observation position of each feature point within the dynamic and static uncertainty region to the corresponding theoretical observation position is determined as the response residual vector of the feature point; Feature points whose magnitude of the response residual vector is greater than a preset residual threshold are identified as candidate dynamic feature points; The candidate dynamic feature points are jointly clustered based on the directional similarity of the response residual vector and the pixel position proximity to obtain multiple clusters. The candidate dynamic feature points with different response residual vector directions correspond to different clusters. Each pixel region covered by a cluster is defined as a dynamic target region.
6. The method according to claim 1, characterized in that, The step of acquiring target frame images and target inertial measurement data within the execution time window of the target micro-maneuver specifically includes: After the target micro-maneuver is started, the cumulative displacement of the UAV is monitored in real time, and the time interval from the start of the maneuver to the moment when the cumulative displacement reaches the preset observation parallax threshold is determined as the execution time window; Within the execution time window, multiple frames of intermediate image sequences and intermediate inertial measurement data synchronized with the intermediate image sequences are continuously acquired; Based on the attitude change parameters corresponding to the intermediate inertial measurement data, the intermediate image with the attitude change parameters less than a preset anti-shake threshold and the smallest attitude change parameters is extracted from the intermediate image sequence to obtain the target frame image; All inertial measurement data generated within the execution time window are determined as the target inertial measurement data.
7. The method according to claim 1, characterized in that, After the steps of removing the SLAM feature points corresponding to the dynamic target region from the dynamic-static uncertainty region and updating the SLAM state information based on the remaining SLAM feature points, the method further includes: The adjacent pixel bands within a preset range outside the dynamic target area are defined as the static supplementary area; The number of remaining SLAM feature points is evaluated, and the target number of supplementary feature points corresponding to the remaining SLAM feature points is determined according to a preset tracking number threshold. Within the static supplementation region, new feature points are extracted based on the target quantity of the supplementation feature points to obtain at least one static supplementation feature point; The static supplementary feature points are added to the feature point tracking queue in the SLAM state information, and the feature point tracking queue is used for SLAM state updates in subsequent frames.
8. A drone, characterized in that, The drone includes: one or more processors and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the UAV to perform the method as described in any one of claims 1-7.
9. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on the drone, the drone causes the drone to perform the method as described in any one of claims 1-7.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is run on the drone, it causes the drone to perform the method as described in any one of claims 1-7.
