Unmanned aerial vehicle autonomous navigation method and system based on visual guidance

Abstract

The invention discloses an unmanned aerial vehicle autonomous navigation method and system based on visual guidance. The method comprises the following steps: acquiring a sliding window image sequence at a current sampling moment; utilizing the four-dimensional point diagram regression model to obtain a four-dimensional point diagram set, a static effective point set and a dynamic effective point set; clustering and grouping the dynamic effective point set to obtain a dynamic entity point set, and generating a time-varying occupation envelope; generating a static occupancy constraint indication quantity, and constructing a time-varying risk field; acquiring current state information and target information of the unmanned aerial vehicle, and generating a navigation track by using a real-time diffusion planning model; and repeating the steps from S1 to S5 at the next sampling moment. According to the method, a cooperation mechanism of the four-dimensional point diagram regression model and the real-time diffusion planning model is introduced, so that a high-frequency rolling navigation trajectory meeting flight constraints can be generated in a complex environment in which dynamic obstacles are frequent and weak GNSS and shielding interference coexist, and the safety of autonomous navigation of the unmanned aerial vehicle is remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a vision-guided autonomous navigation method and system for UAVs. Background Technology

[0002] Existing UAV autonomous navigation methods mostly employ VIO / SLAM based on airborne cameras and local obstacle avoidance planning to achieve path tracking. Typically, the pose is first estimated from continuous images and a map or grid is constructed, and then a local planner is used to generate a short-term trajectory to avoid obstacles. This type of method is highly applicable in weak GNSS environments such as indoors, under forests, and urban low-altitude areas. However, its perception and mapping stages often assume that the scene is approximately static, or only perform instantaneous detection of dynamic objects and then reactively detour.

[0003] Due to the lack of spatiotemporally consistent modeling for dynamic scenes, existing technologies are prone to feature mismatch, map pollution, and pose drift when dynamic targets such as pedestrians and vehicles are present. This results in planning based on outdated or incorrect environmental representations. At the same time, most obstacle avoidance planning uses static cost maps or obstacles in the current frame as constraints, making it difficult to incorporate the future movement of dynamic obstacles into the planning. This can easily lead to risks such as sharp turns, trajectory jitter, and even collisions. In addition, simplified models are often used to improve real-time performance, making it difficult to balance planning frequency and trajectory feasibility in high-speed flight or complex occlusion environments. Summary of the Invention

[0004] To address the technical problems existing in the background art, this invention proposes a vision-guided autonomous navigation method and system for unmanned aerial vehicles (UAVs).

[0005] The vision-guided autonomous navigation method for unmanned aerial vehicles proposed in this invention includes the following steps: S1. A sliding window image sequence is formed by acquiring several consecutive image frames before the current sampling time using the drone's onboard camera. S2. Input the sliding window image sequence into the four-dimensional point map regression model to obtain the four-dimensional point map set. Based on the four-dimensional point map set, determine the static effective point set and the dynamic effective point set, and transform the static effective point set and the dynamic effective point set to the UAV local coordinate system. S3. Cluster the dynamic effective point set to obtain the dynamic entity point set, and generate the time-varying occupancy envelope of the corresponding dynamic entity in the prediction time domain based on the dynamic entity point set. S4. Generate static occupancy constraint indicators based on static effective point sets, and generate dynamic occupancy indicators based on time-varying occupancy envelopes to construct a time-varying risk field in the prediction time domain. S5. Obtain the current status information and target information of the UAV, combine the time-varying risk field and use the real-time diffusion planning model DiffuserLite to generate the navigation trajectory and send it to the flight control for execution; S6. Repeat S1 to S5 at the next sampling time to perform UAV autonomous navigation in a rolling update manner.

[0006] Preferably, a sliding window image sequence is formed by acquiring several consecutive image frames prior to the current sampling time using an airborne camera on the UAV, as follows: Set the sampling time interval between two adjacent image frames to be , will the Second sampling time The acquired image frames are denoted as At the current sampling time Get the current image frame ,in, Using the current sampling sequence number, a sliding window image sequence is formed by selecting several consecutive image frames from the airborne camera in chronological order of sampling time. The sliding window image sequence Represented as , This represents the total number of image frames. Sliding window image sequence Stored in the image buffer and at the next sampling time By removing the earliest image frame And add the latest image frames Update the image sequence of the sliding window. .

[0007] Preferably, the sliding window image sequence is input into the four-dimensional dot plot regression model to obtain a four-dimensional dot plot set, as follows: For sliding window image sequences Each image frame in Preprocessing is performed to obtain the preprocessed image. , ; Preprocess each frame of image Input the feature encoder of the four-dimensional point map regression model to obtain the corresponding feature map. The feature encoder is a ViT encoder; Based on all feature maps within the sliding window, feature maps for every two frames... and Constructing cross-frame association tensors ,in, and ,and ,in, for Feature location index on, for Feature location index on, Indicates transpose; The feature map set corresponding to the sliding window and cross-frame related tensor sets Input a regression network for a four-dimensional point map regression model, and output a set of four-dimensional point maps of each feature map within a sliding window. Each frame of the four-dimensional point map ,and , For the first Frame number The three-dimensional coordinates of the points For the first Frame sampling time; For each frame of the four-dimensional point map in the four-dimensional point map set Determine corresponding points across frames by using cross-frame correlation tensors: ; in, For point In feature map The position index on, and ; Calculate consistency error based on cross-frame corresponding points : ; in, Represents the L2 norm; And generate confidence scores based on consistency errors. , ,and .

[0008] Preferably, the feature map set corresponding to the sliding window and the cross-frame correlation tensor set are input into the regression network of the four-dimensional point map regression model, and the four-dimensional point map set of each feature map within the sliding window is output, as follows: For the first Frame feature map Define feature location index set ,in, For the first Frame number Each feature location index The feature vector at that location is represented as , ; two frames of feature maps and As the reference frame and support frame respectively, for any reference frame With support frame , and Based on cross-frame correlation tensors The association scores between feature positions in the reference frame and feature positions in the support frame are subjected to exponential mapping and normalization to obtain the attention weights for each feature position in the reference frame corresponding to each feature position in the support frame: ; in, , Represents the natural exponential function; Feature maps of support frames based on attention weights The feature vectors at each feature location in the data are weighted and summed to obtain the cross-frame aggregated features from the support frame aggregation to the reference frame. : ; For the same reference frame feature location, the feature vector corresponding to the reference frame feature location is concatenated with the cross-frame aggregated features from the other supporting frames within the sliding window to obtain the fused feature vector. : ; The three-dimensional coordinates of the corresponding points are obtained by linearly mapping the fused feature vectors: ; in, This is the weight matrix. It is the bias vector; The three-dimensional coordinates are combined with the sampling time of the corresponding frame to generate four-dimensional points. All four-dimensional points are aggregated into a four-dimensional point map of the image frame. The four-dimensional point maps of all image frames are then combined into a set of four-dimensional point maps of each frame within a sliding window.

[0009] Preferably, the static effective point set and the dynamic effective point set are determined based on the four-dimensional point map set, and the static effective point set and the dynamic effective point set are transformed to the UAV local coordinate system, as follows: For each point in the four-dimensional point map within each frame of the sliding window Determine the set of valid corresponding frames in the remaining frames. The set of valid corresponding frames consists of a four-dimensional dot graph that satisfies a confidence level not lower than a preset confidence threshold and has a valid cross-frame correspondence. ,in, To pre-set the reliability threshold and satisfy ; Based on each corresponding four-dimensional point map in the set of valid corresponding frames, obtain each point in the four-dimensional point map. The three-dimensional displacement vector between corresponding points across frames in the corresponding four-dimensional point map : ; And calculate the displacement magnitude of the three-dimensional displacement vector. , , Represents the L2 norm; Based on displacement modulus And the difference in sampling time to obtain normalized motion quantity : ; And further obtain each point in the four-dimensional point map. Overall exercise volume within the sliding window and maximum exercise volume : ; ; An average static judgment threshold and a maximum static judgment threshold are set, both of which are obtained based on expert experience. When the total motion volume is less than the average static judgment threshold and the maximum motion volume is less than the maximum static judgment threshold, the points in the four-dimensional point map are included in the static valid point set. , , Otherwise, it is classified into the dynamic valid point set. , , ; Establish the rigid body transformation matrix from the camera coordinate system to the UAV local coordinate system. : ; in, for Rotation matrix, for Translation vector; For each point in the static valid point set and the dynamic valid point set Perform coordinate transformation to convert to coordinates in the UAV's local coordinate system. : .

[0010] Preferably, the dynamic effective point set is clustered and grouped to obtain a dynamic entity point set, and a time-varying occupancy envelope of the corresponding dynamic entity is generated in the prediction time domain based on the dynamic entity point set, as follows: For dynamic effective point set Perform clustering and grouping processing. Corresponding sampling time When the three-dimensional Euclidean distance between any two dynamic valid points in the dynamic valid point set is less than or equal to the preset clustering distance threshold... At that time, the clustering distance threshold Designed based on expert experience. By assigning two dynamic valid points to the same dynamic cluster, we obtain a set of dynamic clusters. ; Calculate the cluster centroid for each dynamic cluster. The maximum distance from each point within the dynamic cluster to the corresponding cluster centroid is calculated as the cluster radius. : ; ; Establish dynamic cluster associations between adjacent sampling times. For any dynamic cluster at the previous sampling time, determine the dynamic cluster with the smallest centroid distance from the dynamic cluster at the previous sampling time in the dynamic cluster set at the next sampling time as the associated cluster, and use the sequence of continuously associated dynamic clusters as the dynamic entity point set. ,in, For dynamic entity numbering, For dynamic entities The set of sample numbers covered For dynamic entities In sampling sequence number The corresponding cluster index; For each dynamic entity point set, a centroid time series set is constructed based on the cluster centroid of the dynamic cluster corresponding to the dynamic entity at each sampling time and the sampling time. And adopt a linear motion model Obtain the velocity vector of a dynamic entity : ; Set the prediction time domain as In the prediction time domain The predicted centroid of the dynamic entity at each prediction time is calculated based on the velocity vector and the cluster centroid at the current sampling time. : ; Based on the cluster radius of the dynamic cluster corresponding to the dynamic entity at the current sampling time, and superimposed with a preset safety margin. Determine the envelope radius Safety margin Designed based on expert experience. The prediction centroid at each prediction time and the envelope radius are used to generate the time-varying occupancy envelope of the dynamic entity in the prediction time domain. : .

[0011] Preferably, a static occupancy constraint indicator is generated based on a static effective point set, and a dynamic occupancy indicator is generated based on a time-varying occupancy envelope, thus constructing a time-varying risk field in the prediction time domain, as follows: Set local spatial range Static expansion radius and spatial discrete resolution , , , And within the local spatial range, a set of voxel center points is constructed according to the spatial discrete resolution. , , , , It is a non-negative integer; For any voxel center point, calculate the nearest distance to the static effective point set, and generate a static occupancy constraint indicator based on the nearest distance. When the nearest distance is less than or equal to the static expansion radius, set the static occupancy constraint indicator to 1, and when the nearest distance is greater than the static expansion radius, set the static occupancy constraint indicator to 0. Get the set of IDs of dynamic entities and the corresponding time-varying occupancy envelope Set a time away from the walk And based on the prediction time domain Construct a set of discrete prediction times , , For time discrete sequence numbers; For each discrete prediction time and each voxel center point, a dynamic occupancy indicator is generated. When there is a dynamic entity that causes the voxel center point to fall within the time-varying occupancy envelope of the dynamic entity at the discrete prediction time, the dynamic occupancy indicator is set to 1; otherwise, the dynamic occupancy indicator is set to 0. The static occupancy constraint indicator and the dynamic occupancy indicator are merged to generate a time-varying risk value. : ; in, , As a weighting factor, voxel center point Static occupancy constraint indicator, Discrete prediction time voxel center point The dynamic occupancy indicator; The time-varying risk field in the prediction time domain is constructed by combining the time-varying risk values ​​corresponding to all discrete prediction times and all voxel center points.

[0012] Preferably, the current state information and target information of the UAV are acquired, combined with the time-varying risk field, and a navigation trajectory is generated using the real-time diffusion planning model DiffuserLite and sent to the flight control system for execution, as follows: Obtain the current state information of the drone and construct the current state vector. , ,in, These are the current position coordinates of the UAV in its local coordinate system. The velocity components of the UAV in the UAV's local coordinate system at the current moment; Obtain target information and construct target vector ,in, The target information refers to the navigation target that the UAV needs to reach or follow within the current planning period, where the target is the position coordinate of the target in the local coordinate system of the UAV. The conditional input for constructing a real-time diffusion planning model, DiffuserLite, based on the current state vector, the target vector, and the time-varying risk value of the time-varying risk field. ,in, ; Set the number of candidate navigation trajectories to Set the diffusion iteration steps to be The real-time diffusion planning model DiffuserLite is used to generate an initial noise sequence for each candidate navigation trajectory. The initial noise sequence consists of a length of The three-dimensional position point sequence constitutes, , To predict the time domain, The time spent walking is long; Perform an iteration step on the initial noise sequence for each candidate navigation trajectory. The reverse diffusion denoising iteration up to iteration step 0 yields the corresponding denoised sequence; The denoised sequence is parsed into candidate navigation trajectories, and flight constraints are applied to the candidate navigation trajectories to obtain candidate navigation trajectories that satisfy the flight constraints, including maximum speed constraints and maximum acceleration constraints. For each candidate navigation trajectory that meets the flight constraints, a planning refinement process is performed to generate a navigation trajectory; Control commands are generated based on the navigation trajectory and sent to the flight controller for execution.

[0013] Preferably, a planning refinement process is performed on each candidate navigation trajectory that satisfies the flight constraints to generate a navigation trajectory, as follows: Set refined time domain With refinement of the distance between the walk , , Extract the near-end trajectory segment of the candidate navigation trajectory as a refined endpoint constraint; The condition input The real-time diffusion planning model DiffuserLite generates a sequence of near-end refined segment positions that satisfy the refined endpoint constraints. The near-end refined segment position sequence is then spliced ​​with the far-end trajectory segment of the candidate navigation trajectory that satisfies the flight constraints to obtain the navigation trajectory.

[0014] Vision-guided autonomous navigation systems for unmanned aerial vehicles (UAVs) include: The image acquisition module is used to acquire a series of consecutive image frames prior to the current sampling time through the UAV's onboard camera, forming a sliding window image sequence. The four-dimensional point map regression module is used to input the sliding window image sequence into the four-dimensional point map regression model, output a four-dimensional point map set, determine the static effective point set and the dynamic effective point set based on the four-dimensional point map set, and transform the static effective point set and the dynamic effective point set to the UAV local coordinate system; The dynamic entity clustering module is used to cluster and group dynamic valid point sets to obtain dynamic entity point sets, and generate time-varying occupancy envelopes of corresponding dynamic entities in the prediction time domain based on the dynamic entity point sets. The time-varying risk field construction module is used to generate static occupancy constraint indicators based on static effective point sets and dynamic occupancy indicators based on time-varying occupancy envelopes, thereby constructing a time-varying risk field in the prediction time domain. The real-time diffusion planning module is used to acquire the current state information and target information of the UAV, combine the time-varying risk field, and use the real-time diffusion planning model DiffuserLite to generate a navigation trajectory and send it to the flight control system for execution.

[0015] The vision-guided autonomous navigation method and system for unmanned aerial vehicles proposed in this invention have the following beneficial technical effects: 1. This application obtains a four-dimensional point map set, a static effective point set, and a dynamic effective point set by inputting a sliding window image sequence into a dynamic scene four-dimensional point map regression model. After coordinate transformation, the dynamic points are clustered and associated to form a dynamic entity point set. Furthermore, a time-varying occupancy envelope of the dynamic entities is generated in the prediction time domain. Based on this, the static occupancy constraint and the time-varying occupancy envelope are fused to construct a time-varying risk field that changes with time. This enables the formation of a spatiotemporally consistent environmental representation in a dynamic environment, reduces the risk of feature mismatch and map pollution caused by dynamic targets, and explicitly incorporates the future occupancy range of dynamic obstacles into navigation constraints, thereby improving the stability and safety of autonomous navigation in complex dynamic scenes.

[0016] 2. This application uses the current state information of the UAV, target information, and the time-varying risk field as input conditions to the real-time diffusion planning model DiffuserLite. Under the premise of satisfying flight constraints, candidate navigation trajectories are generated and a planning refinement process is performed to obtain the output navigation trajectory, realizing real-time planning with rolling updates. By directly introducing time-varying risk constraints in the planning stage and combining them with the refinement mechanism, a high replanning frequency can be maintained while ensuring trajectory feasibility, smoothness, and safety margin. This reduces the near-sharp turns and trajectory jitter that occur in traditional reactive obstacle avoidance during high-speed flight, thereby improving the autonomous navigation efficiency and mission completion reliability of the UAV in scenarios with weak GNSS, obstruction, and dynamic interference. Attached Figure Description

[0017] Figure 1 The flowchart shows the vision-guided autonomous navigation method for unmanned aerial vehicles (UAVs) according to the present invention. Figure 2 This is a schematic diagram of the vision-guided autonomous navigation system for unmanned aerial vehicles (UAVs) according to the present invention. Detailed Implementation

[0018] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar symbols denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0019] like Figure 1 The vision-guided autonomous navigation method for unmanned aerial vehicles (UAVs) shown includes the following steps: S1. A sliding window image sequence is formed by acquiring several consecutive image frames before the current sampling time using the drone's onboard camera. In an optional embodiment, a sliding window image sequence is formed by acquiring several consecutive image frames prior to the current sampling time using an airborne camera on a drone, as follows: Set the sampling time interval between two adjacent image frames to be , will the Second sampling time The acquired image frames are denoted as At the current sampling time Get the current image frame ,in, Using the current sampling sequence number, a sliding window image sequence is formed by selecting several consecutive image frames from the airborne camera in chronological order of sampling time. The sliding window image sequence Represented as , This represents the total number of image frames. Sliding window image sequence Stored in the image buffer and at the next sampling time By removing the earliest image frame And add the latest image frames Update the image sequence of the sliding window. .

[0020] S2. Input the sliding window image sequence into the four-dimensional point map regression model to obtain the four-dimensional point map set. Based on the four-dimensional point map set, determine the static effective point set and the dynamic effective point set, and transform the static effective point set and the dynamic effective point set to the UAV local coordinate system. In an optional embodiment, the sliding window image sequence is input into a four-dimensional point plot regression model to obtain a four-dimensional point plot set, as follows: For sliding window image sequences Each image frame in Preprocessing is performed to obtain the preprocessed image. , ; Preprocess each frame of image Input the feature encoder of the four-dimensional point map regression model to obtain the corresponding feature map. The feature encoder is a ViT encoder; Based on all feature maps within the sliding window, feature maps for every two frames... and Constructing cross-frame association tensors ,in, and ,and ,in, for Feature location index on, for Feature location index on, Indicates transpose; The feature map set corresponding to the sliding window and cross-frame related tensor sets Input a regression network for a four-dimensional point map regression model, and output a set of four-dimensional point maps of each feature map within a sliding window. Each frame of the four-dimensional point map ,and , For the first Frame number The three-dimensional coordinates of the points For the first Frame sampling time; For each frame of the four-dimensional point map in the four-dimensional point map set Determine corresponding points across frames by using cross-frame correlation tensors: ; in, For point In feature map The position index on, and ; Calculate consistency error based on cross-frame corresponding points : ; in, Represents the L2 norm; And generate confidence scores based on consistency errors. , ,and .

[0021] In an optional embodiment, the feature map set corresponding to the sliding window and the cross-frame correlation tensor set are input into the regression network of the four-dimensional point map regression model, and the four-dimensional point map set of each feature map within the sliding window is output, as follows: For the first Frame feature map Define feature location index set ,in, For the first Frame number Each feature location index The feature vector at that location is represented as , ; two frames of feature maps and As the reference frame and support frame respectively, for any reference frame With support frame , and Based on cross-frame correlation tensors The association scores between feature positions in the reference frame and feature positions in the support frame are subjected to exponential mapping and normalization to obtain the attention weights for each feature position in the reference frame corresponding to each feature position in the support frame: ; in, , Represents the natural exponential function; Feature maps of support frames based on attention weights The feature vectors at each feature location in the data are weighted and summed to obtain the cross-frame aggregated features from the support frame aggregation to the reference frame. : ; For the same reference frame feature location, the feature vector corresponding to the reference frame feature location is concatenated with the cross-frame aggregated features from the other supporting frames within the sliding window to obtain the fused feature vector. : ; The three-dimensional coordinates of the corresponding points are obtained by linearly mapping the fused feature vectors: ; in, This is the weight matrix. It is the bias vector; The three-dimensional coordinates are combined with the sampling time of the corresponding frame to generate four-dimensional points. All four-dimensional points are aggregated into a four-dimensional point map of the image frame. The four-dimensional point maps of all image frames are then combined into a set of four-dimensional point maps of each frame within a sliding window.

[0022] In an optional embodiment, a static effective point set and a dynamic effective point set are determined based on a four-dimensional point map set, and the static effective point set and the dynamic effective point set are transformed to the UAV local coordinate system, as follows: For each point in the four-dimensional point map within each frame of the sliding window Determine the set of valid corresponding frames in the remaining frames. The set of valid corresponding frames consists of a four-dimensional dot graph that satisfies a confidence level not lower than a preset confidence threshold and has a valid cross-frame correspondence. ,in, To pre-set the reliability threshold and satisfy ; Based on each corresponding four-dimensional point map in the set of valid corresponding frames, obtain each point in the four-dimensional point map. The three-dimensional displacement vector between corresponding points across frames in the corresponding four-dimensional point map : ; And calculate the displacement magnitude of the three-dimensional displacement vector. , , Represents the L2 norm; Based on displacement modulus And the difference in sampling time to obtain normalized motion quantity : ; And further obtain each point in the four-dimensional point map. Overall exercise volume within the sliding window and maximum exercise volume : ; ; An average static judgment threshold and a maximum static judgment threshold are set, both of which are obtained based on expert experience. When the total motion volume is less than the average static judgment threshold and the maximum motion volume is less than the maximum static judgment threshold, the points in the four-dimensional point map are included in the static valid point set. , , Otherwise, it is classified into the dynamic valid point set. , , ; Establish the rigid body transformation matrix from the camera coordinate system to the UAV local coordinate system. : ; in, for Rotation matrix, for Translation vector; For each point in the static valid point set and the dynamic valid point set Perform coordinate transformation to convert to coordinates in the UAV's local coordinate system. : .

[0023] S3. Cluster the dynamic effective point set to obtain the dynamic entity point set, and generate the time-varying occupancy envelope of the corresponding dynamic entity in the prediction time domain based on the dynamic entity point set. In an optional embodiment, the dynamic valid point set is clustered to obtain a dynamic entity point set, and a time-varying occupancy envelope of the corresponding dynamic entity is generated in the prediction time domain based on the dynamic entity point set, as follows: For dynamic effective point set Perform clustering and grouping processing. Corresponding sampling time When the three-dimensional Euclidean distance between any two dynamic valid points in the dynamic valid point set is less than or equal to the preset clustering distance threshold... At that time, the clustering distance threshold Designed based on expert experience. By assigning two dynamic valid points to the same dynamic cluster, we obtain a set of dynamic clusters. ; Calculate the cluster centroid for each dynamic cluster. The maximum distance from each point within the dynamic cluster to the corresponding cluster centroid is calculated as the cluster radius. : ; ; Establish dynamic cluster associations between adjacent sampling times. For any dynamic cluster at the previous sampling time, determine the dynamic cluster with the smallest centroid distance from the dynamic cluster at the previous sampling time in the dynamic cluster set at the next sampling time as the associated cluster, and use the sequence of continuously associated dynamic clusters as the dynamic entity point set. ,in, For dynamic entity numbering, For dynamic entities The set of sample numbers covered For dynamic entities In sampling sequence number The corresponding cluster index; For each dynamic entity point set, a centroid time series set is constructed based on the cluster centroid of the dynamic cluster corresponding to the dynamic entity at each sampling time and the sampling time. And adopt a linear motion model Obtain the velocity vector of a dynamic entity : ; Set the prediction time domain as In the prediction time domain The predicted centroid of the dynamic entity at each prediction time is calculated based on the velocity vector and the cluster centroid at the current sampling time. : ; Based on the cluster radius of the dynamic cluster corresponding to the dynamic entity at the current sampling time, and superimposed with a preset safety margin. Determine the envelope radius Safety margin Designed based on expert experience. The prediction centroid at each prediction time and the envelope radius are used to generate the time-varying occupancy envelope of the dynamic entity in the prediction time domain. : .

[0024] S4. Generate static occupancy constraint indicators based on static effective point sets, and generate dynamic occupancy indicators based on time-varying occupancy envelopes to construct a time-varying risk field in the prediction time domain. In an optional embodiment, a static occupancy constraint indicator is generated based on a static valid point set, and a dynamic occupancy indicator is generated based on a time-varying occupancy envelope, constructing a time-varying risk field in the prediction time domain, as follows: Set local spatial range Static expansion radius and spatial discrete resolution , , , And within the local spatial range, a set of voxel center points is constructed according to the spatial discrete resolution. , , , , It is a non-negative integer; For any voxel center point, calculate the nearest distance to the static effective point set, and generate a static occupancy constraint indicator based on the nearest distance. When the nearest distance is less than or equal to the static expansion radius, set the static occupancy constraint indicator to 1, and when the nearest distance is greater than the static expansion radius, set the static occupancy constraint indicator to 0. Get the set of IDs of dynamic entities and the corresponding time-varying occupancy envelope Set a time away from the walk And based on the prediction time domain Construct a set of discrete prediction times , , For time discrete sequence numbers; For each discrete prediction time and each voxel center point, a dynamic occupancy indicator is generated. When there is a dynamic entity that causes the voxel center point to fall within the time-varying occupancy envelope of the dynamic entity at the discrete prediction time, the dynamic occupancy indicator is set to 1; otherwise, the dynamic occupancy indicator is set to 0. The static occupancy constraint indicator and the dynamic occupancy indicator are merged to generate a time-varying risk value. : ; in, , As a weighting factor, voxel center point Static occupancy constraint indicator, Discrete prediction time voxel center point The dynamic occupancy indicator; The time-varying risk field in the prediction time domain is constructed by combining the time-varying risk values ​​corresponding to all discrete prediction times and all voxel center points.

[0025] S5. Obtain the current status information and target information of the UAV, combine the time-varying risk field and use the real-time diffusion planning model DiffuserLite to generate the navigation trajectory and send it to the flight control for execution; In an optional embodiment, the current state information and target information of the UAV are obtained, combined with the time-varying risk field, and a navigation trajectory is generated using the real-time diffusion planning model DiffuserLite and sent to the flight control system for execution, as follows: Obtain the current state information of the drone and construct the current state vector. , ,in, These are the current position coordinates of the UAV in its local coordinate system. The velocity components of the UAV in the UAV's local coordinate system at the current moment; Obtain target information and construct target vector ,in, The target information refers to the navigation target that the UAV needs to reach or follow within the current planning period, where the target is the position coordinate of the target in the local coordinate system of the UAV. The conditional input for constructing a real-time diffusion planning model, DiffuserLite, based on the current state vector, the target vector, and the time-varying risk value of the time-varying risk field. ,in, ; Set the number of candidate navigation trajectories to Set the diffusion iteration steps to be The real-time diffusion planning model DiffuserLite is used to generate an initial noise sequence for each candidate navigation trajectory. The initial noise sequence consists of a length of The three-dimensional position point sequence constitutes, , To predict the time domain, The time spent walking is long; Perform an iteration step on the initial noise sequence for each candidate navigation trajectory. The reverse diffusion denoising iteration up to iteration step 0 yields the corresponding denoised sequence; The denoised sequence is parsed into candidate navigation trajectories, and flight constraints are applied to the candidate navigation trajectories to obtain candidate navigation trajectories that satisfy the flight constraints, including maximum speed constraints and maximum acceleration constraints. For each candidate navigation trajectory that meets the flight constraints, a planning refinement process is performed to generate a navigation trajectory; Control commands are generated based on the navigation trajectory and sent to the flight controller for execution.

[0026] In an optional embodiment, a planning refinement process is performed on each candidate navigation trajectory that satisfies the flight constraints to generate a navigation trajectory, as follows: Set refined time domain With refinement of the distance between the walk , , Extract the near-end trajectory segment of the candidate navigation trajectory as a refined endpoint constraint; The condition input The real-time diffusion planning model DiffuserLite generates a sequence of near-end refined segment positions that satisfy the refined endpoint constraints. The near-end refined segment position sequence is then spliced ​​with the far-end trajectory segment of the candidate navigation trajectory that satisfies the flight constraints to obtain the navigation trajectory.

[0027] S6. Repeat S1 to S5 at the next sampling time to perform UAV autonomous navigation in a rolling update manner.

[0028] like Figure 2 The vision-guided autonomous navigation system for unmanned aerial vehicles shown includes: The image acquisition module is used to acquire a series of consecutive image frames prior to the current sampling time through the UAV's onboard camera, forming a sliding window image sequence. The four-dimensional point map regression module is used to input the sliding window image sequence into the four-dimensional point map regression model, output a four-dimensional point map set, determine the static effective point set and the dynamic effective point set based on the four-dimensional point map set, and transform the static effective point set and the dynamic effective point set to the UAV local coordinate system; The dynamic entity clustering module is used to cluster and group dynamic valid point sets to obtain dynamic entity point sets, and generate time-varying occupancy envelopes of corresponding dynamic entities in the prediction time domain based on the dynamic entity point sets. The time-varying risk field construction module is used to generate static occupancy constraint indicators based on static effective point sets and dynamic occupancy indicators based on time-varying occupancy envelopes, thereby constructing a time-varying risk field in the prediction time domain. The real-time diffusion planning module is used to acquire the current state information and target information of the UAV, combine the time-varying risk field, and use the real-time diffusion planning model DiffuserLite to generate a navigation trajectory and send it to the flight control system for execution.

[0029] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A vision-guided autonomous navigation method for unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: S1. A sliding window image sequence is formed by acquiring several consecutive image frames before the current sampling time using the drone's onboard camera. S2. Input the sliding window image sequence into the four-dimensional point map regression model to obtain the four-dimensional point map set. Based on the four-dimensional point map set, determine the static effective point set and the dynamic effective point set, and transform the static effective point set and the dynamic effective point set to the UAV local coordinate system. S3. Cluster the dynamic effective point set to obtain the dynamic entity point set, and generate the time-varying occupancy envelope of the corresponding dynamic entity in the prediction time domain based on the dynamic entity point set. S4. Generate static occupancy constraint indicators based on static effective point sets, and generate dynamic occupancy indicators based on time-varying occupancy envelopes to construct a time-varying risk field in the prediction time domain. S5. Obtain the current status information and target information of the UAV, combine the time-varying risk field and use the real-time diffusion planning model DiffuserLite to generate the navigation trajectory and send it to the flight control for execution; S6. Repeat S1 to S5 at the next sampling time to perform UAV autonomous navigation in a rolling update manner.

2. The vision-guided autonomous navigation method for unmanned aerial vehicles according to claim 1, characterized in that, A sliding window image sequence is constructed by acquiring several consecutive image frames prior to the current sampling time using an airborne camera on a drone, as shown below: Set the sampling time interval between two adjacent image frames, acquire the current image frame at the current sampling time, and select several consecutive image frames from the airborne camera in the order of sampling time to form a sliding window image sequence; The sliding window image sequence is stored in the image buffer, and at the next sampling time, the sliding window image sequence is updated by removing the earliest image frame and adding the latest image frame.

3. The vision-guided autonomous navigation method for unmanned aerial vehicles according to claim 1, characterized in that, The sliding window image sequence is input into the four-dimensional dot plot regression model to obtain a set of four-dimensional dot plots, as follows: Preprocessing is performed on each frame of the sliding window image sequence to obtain a preprocessed image; Each preprocessed image frame is input into the feature encoder of the four-dimensional point map regression model to obtain the corresponding feature map. Construct a cross-frame association tensor for every two different feature maps based on all feature maps within the sliding window; Input the feature map set corresponding to the sliding window and the cross-frame correlation tensor set into the regression network of the four-dimensional point map regression model, and output the four-dimensional point map set of each feature map within the sliding window; For each frame point in the four-dimensional point graph set, the corresponding cross-frame point is determined by the cross-frame association tensor. The consistency error is calculated based on the corresponding points across frames, and the confidence level is generated based on the consistency error.

4. The vision-guided autonomous navigation method for unmanned aerial vehicles according to claim 3, characterized in that, The feature map set corresponding to the sliding window and the cross-frame correlation tensor set are input into the regression network of the four-dimensional point map regression model, and the output is the four-dimensional point map set of each feature map within the sliding window, as follows: Two different feature maps are used as the reference frame and the support frame, respectively. Based on the association scores between the feature positions of the reference frame and each feature position of the support frame in the cross-frame association tensor, exponential mapping and normalization are performed to obtain the attention weight of each feature position of the reference frame corresponding to each feature position of the support frame. Based on the attention weights, the feature vectors at each feature position in the feature map of the support frame are weighted and summed to obtain the cross-frame aggregated features from the support frame to the reference frame. For the same reference frame feature position, the feature vector corresponding to the reference frame feature position is concatenated with the cross-frame aggregated features from the other supporting frames within the sliding window to obtain the fused feature vector; The three-dimensional coordinates of the corresponding points are obtained by linearly mapping the fused feature vectors. The three-dimensional coordinates are combined with the sampling time of the corresponding frame to generate four-dimensional points. All four-dimensional points are collected into a four-dimensional point map of the image frame. The four-dimensional point maps of all image frames are combined into a set of four-dimensional point maps of each frame within the sliding window.

5. The vision-guided autonomous navigation method for unmanned aerial vehicles according to claim 1, characterized in that, Based on the four-dimensional point map set, the static and dynamic valid point sets are determined, and then the static and dynamic valid point sets are transformed to the UAV local coordinate system, as follows: For each point in the four-dimensional dot graph of each frame within the sliding window, a set of valid corresponding frames in the remaining frames is determined. The set of valid corresponding frames consists of four-dimensional dot graphs that satisfy a confidence level not lower than a preset confidence threshold and have a valid cross-frame correspondence. Based on each corresponding four-dimensional point map in the effective corresponding frame set, obtain the three-dimensional displacement vector between each point in the four-dimensional point map and the corresponding cross-frame point in the corresponding four-dimensional point map, and calculate the displacement magnitude of the three-dimensional displacement vector. The normalized motion is obtained based on the displacement modulus and the difference in sampling time, and the comprehensive motion and maximum motion of each point in the four-dimensional point map within the sliding window are further obtained. Set an average static judgment threshold and a maximum static judgment threshold. When the total motion volume is less than the average static judgment threshold and the maximum motion volume is less than the maximum static judgment threshold, the points in the four-dimensional point map are assigned to the static valid point set; otherwise, they are assigned to the dynamic valid point set. Establish a rigid body transformation matrix from the camera coordinate system to the UAV local coordinate system, and perform coordinate transformation on each point in the static effective point set and the dynamic effective point set to transform it into coordinates in the UAV local coordinate system.

6. The vision-guided autonomous navigation method for unmanned aerial vehicles according to claim 1, characterized in that, The dynamic effective point set is clustered and grouped to obtain the dynamic entity point set. Based on the dynamic entity point set, the time-varying occupancy envelope of the corresponding dynamic entity is generated in the prediction time domain, as follows: Clustering and grouping processing is performed on the dynamic valid point set. When the three-dimensional Euclidean distance between any two dynamic valid points in the dynamic valid point set is less than or equal to the preset clustering distance threshold, the two dynamic valid points are assigned to the same dynamic cluster to obtain a dynamic cluster set. For each dynamic cluster, calculate the cluster centroid and use the maximum distance from each point within the dynamic cluster to the corresponding cluster centroid as the cluster radius; For any dynamic cluster at the previous sampling time, the dynamic cluster with the smallest centroid distance to the dynamic cluster at the previous sampling time is determined from the dynamic cluster set at the next sampling time as the associated cluster, and the sequence of dynamic clusters obtained by continuous association is taken as the dynamic entity point set. For each set of dynamic entity points, a centroid time series set is constructed based on the cluster centroid of the dynamic cluster corresponding to the dynamic entity at each sampling time and the sampling time, and the velocity vector of the dynamic entity is obtained by using a linear motion model. In the prediction time domain, the predicted centroid of the dynamic entity at each prediction time is calculated based on the velocity vector and the cluster centroid at the current sampling time. The envelope radius is determined by superimposing a preset safety margin on the cluster radius of the dynamic entity corresponding to the dynamic cluster at the current sampling time, and the time-varying occupancy envelope of the dynamic entity in the prediction time domain is generated by the prediction centroid of each prediction time and the envelope radius.

7. The vision-guided autonomous navigation method for unmanned aerial vehicles according to claim 1, characterized in that, Static occupancy constraint indicators are generated based on static valid point sets, and dynamic occupancy indicators are generated based on time-varying occupancy envelopes. A time-varying risk field is constructed in the prediction time domain as follows: Define the local spatial range, static expansion radius, and spatial discrete resolution, and construct a set of voxel center points within the local spatial range according to the spatial discrete resolution; For any voxel center point, calculate the nearest distance to the static effective point set, and generate a static occupancy constraint indicator based on the nearest distance. When the nearest distance is less than or equal to the static expansion radius, set the static occupancy constraint indicator to 1, and when the nearest distance is greater than the static expansion radius, set the static occupancy constraint indicator to 0. Obtain the set of dynamic entity numbers and their corresponding time-varying occupancy envelopes, set the time discrete step size, and construct a set of discrete prediction time points based on the prediction time domain; For each discrete prediction time and each voxel center point, a dynamic occupancy indicator is generated. When there is a dynamic entity that causes the voxel center point to fall within the time-varying occupancy envelope of the dynamic entity at the discrete prediction time, the dynamic occupancy indicator is set to 1; otherwise, the dynamic occupancy indicator is set to 0. The static occupancy constraint indicator and the dynamic occupancy indicator are fused to generate a time-varying risk value, and the time-varying risk values ​​corresponding to all discrete prediction times and all voxel center points are used to form a time-varying risk field in the prediction time domain.

8. The vision-guided autonomous navigation method for unmanned aerial vehicles according to claim 1, characterized in that, The current state and target information of the UAV are acquired, combined with a time-varying risk field, and a navigation trajectory is generated using the real-time diffusion planning model DiffuserLite, and then sent to the flight control system for execution, as follows: Acquire the current state information and target information of the UAV and construct the current state vector and target vector; The conditional inputs of the real-time diffusion planning model DiffuserLite are constructed based on the current state vector, the target vector, and the time-varying risk value of the time-varying risk field. The real-time diffusion planning model DiffuserLite is used to generate the corresponding initial noise sequence for each candidate navigation trajectory. For each candidate navigation trajectory, perform backdiffusion denoising iteration on the initial noise sequence to obtain the corresponding denoised sequence; The denoised sequence is parsed into candidate navigation trajectories, and flight constraints are applied to the candidate navigation trajectories to obtain candidate navigation trajectories that satisfy the flight constraints, including maximum speed constraints and maximum acceleration constraints. For each candidate navigation trajectory that meets the flight constraints, a planning refinement process is performed to generate the navigation trajectory. Based on the navigation trajectory, control commands are generated and sent to the flight control system for execution.

9. The vision-guided autonomous navigation method for unmanned aerial vehicles according to claim 8, characterized in that, For each candidate navigation trajectory that meets the flight constraints, a planning refinement process is performed to generate the navigation trajectory, as follows: Set the refined time domain and refined discrete step length, and extract the near-end trajectory segment of the candidate navigation trajectory as the refined endpoint constraint; Under the given input conditions, the real-time diffusion planning model DiffuserLite generates a sequence of near-end refined segment positions that satisfy the refined endpoint constraints. The near-end refined segment position sequence is then concatenated with the far-end trajectory segment of the candidate navigation trajectory that satisfies the flight constraints to obtain the navigation trajectory.

10. A vision-guided autonomous navigation system for unmanned aerial vehicles (UAVs), used in accordance with any one of claims 1 to 9, characterized in that, include: The image acquisition module is used to acquire a series of consecutive image frames prior to the current sampling time through the UAV's onboard camera, forming a sliding window image sequence. The four-dimensional point map regression module is used to input the sliding window image sequence into the four-dimensional point map regression model, output a four-dimensional point map set, determine the static effective point set and the dynamic effective point set based on the four-dimensional point map set, and transform the static effective point set and the dynamic effective point set to the UAV local coordinate system; The dynamic entity clustering module is used to cluster and group dynamic valid point sets to obtain dynamic entity point sets, and generate time-varying occupancy envelopes of corresponding dynamic entities in the prediction time domain based on the dynamic entity point sets. The time-varying risk field construction module is used to generate static occupancy constraint indicators based on static effective point sets and dynamic occupancy indicators based on time-varying occupancy envelopes, thereby constructing a time-varying risk field in the prediction time domain. The real-time diffusion planning module is used to acquire the current state information and target information of the UAV, combine the time-varying risk field, and use the real-time diffusion planning model DiffuserLite to generate a navigation trajectory and send it to the flight control system for execution.

Images