A lightweight feature extraction method based on unmanned aerial vehicle vision slam

Abstract

This invention discloses a lightweight feature extraction method based on UAV visual SLAM, comprising the following steps: Step 1: Acquire continuous RGB image frames from the UAV to form a flight image sequence; Step 2: Input the data into a lightweight feature encoding network to generate feature maps of the current frame and adjacent frames; Step 3: Generate a keypoint score map and a dense descriptor map; Step 4: Generate a trajectory locking direction and a trajectory locking sampling grid; Step 5: Generate a descriptor separability map and an interlocking filtering map; Step 6: Determine candidate sampling positions and generate sampling weights; Step 7: Generate sub-pixel keypoint coordinates; Step 8: Sample feature descriptors and output SLAM image features. This invention improves the localization accuracy, matching stability, and real-time processing capability of UAV visual SLAM feature extraction.

Description

Technical Field

[0001] This invention relates to the field of UAV visual SLAM technology, and in particular to a lightweight feature extraction method based on UAV visual SLAM. Background Technology

[0002] With the increasing demand for autonomous inspection, low-altitude mapping, and navigation in complex environments using unmanned aerial vehicles (UAVs), lightweight image feature extraction technology for UAV visual SLAM front-ends has attracted widespread attention. Existing UAV visual SLAM feature extraction methods mainly rely on traditional keypoint detection, SuperPoint-like deep networks, or ordinary differentiable localization models for keypoint extraction and descriptor generation. However, these methods generally suffer from the following problems in practical applications: During continuous drone flight, image frames exhibit rapid changes in viewpoint, scale, and texture direction. Fixed-window keypoint sampling struggles to adapt to image patch migration trends, leading to insufficient keypoint repeatability between adjacent frames and decreased trajectory tracking stability. Existing keypoint head branches and descriptor head branches typically output in parallel, with keypoint selection primarily based on fractional responses. Descriptor separability fails to participate in sampling position determination beforehand, easily retaining unstable feature points in low-texture and repetitive texture regions. Traditional NMS keypoint selection processes are non-differentiable, and ordinary soft-argmax struggles to constrain the shape of the fractional map distribution, making sub-pixel keypoint coordinate generation prone to positional shifts, affecting feature descriptor sampling consistency and the reliability of SLAM image feature matching.

[0003] Therefore, how to provide a lightweight feature extraction method based on UAV visual SLAM is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] One objective of this invention is to propose a lightweight feature extraction method based on UAV visual SLAM. This invention utilizes a lightweight feature encoding network, trajectory-locked sampling grid, interlocked filtering map, and improved sampling argmax sampling to complete the extraction of sub-pixel key points and generation of feature descriptors in continuous RGB image frames of UAVs. It has the advantages of low computational overhead, high key point localization accuracy, strong descriptor matching stability, and strong adaptability to changes in flight perspective.

[0005] A lightweight feature extraction method based on UAV visual SLAM according to an embodiment of the present invention includes the following steps: Step 1: Acquire continuous RGB image frames from the drone to form a flight image sequence; Step 2: Input the flight image sequence into a lightweight feature coding network to generate feature maps of the current frame and adjacent frames; Step 3: Input the current frame feature map into the key point head branch and the descriptor head branch to generate the key point score map and the dense descriptor map; Step 4: Extract the image patch migration direction, scale change amount and texture main direction based on the feature maps of adjacent frames, generate the trajectory locking direction, and non-uniformly grid the key point fraction map to form the trajectory locking sampling grid; Step 5: Generate a descriptor separability graph based on the similarity of neighboring descriptors in the dense descriptor graph, and align the descriptor separability graph with the keypoint score graph to form an interlocked filtering graph; Step 6: Determine candidate sampling positions within the trajectory-locked sampling grid based on the interlocked screening diagram, and perform improved sampling argmax sampling to generate sampling weights; Step 7: Perform reparameterized sampling on the sampling weights and generate sub-pixel keypoint coordinates by combining the expected sampling position error; Step 8: Sample feature descriptors in the dense descriptor map according to the sub-pixel keypoint coordinates, and output SLAM image features based on the sub-pixel keypoint coordinates and feature descriptors.

[0006] Optionally, step one specifically includes: The UAV's visual sensor continuously acquires RGB image frames of the flight environment at a preset acquisition frequency, and configures the acquisition time, frame number and image size identifier for each RGB image frame to form an original image frame set. The original image frame set is sorted by time according to the acquisition time, the current RGB image frame and adjacent RGB image frames are determined according to the frame number, and the temporal adjacency relationship between the current RGB image frame and adjacent RGB image frames is established. Based on the image size identifier, the current RGB image frame and adjacent RGB image frames are size-normalized, pixel normalization is performed based on the pixel value range, and channel unification is performed based on the channel arrangement rules, forming a normalized image frame set; The regular image frame set is continuously arranged according to the temporal adjacency relationship, and the result of the continuous arrangement is determined as a flight image sequence.

[0007] Optionally, step two specifically involves: Read the current RGB image frame and adjacent RGB image frames from the flight image sequence, map the current RGB image frame to the current image tensor, map adjacent RGB image frames to adjacent image tensors, and input the current image tensor and adjacent image tensors into a lightweight feature encoding network; The lightweight feature coding network includes a shallow texture coding unit, a scale compression coding unit, and an inter-frame sharing mapping unit. The shallow texture coding unit performs edge texture mapping on the current image tensor and the adjacent image tensors to generate the current shallow feature record and the adjacent shallow feature record. The scale compression coding unit performs channel compression, spatial downsampling, and hierarchical aggregation on the current shallow feature record and adjacent shallow feature records to generate the current scale feature record and adjacent scale feature records; The inter-frame shared mapping unit performs feature alignment mapping on the current scale feature record and the adjacent scale feature record according to the unified network parameters, and generates the current frame feature map and the adjacent frame feature map based on the feature alignment mapping result.

[0008] Optionally, step three specifically includes: Extract feature locations, channel response values, and scale level identifiers from the current frame feature map. Establish branch input records according to feature locations and input the branch input records into the key point head branch and descriptor head branch. The key point branch performs key point response mapping based on the channel response values ​​in the branch input record, and performs resolution unification and response fusion on the key point response mapping results based on the scale level identifier to generate key point response records; Perform numerical normalization and positional arrangement on the keypoint response records, and write the normalized keypoint response values ​​into the feature positions to form a keypoint score map. The descriptor head branch performs descriptor vector mapping based on the channel response values ​​in the branch input record, and performs resolution unification and vector fusion on the descriptor vector mapping result based on the scale level identifier to generate a descriptor response record; The descriptor response record is normalized and its position is arranged. The normalized descriptor vector is written to the feature position to form a dense descriptor graph.

[0009] Optionally, step four specifically includes: The feature map of the current frame and the feature map of the adjacent frames are divided into feature blocks according to the feature position. The position offset is calculated based on the position change between the feature block of the current frame and the feature block of the adjacent frames, and the image block migration direction is generated based on the position offset. Based on the changes in the response range of the current frame feature block and the adjacent frame feature blocks at the scale level, the scale level expansion and scale level contraction are calculated, and the scale change is generated based on the scale level expansion and scale level contraction. The texture direction response is calculated based on the channel response value distribution in the current frame feature block, and the direction with the largest response intensity is determined as the main texture direction. The image patch migration direction, scale change amount, and texture main direction are fused to generate a trajectory locking direction, and the local fraction map region in the key point fraction map is determined based on the trajectory locking direction. Grid density and grid spacing are adjusted in the local fractional map region along the trajectory locking direction to form a non-uniform grid result, and the non-uniform grid result is determined as the trajectory locking sampling grid.

[0010] Optionally, step five specifically includes: Read the center descriptor vector from the dense descriptor graph according to the feature position, define the neighborhood position range around the feature position where the center descriptor vector is located, and extract the neighborhood descriptor vector in the neighborhood position range; Calculate the vector similarity between the center descriptor vector and the neighborhood descriptor vector, arrange the vector similarity according to the feature positions in the neighborhood location range, and generate the neighborhood descriptor similarity; The degree of distinction between the center descriptor vector and the neighbor descriptor vectors is calculated based on the similarity of the neighbor descriptors. The degree of distinction is written into the feature position of the center descriptor vector to generate a descriptor separability map. Perform position matching between the feature locations in the descriptor separability graph and the feature locations in the keypoint score graph, and write the discriminative power in the descriptor separability graph and the keypoint response value in the keypoint score graph to the same matching location. Joint filtering is performed based on the degree of differentiation and key point response values ​​in the same matching position to generate interlocked filtering values. The interlocked filtering values ​​are then arranged according to feature positions to form an interlocked filtering map.

[0011] Optionally, step six specifically includes: Read the interlocking screening value from the interlocking screening map, read the grid point position, grid point spacing and grid point density from the trajectory locking sampling grid, and establish the position mapping relationship between the interlocking screening map and the trajectory locking sampling grid according to the feature position; Based on the position mapping relationship, the interlocking filter value is written into the grid position in the trajectory locking sampling grid to generate a grid filter record. The grid filter record is then judged for retention based on the interlocking filter value to determine the candidate sampling position. Candidate sampling positions are arranged according to the trajectory locking direction. Candidate sampling scores are generated based on the interlocking filter value, grid spacing and grid density corresponding to the candidate sampling positions, and the candidate sampling scores are written into the candidate sampling positions. Normalize the candidate sampling scores to generate a candidate sampling distribution, and introduce a Gumbel perturbation term into the candidate sampling distribution to perform improved sampling argmax sampling. The sampling probabilities corresponding to the candidate sampling positions are calculated based on the sampling results of the improved sampling argmax, and the sampling probabilities are arranged according to the candidate sampling positions to generate sampling weights.

[0012] Optionally, step seven specifically includes: Read the sampling probability corresponding to the candidate sampling position from the sampling weight, associate and organize the candidate sampling position, sampling probability and trajectory locking direction, and generate a sampling weight record; Based on the sampling weight record, a reparameterized sampling variable is constructed, the sampling probability is converted into a differentiable sampling coefficient, and the independent noise variable is written into the differentiable sampling coefficient to generate the reparameterized sampling coefficient; Based on the reparameterized sampling coefficients, the candidate sampling positions are weighted and sampled to generate sampling sample positions. The sampling sample positions are then arranged according to the trajectory locking direction to form a sampling sample position sequence. The sampling position difference is calculated between the sampled position and the candidate sampling position to generate the sampling position error. The sampling position error is then weighted and statistically analyzed according to the sampling weight to generate the expected sampling position error. Based on the expected sampling position error, position convergence processing is performed on the sampling sample position sequence, and the position convergence processing result is mapped to the coordinate range of the key point fraction map to generate sub-pixel key point coordinates.

[0013] Optionally, step eight specifically includes: Read the coordinate positions from the sub-pixel keypoint coordinates, map the coordinate positions to the feature position range of the dense descriptor map, and generate the descriptor sampling positions; Based on the descriptor sampling position, the adjacent feature positions are located in the dense descriptor graph. The descriptor vectors in the adjacent feature positions are read, and interpolation weights are generated based on the positional distance between the descriptor sampling position and the adjacent feature position. Weighted fusion of the descriptor vector is performed based on the interpolation weights to generate a feature descriptor that matches the coordinates of sub-pixel key points; Sub-pixel keypoint coordinates and feature descriptors are written into the same feature record. The feature records are arranged according to the frame number in the flight image sequence to generate a SLAM feature record set. Subpixel keypoint coordinates, feature descriptors, and frame numbers are extracted from the SLAM feature record set, and then organized according to the frame numbers to generate SLAM image features.

[0014] The beneficial effects of this invention are: This invention arranges continuous RGB image frames of a UAV into a flight image sequence and uses a lightweight feature encoding network to synchronously generate feature maps of the current frame and adjacent frames. This allows key point extraction to no longer be limited to the response of a single static image frame, but can combine the image block migration direction, scale change, and main texture direction between adjacent frames to generate a trajectory locking direction. Then, non-uniform gridding is performed on the key point fractional map to form a trajectory locking sampling grid. This enables the sampling area to adapt to the changes in viewpoint, scale, and texture extension during the flight of the UAV, reducing key point drift caused by fixed window sampling and improving the repeatability of feature points in adjacent image frames and the stability of trajectory tracking.

[0015] This invention generates a descriptor separability map based on the similarity of neighboring descriptors in a dense descriptor map, and aligns the descriptor separability map with a keypoint score map to form an interlocked filtering map. This allows keypoint response intensity and descriptor discriminative ability to jointly participate in the determination of candidate sampling positions. Furthermore, it generates sampling weights and sub-pixel keypoint coordinates through improved sampling argmax sampling and reparameterized sampling, enabling the keypoint detection process to have differentiable training capabilities. At the same time, it can suppress unstable feature points in low-texture and repetitive texture regions, improve the positional consistency between sub-pixel keypoint coordinates and feature descriptors, and thus enhance the matching reliability, positioning accuracy, and real-time processing adaptability of SLAM image features in complex UAV flight scenarios. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a lightweight feature extraction method based on UAV visual SLAM proposed in this invention; Figure 2 This is a schematic diagram of trajectory locking sampling grid construction for a lightweight feature extraction method based on UAV visual SLAM proposed in this invention; Figure 3 This is a schematic diagram illustrating the interlocked filtering map generation of a lightweight feature extraction method based on UAV visual SLAM proposed in this invention. Detailed Implementation

[0017] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0018] refer to Figures 1-3 A lightweight feature extraction method based on UAV visual SLAM includes the following steps: Step 1: Acquire continuous RGB image frames from the drone to form a flight image sequence; Step 2: Input the flight image sequence into a lightweight feature coding network to generate feature maps of the current frame and adjacent frames; Step 3: Input the current frame feature map into the key point head branch and the descriptor head branch to generate the key point score map and the dense descriptor map; Step 4: Extract the image patch migration direction, scale change amount and texture main direction based on the feature maps of adjacent frames, generate the trajectory locking direction, and non-uniformly grid the key point fraction map to form the trajectory locking sampling grid; Step 5: Generate a descriptor separability graph based on the similarity of neighboring descriptors in the dense descriptor graph, and align the descriptor separability graph with the keypoint score graph to form an interlocked filtering graph; Step 6: Determine candidate sampling positions within the trajectory-locked sampling grid based on the interlocked screening diagram, and perform improved sampling argmax sampling to generate sampling weights; Step 7: Perform reparameterized sampling on the sampling weights and generate sub-pixel keypoint coordinates by combining the expected sampling position error; Step 8: Sample feature descriptors in the dense descriptor map according to the sub-pixel keypoint coordinates, and output SLAM image features based on the sub-pixel keypoint coordinates and feature descriptors.

[0019] In this embodiment, step one specifically includes: The UAV's visual sensor is controlled to continuously sample images at a preset acquisition frequency. Each frame of color imaging in the flight environment is recorded as an RGB image frame, and the acquisition time, frame number and image size identifier are written into the same frame record to form an original set of image frames. The original image frame set is arranged sequentially according to the acquisition time, so that the RGB image frames in the original image frame set maintain the acquisition time order. Then, the RGB image frame currently being processed is selected as the current RGB image frame according to the frame number, and the RGB image frames that are consecutive to the current RGB image frame are selected as the adjacent RGB image frames, thus establishing the temporal adjacency relationship between the current RGB image frame and the adjacent RGB image frames. Based on the image size identifier, the current RGB image frame and adjacent RGB image frames are adjusted to the same image size. Based on the pixel value range, the current RGB image frame and adjacent RGB image frames are numerically normalized. Based on the channel arrangement rules, the current RGB image frame and adjacent RGB image frames are adjusted to the same channel order to form a regular image frame set. The regularized image frame set is continuously arranged according to the temporal adjacency relationship, so that the current RGB image frame and the adjacent RGB image frames in the regularized image frame set maintain the frame order continuity, and the continuous arrangement result is determined as the flight image sequence.

[0020] In this embodiment, step two specifically includes: Read the current RGB image frame and adjacent RGB image frames in the flight image sequence according to the frame order. Convert the pixel values ​​in the current RGB image frame into the current image tensor according to the channel arrangement rule. Convert the pixel values ​​in adjacent RGB image frames into adjacent image tensors according to the channel arrangement rule. Then feed the current image tensor and adjacent image tensors into a lightweight feature encoding network. The lightweight feature coding network includes a shallow texture coding unit, a scale compression coding unit, and an inter-frame sharing mapping unit. The shallow texture coding unit performs convolution mapping and local response extraction on the current image tensor and the adjacent image tensors, writes edge change information and texture distribution information into the corresponding feature positions, and generates the current shallow feature record and the adjacent shallow feature record. The scale compression coding unit reads the current shallow feature record and the adjacent shallow feature record, performs compression mapping on the channel dimension, performs downsampling processing on the spatial dimension, and aggregates the compression mapping result and the downsampling processing result according to the hierarchical position to generate the current scale feature record and the adjacent scale feature record. The inter-frame shared mapping unit processes the current scale feature records and adjacent scale feature records according to the unified network parameters, so that the current scale feature records and adjacent scale feature records enter the same feature expression space, and then performs feature alignment mapping on the feature positions in the same feature expression space, and generates the current frame feature map and adjacent frame feature maps based on the feature alignment mapping results.

[0021] In this embodiment, step three specifically includes: Read the channel response values ​​one by one from the feature map of the current frame according to the feature position, and read the scale level identifier associated with the channel response value. Write the feature position, channel response value and scale level identifier into the same recording unit. Arrange the recording units according to the feature position to form a branch input record. Then send the branch input record into the key point head branch and the descriptor head branch. The key point head branch reads the channel response value from the branch input record, performs key point response mapping on the channel response value, writes the mapping result that can characterize the salience of the key point into the corresponding feature position, and then adjusts the mapping results under different scale levels to a uniform resolution according to the scale level identifier, and performs response fusion on the mapping results located at the same feature position under the uniform resolution to generate key point response records. The keypoint response values ​​in the keypoint response record are numerically normalized to ensure that the keypoint response values ​​are within a uniform range. Then, the normalized keypoint response values ​​are arranged according to their feature positions. The normalized keypoint response values ​​are written into the feature positions corresponding to the current frame feature map to form a keypoint score map. The descriptor head branch reads the channel response values ​​from the branch input record, performs descriptor vector mapping on the channel response values, writes the vector results that can represent the local image content into the corresponding feature positions, adjusts the vector results under different scale levels to a uniform resolution according to the scale level identifier, and performs vector fusion on the vector results located at the same feature positions under the uniform resolution to generate a descriptor response record. Vector normalization is performed on the descriptor vectors in the descriptor response record to make the descriptor vectors have a uniform vector scale. Then, the normalized descriptor vectors are arranged according to their feature positions. The normalized descriptor vectors are written into the feature positions corresponding to the current frame feature map to form a dense descriptor map.

[0022] In this embodiment, step four specifically includes: The current frame feature map is divided into feature blocks according to feature positions. The adjacent frame feature maps are divided into feature blocks according to feature positions. The current frame feature blocks and the adjacent frame feature blocks are established according to feature positions. The position change from the current frame feature block to the adjacent frame feature block is calculated based on the position change. The position change is converted into a position offset. The image block migration direction is generated based on the direction component of the position offset. Read the response range of the current frame feature block at the scale level, read the response range of the adjacent frame feature block at the scale level, compare the two response ranges by boundary, calculate the scale level expansion based on the part of the response range that increases, calculate the scale level contraction based on the part of the response range that decreases, and combine the scale level expansion and scale level contraction to generate the scale change. Read the channel response value distribution in the current frame feature block, calculate the directional response intensity according to the direction of change of the channel response value in the neighborhood of the feature position, arrange the directional response intensity according to the direction to form the texture directional response, and determine the direction with the largest response intensity in the texture directional response as the main texture direction; The image patch migration direction, scale change amount and texture main direction are mapped to the same directional coordinate system. Directional fusion is performed according to the directional consistency relationship to generate trajectory locking direction. Based on the trajectory locking direction, the local fraction map region corresponding to the feature block of the current frame is located in the key point fraction map. Grid points are placed in the local fractional map area along the trajectory locking direction. The grid point density is increased and the grid point spacing is reduced in the trajectory locking direction. The grid point density is decreased and the grid point spacing is increased in the direction deviating from the trajectory locking direction, forming a non-uniform gridded result. The non-uniform gridded result is determined as the trajectory locking sampling grid.

[0023] In this embodiment, step five specifically includes: Descriptor vectors are read one by one from the dense descriptor graph according to their feature positions. The descriptor vectors read are used as the center descriptor vectors. A neighborhood position range is defined around the feature position of the center descriptor vector. Neighbor descriptor vectors are read according to the feature positions in the neighborhood position range, so that the center descriptor vector and the neighborhood descriptor vectors form a positional correspondence. The similarity between the center descriptor vector and the neighborhood descriptor vector in terms of vector direction and vector magnitude is calculated. The similarity is converted into vector similarity. Then, the vector similarity is arranged according to the feature positions in the neighborhood location range to generate neighborhood descriptor similarity. Based on the similarity statistics of neighboring descriptors, the distinguishability of the center descriptor vector relative to the neighboring descriptor vectors is statistically analyzed. Positions with low similarity concentration and high similarity difference are identified as positions with high distinguishability. The distinguishability is then written into the feature position of the center descriptor vector to generate a descriptor separability map. Perform one-to-one position matching between the feature positions in the descriptor separability map and the feature positions in the keypoint score map. Within the same matching position, read the discrimination level in the descriptor separability map and the keypoint response value in the keypoint score map, and write the discrimination level and keypoint response value into the same matching position. Joint filtering is performed based on the distinguishability and key point response value in the same matching position, so that the distinguishability and key point response value jointly participate in the retention determination of the matching position, generating interlocked filtering values, and arranging the interlocked filtering values ​​according to the feature position to form an interlocked filtering map.

[0024] This invention reads descriptor vectors item by item from a dense descriptor graph and establishes a positional correspondence between the central descriptor vector and neighboring descriptor vectors. This allows each feature location to evaluate its matching potential based on the distribution of neighboring descriptors, avoiding reliance solely on response intensity in the keypoint score graph for candidate location retention. By converting the proximity of vector direction and magnitude into vector similarity and arranging them according to the range of neighboring locations to form a neighborhood descriptor similarity, the invention can structurally represent the descriptor confusion state within a local region, enabling the early identification of feature locations prone to mismatches in low-texture and repetitive texture regions. Furthermore, by relying on neighborhood descriptors... The descriptor similarity statistics method calculates the distinguishability of the center descriptor vector relative to its neighboring descriptor vectors and writes this distinguishability to the feature position of the center descriptor vector, forming a descriptor separability graph. This transforms the descriptor distinguishability from an implicit vector relationship into a graph structure parameter that can participate in the selection process. By performing one-to-one position matching between the descriptor separability graph and the keypoint score graph, and jointly reading the distinguishability and keypoint response values ​​within the same matching position, the saliency of keypoints and the distinguishability of descriptors can jointly determine the retention result of candidate positions. By generating interlocked selection values ​​and arranging them according to feature positions to form an interlocked selection graph, the collaborative selection of keypoint response and descriptor stability can be completed before the improved sampling argmax sampling. This improves the positional consistency and matching reliability between sub-pixel keypoint coordinates and feature descriptors, enhancing the feature extraction stability of UAV visual SLAM in scenarios with sparse textures, repetitive structures, and changing flight perspectives.

[0025] In this embodiment, step six specifically includes: Interlocking screening values ​​are read from the interlocking screening map according to the feature positions. Grid positions, grid spacing, and grid density are read from the trajectory locking sampling grid. The feature positions in the interlocking screening map are converted to the coordinate range of the trajectory locking sampling grid. The converted feature positions are matched with the grid positions to establish a correlation between the interlocking screening values, grid positions, grid spacing, and grid density at the same spatial location, thus establishing a positional mapping relationship between the interlocking screening map and the trajectory locking sampling grid. Based on the location mapping relationship, the interlocking filter value is written into the grid position of the completed location matching to form a grid filter record containing grid position, grid spacing, grid density and interlocking filter value. Then, the grid filter record is retained according to the interlocking filter value, so that the grid position that meets the filtering requirements of both key point response intensity and descriptor separability is retained, and the retained grid position is determined as the candidate sampling position. Candidate sampling positions are arranged according to the trajectory locking direction, so that the candidate sampling positions form a sampling order consistent with the flight trajectory within the trajectory locking sampling grid. Then, the interlocking filter value, grid spacing and grid density associated with the candidate sampling positions are read. The interlocking filter value is used as the candidate position validity parameter, the grid density is used as the trajectory direction aggregation parameter, and the grid spacing is used as the sampling sparsity parameter. The candidate position validity parameter, trajectory direction aggregation parameter and sampling sparsity parameter are combined and calculated to generate a candidate sampling score, and the candidate sampling score is written to the candidate sampling position. The candidate sampling scores are normalized to transform them into a candidate sampling distribution, which represents the relative probability of each candidate sampling position being selected. A Gumbel perturbation term is then introduced into the candidate sampling distribution to generate sampling perturbation while maintaining the differentiable sampling relationship. The improved sampling argmax sampling selects candidate sampling positions based on the candidate sampling distribution after introducing the Gumbel perturbation term, and the selection process of candidate sampling positions can participate in backpropagation. Based on the improved sampling argmax sampling results, the degree of selection of candidate sampling positions during the sampling process is statistically analyzed, and the degree of selection is converted into sampling probability. Then, the sampling probabilities are arranged according to the order of the candidate sampling positions in the trajectory locking direction. The arranged sampling probabilities are determined as sampling weights, so that the sampling weights simultaneously reflect the screening results of the interlocking screening map and the directional structure of the trajectory locking sampling grid.

[0026] This invention transforms the feature positions in the interlocked screening image into the coordinate range of the trajectory-locked sampling grid and performs position matching with the grid positions, thus establishing a spatial correlation between the interlocked screening value, grid position, grid spacing, and grid density. This unifies the keypoint response intensity, descriptor separability, and flight trajectory direction structure within the sampling grid for processing. By writing the interlocked screening value into the grid positions after position matching and performing retention judgment on the grid screening records according to the interlocked screening value, grid positions with insufficient keypoint response and insufficient descriptor distinguishability can be excluded before sampling, giving candidate sampling positions higher localizability and matchability. By arranging candidate sampling positions according to the trajectory-locking direction and using the interlocked screening value, grid density, and grid spacing as candidate position validity parameters, trajectory direction aggregation parameters, and sampling sparsity parameters respectively in the candidate sampling score calculation, the candidate sampling distribution can simultaneously reflect the saliency of image features and the constraints of UAV flight trajectory. By normalizing the candidate sampling score and introducing a Gumbel perturbation term, improved sampling is achieved. argmax sampling can select candidate sampling positions while maintaining differentiable sampling relationships, enabling the sampling process to participate in backpropagation. By converting the degree of selection of candidate sampling positions into sampling probabilities and arranging them according to the trajectory locking direction to form sampling weights, the sampling weights can simultaneously reflect the screening results of the interlocked screening map and the directional structure of the trajectory locking sampling grid, improving the adaptability of the sub-pixel keypoint coordinate generation process to flight trajectory changes, local texture differences, and descriptor separability.

[0027] In this embodiment, step seven specifically includes: The sampling probability is read from the sampling weights item by item according to the candidate sampling positions. Each candidate sampling position, the sampling probability corresponding to each candidate sampling position and the trajectory locking direction are written into the same recording unit. Then, the recording units are sorted according to the order of the candidate sampling positions in the trajectory locking direction to generate the sampling weight record. Based on the sampling weight record, a reparameterized sampling variable is established, the sampling probability is converted into a differentiable sampling coefficient that can participate in gradient propagation, and then the independent noise variable is written into the differentiable sampling coefficient, so that the random changes in the sampling process are carried by the independent noise variable, and the reparameterized sampling coefficient is generated. Candidate sampling positions are read based on the reparameterized sampling coefficients, and the candidate sampling positions are weighted and combined according to the reparameterized sampling coefficients to generate sampling sample positions. Then, the sampling sample positions are arranged in order according to the trajectory locking direction to form a sampling sample position sequence. The sampling position and the candidate sampling position are placed in the same coordinate range for position comparison. The position difference between the sampling position and the candidate sampling position is calculated to generate the sampling position error. Then, the sampling position error is weighted and statistically analyzed according to the sampling weight so that the candidate sampling position with higher sampling probability can make a higher contribution to the statistical results, and the expected sampling position error is generated. The position distribution in the sampled sample position sequence is converged based on the expected sampling position error, so that the sampled sample position sequence is concentrated in the position with lower expected sampling position error. The position convergence result is then mapped to the coordinate range of the key point fraction map to generate sub-pixel key point coordinates.

[0028] This invention writes candidate sampling positions, sampling probabilities, and trajectory locking directions into the same recording unit, and organizes them according to the trajectory locking direction to generate sampling weight records. This ensures that the sampling weights are no longer isolated probability values, but are consistent with the directional structure in the trajectory locking sampling grid. By establishing reparameterized sampling variables based on the sampling weight records, the sampling probabilities are converted into differentiable sampling coefficients, and independent noise variables carry the random changes in sampling. This allows the candidate sampling position selection process to participate in network backpropagation, overcoming the problem of gradient transmission interruption caused by discrete sampling. By weighting and combining candidate sampling positions with reparameterized sampling coefficients and arranging them according to the trajectory locking direction to generate a sampling sample position sequence, the sampling sample positions can be optimized. By continuously distributing samples along the UAV's flight visual trajectory, the motion consistency between keypoint positions and flight image sequences is improved. By comparing the positions of sampled samples and candidate sampled positions within the same coordinate range and weighting the sampling position errors according to sampling weights, the candidate sampled positions with higher sampling probabilities exert a stronger constraint on the expected sampling position error. By performing position convergence processing on the sampled sample position sequence based on the expected sampling position error and mapping it to the coordinate range of the keypoint fractional map to generate sub-pixel keypoint coordinates, the accuracy of keypoint localization can be improved, the position fluctuations caused by random sampling can be reduced, and the feature stability of UAV visual SLAM in continuous frame matching and pose estimation can be enhanced.

[0029] In this embodiment, step eight specifically includes: Read the coordinate position from the sub-pixel keypoint coordinates, decompose the coordinate position into row coordinates and column coordinates, calculate the coordinate mapping ratio based on the coordinate range of the keypoint fraction map and the feature position range of the dense descriptor map, and then perform position conversion on the row coordinates and column coordinates according to the coordinate mapping ratio to convert the sub-pixel keypoint coordinates into sampling coordinates in the dense descriptor map, and determine the sampling coordinates as the descriptor sampling position. Based on the descriptor sampling position, the feature position cell in the dense descriptor graph is determined. The feature position adjacent to the descriptor sampling position is searched along the row and column directions of the feature position cell. The searched feature position is determined as the adjacent feature position. The descriptor vector in the adjacent feature position is read, and the row distance and column distance from the descriptor sampling position to the adjacent feature position are calculated. The interpolation weight is generated based on the row distance and column distance. Based on the interpolation weights, the descriptor vectors in adjacent feature positions are weighted and fused. During the weighted fusion process, the descriptor vector in each adjacent feature position is multiplied by the corresponding interpolation weight, and then the multiplication result is summed so that the descriptor vectors around the descriptor sampling position participate in the fusion according to the distance contribution, generating feature descriptors that match the coordinates of sub-pixel key points. Sub-pixel keypoint coordinates and feature descriptors are written into the same feature record, so that the feature record simultaneously records the keypoint position and descriptor expression. Then, the frame number corresponding to the sub-pixel keypoint coordinates is read from the flight image sequence, the frame number is written into the feature record, and the feature records are arranged in the order of execution according to the frame number to generate a SLAM feature record set. The frame number, subpixel keypoint coordinates, and feature descriptors are read from the SLAM feature record set. The correspondence between image frames and feature records is established according to the frame number. The subpixel keypoint coordinates are determined as keypoint location items, and the feature descriptors are determined as descriptor items. The keypoint location items, descriptor items, and frame number are then combined to generate SLAM image features.

[0030] Example 1: To verify the feasibility of the present invention in practice, it was applied to a visual SLAM scenario of low-altitude inspection by drones in a closed park. The inspection objects included park roads, building facades, pipeline corridors, equipment fences, and open ground areas. The scenario also featured repeated road textures, sparse wall textures, significant changes in viewing angles at corners, dense linear structures in corridor areas, and large changes in image scale when the drone turned. In this scenario, ordinary key point detection methods are prone to problems such as key points being concentrated in repeated texture locations, broken feature point matching between adjacent frames, and mis-preservation of low-texture areas, which in turn affect the matching stability and positioning accuracy of the visual SLAM front end.

[0031] In this scenario, the UAV's visual sensor continuously acquires RGB image frames and organizes them into a flight image sequence. This sequence is then input into a lightweight feature encoding network, which simultaneously generates feature maps for the current frame and adjacent frames. The system determines the image patch migration direction based on the changes in feature block positions in adjacent frame feature maps, the scale change based on the scale-level response range, and the texture main direction based on the channel response distribution in the current frame feature block, thus forming a trajectory locking direction. For turning areas and building edge areas, the trajectory locking direction enables the local fractional map regions in the keypoint fractional map to be non-uniformly gridded along the visual motion trend, forming a trajectory-locked sampling grid. This ensures that candidate sampling positions are more distributed in areas with cross-frame continuity, rather than fixed in positions with high single-frame response but unstable cross-frame performance.

[0032] Simultaneously, the system reads the center descriptor vector and neighborhood descriptor vectors from the dense descriptor map, calculates the neighborhood descriptor similarity, and forms a descriptor separability map. This map is then aligned with the keypoint score map to form an interlocked filtering map. For areas prone to repetitive textures, such as walls, floors, and fences, the interlocked filtering map incorporates both keypoint response values ​​and descriptor distinguishability into the candidate sampling location determination, preventing locations with high score responses but severe descriptor confusion from entering the improved sampling argmax sampling process. Subsequently, the system determines candidate sampling locations within the trajectory-locked sampling grid, generates sampling weights through improved sampling argmax sampling, generates sub-pixel keypoint coordinates through reparameterized sampling and sampling location error expectation, samples feature descriptors in the dense descriptor map, and finally outputs SLAM image features.

[0033] During the test, the same flight altitude range, the same inspection path length, the same image resolution, and the same embedded computing platform were used. The ordinary fixed-window keypoint sampling method was used as a comparison scheme, and the present invention was used as the test scheme. Continuous image frames were collected during daytime and low-light inspection periods, covering straight road sections, corner areas, open areas, and texture repetition areas. Keypoint repetition rate, effective matching rate, average single-frame processing time, trajectory drift, and low texture error retention rate were statistically analyzed. The test results are shown in Table 1. Table 1 Comparison of Lightweight SLAM Feature Extraction Performance

[0034] As shown in Table 1, under the same image acquisition conditions and computing platform, the average single-frame processing time of this invention decreased from 31.8ms to 24.6ms, indicating that the lightweight feature encoding network and the pre-sampling screening mechanism reduced the participation of invalid candidate positions in the calculation; the key point repetition rate of adjacent frames increased from 62.4% to 81.7%, indicating that the trajectory-locked sampling grid can make the key point sampling position more consistent with the image patch migration trend during the continuous flight of the UAV; the feature matching efficiency increased from 58.9% to 76.5%, and the false retention rate of low texture areas and the false matching rate of repeated texture areas both decreased significantly, indicating that the interlocked screening map formed by the descriptor separability map and the key point fraction map can eliminate candidate positions that are not suitable for stable matching in advance; the single-segment inspection trajectory drift decreased from 1.42m to 0.63m, indicating that the sub-pixel key point coordinates generated by the improved sampling argmax sampling have better position consistency and matching reliability with the feature descriptor, which can improve the positioning stability of UAV visual SLAM in complex inspection scenarios.

[0035] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A lightweight feature extraction method based on UAV vision SLAM, characterized in that, Includes the following steps: Step 1: Acquire continuous RGB image frames from the drone to form a flight image sequence; Step 2: Input the flight image sequence into a lightweight feature coding network to generate feature maps of the current frame and adjacent frames; Step 3: Input the current frame feature map into the key point head branch and the descriptor head branch to generate the key point score map and the dense descriptor map; Step 4: Extract the image patch migration direction, scale change amount and texture main direction based on the feature maps of adjacent frames, generate the trajectory locking direction, and non-uniformly grid the key point fraction map to form the trajectory locking sampling grid; Step 5: Generate a descriptor separability graph based on the similarity of neighboring descriptors in the dense descriptor graph, and align the descriptor separability graph with the keypoint score graph to form an interlocked filtering graph; Step 6: Determine candidate sampling positions within the trajectory-locked sampling grid based on the interlocked screening diagram, and perform improved sampling argmax sampling to generate sampling weights; Step 7: Perform reparameterized sampling on the sampling weights and generate sub-pixel keypoint coordinates by combining the expected sampling position error; Step 8: Sample feature descriptors in the dense descriptor map according to the sub-pixel keypoint coordinates, and output SLAM image features based on the sub-pixel keypoint coordinates and feature descriptors. 2.The lightweight feature extraction method based on UAV vision SLAM of claim 1, wherein, Step one specifically involves: The UAV's visual sensor continuously acquires RGB image frames of the flight environment at a preset acquisition frequency, and configures the acquisition time, frame number and image size identifier for each RGB image frame to form an original image frame set. The original image frame set is sorted by time according to the acquisition time, the current RGB image frame and adjacent RGB image frames are determined according to the frame number, and the temporal adjacency relationship between the current RGB image frame and adjacent RGB image frames is established. Based on the image size identifier, the current RGB image frame and adjacent RGB image frames are size-normalized, pixel normalization is performed based on the pixel value range, and channel unification is performed based on the channel arrangement rules, forming a normalized image frame set; The regular image frame set is continuously arranged according to the temporal adjacency relationship, and the result of the continuous arrangement is determined as a flight image sequence.

3. The lightweight feature extraction method based on UAV visual SLAM according to claim 1, characterized in that, Step two specifically involves: Read the current RGB image frame and adjacent RGB image frames from the flight image sequence, map the current RGB image frame to the current image tensor, map adjacent RGB image frames to adjacent image tensors, and input the current image tensor and adjacent image tensors into a lightweight feature encoding network; The lightweight feature coding network includes a shallow texture coding unit, a scale compression coding unit, and an inter-frame sharing mapping unit. The shallow texture coding unit performs edge texture mapping on the current image tensor and the adjacent image tensors to generate the current shallow feature record and the adjacent shallow feature record. The scale compression coding unit performs channel compression, spatial downsampling, and hierarchical aggregation on the current shallow feature record and adjacent shallow feature records to generate the current scale feature record and adjacent scale feature records; The inter-frame shared mapping unit performs feature alignment mapping on the current scale feature record and the adjacent scale feature record according to the unified network parameters, and generates the current frame feature map and the adjacent frame feature map based on the feature alignment mapping result.

4. The lightweight feature extraction method based on UAV visual SLAM according to claim 1, characterized in that, Step three specifically involves: Extract feature location, channel response value and scale level identifier from the feature map of the current frame, establish branch input record according to feature location, and input the branch input record into the key point head branch and descriptor head branch; The key point branch performs key point response mapping based on the channel response values ​​in the branch input record, and performs resolution unification and response fusion on the key point response mapping results based on the scale level identifier to generate key point response records; Perform numerical normalization and positional arrangement on the keypoint response records, and write the normalized keypoint response values ​​into the feature positions to form a keypoint score map. The descriptor head branch performs descriptor vector mapping based on the channel response values ​​in the branch input record, and performs resolution unification and vector fusion on the descriptor vector mapping result based on the scale level identifier to generate a descriptor response record; The descriptor response record is normalized and its position is arranged. The normalized descriptor vector is written to the feature position to form a dense descriptor graph.

5. A lightweight feature extraction method based on UAV visual SLAM according to claim 1, characterized in that, Step four specifically involves: The feature map of the current frame and the feature map of the adjacent frames are divided into feature blocks according to the feature position. The position offset is calculated based on the position change between the feature block of the current frame and the feature block of the adjacent frames, and the image block migration direction is generated based on the position offset. Based on the changes in the response range of the current frame feature block and the adjacent frame feature blocks at the scale level, the scale level expansion and scale level contraction are calculated, and the scale change is generated based on the scale level expansion and scale level contraction. The texture direction response is calculated based on the channel response value distribution in the current frame feature block, and the direction with the largest response intensity in the texture direction response is determined as the main texture direction; The image patch migration direction, scale change amount, and texture main direction are fused to generate a trajectory locking direction, and the local fraction map region in the key point fraction map is determined based on the trajectory locking direction. Grid density and grid spacing are adjusted in the local fractional map region along the trajectory locking direction to form a non-uniform grid result, and the non-uniform grid result is determined as the trajectory locking sampling grid.

6. A lightweight feature extraction method based on UAV visual SLAM according to claim 1, characterized in that, Step five specifically involves: Read the center descriptor vector from the dense descriptor graph according to the feature position, define the neighborhood position range around the feature position where the center descriptor vector is located, and extract the neighborhood descriptor vector in the neighborhood position range; Calculate the vector similarity between the center descriptor vector and the neighborhood descriptor vector, arrange the vector similarity according to the feature positions in the neighborhood location range, and generate the neighborhood descriptor similarity; The degree of distinction between the center descriptor vector and the neighbor descriptor vectors is calculated based on the similarity of the neighbor descriptors. The degree of distinction is written into the feature position of the center descriptor vector to generate a descriptor separability map. Perform position matching between the feature locations in the descriptor separability graph and the feature locations in the keypoint score graph, and write the discriminative power in the descriptor separability graph and the keypoint response value in the keypoint score graph to the same matching location. Joint filtering is performed based on the degree of differentiation and key point response values ​​at the same matching position to generate interlocked filtering values. These interlocked filtering values ​​are then arranged according to feature positions to form an interlocked filtering map.

7. A lightweight feature extraction method based on UAV visual SLAM according to claim 1, characterized in that, Step six specifically involves: Read the interlocking screening value from the interlocking screening map, read the grid point position, grid point spacing and grid point density from the trajectory locking sampling grid, and establish the position mapping relationship between the interlocking screening map and the trajectory locking sampling grid according to the feature position; Based on the position mapping relationship, the interlocking filter value is written into the grid position in the trajectory locking sampling grid to generate a grid point filter record. The grid point filter record is then judged for retention based on the interlocking filter value to determine the candidate sampling position. Candidate sampling positions are arranged according to the trajectory locking direction. Candidate sampling scores are generated based on the interlocking filter value, grid spacing and grid density corresponding to the candidate sampling positions, and the candidate sampling scores are written into the candidate sampling positions. Normalize the candidate sampling scores to generate a candidate sampling distribution, and introduce a Gumbel perturbation term into the candidate sampling distribution to perform improved sampling argmax sampling. The sampling probabilities corresponding to the candidate sampling positions are calculated based on the sampling results of the improved sampling argmax, and the sampling probabilities are arranged according to the candidate sampling positions to generate sampling weights.

8. A lightweight feature extraction method based on UAV visual SLAM according to claim 1, characterized in that, Step seven specifically involves: Read the sampling probability corresponding to the candidate sampling position from the sampling weight, associate and organize the candidate sampling position, sampling probability and trajectory locking direction, and generate a sampling weight record; Based on the sampling weight record, a reparameterized sampling variable is constructed, the sampling probability is converted into a differentiable sampling coefficient, and the independent noise variable is written into the differentiable sampling coefficient to generate the reparameterized sampling coefficient; Based on the reparameterized sampling coefficients, the candidate sampling positions are weighted and sampled to generate sampling sample positions. The sampling sample positions are then arranged according to the trajectory locking direction to form a sampling sample position sequence. The sampling position difference is calculated between the sampled position and the candidate sampling position to generate the sampling position error. The sampling position error is then weighted and statistically analyzed according to the sampling weight to generate the expected sampling position error. Based on the expected sampling position error, position convergence processing is performed on the sampling sample position sequence, and the position convergence processing result is mapped to the coordinate range of the key point fraction map to generate sub-pixel key point coordinates.

9. A lightweight feature extraction method based on UAV visual SLAM according to claim 1, characterized in that, Step eight specifically involves: Read the coordinate positions from the sub-pixel keypoint coordinates, map the coordinate positions to the feature position range of the dense descriptor map, and generate the descriptor sampling positions; Based on the descriptor sampling position, the adjacent feature positions are located in the dense descriptor graph. The descriptor vectors in the adjacent feature positions are read, and interpolation weights are generated based on the positional distance between the descriptor sampling position and the adjacent feature position. Weighted fusion of the descriptor vector is performed based on the interpolation weights to generate a feature descriptor that matches the coordinates of sub-pixel key points; Sub-pixel keypoint coordinates and feature descriptors are written into the same feature record. The feature records are arranged according to the frame number in the flight image sequence to generate a SLAM feature record set. Subpixel keypoint coordinates, feature descriptors, and frame numbers are extracted from the SLAM feature record set, and then organized according to the frame numbers to generate SLAM image features.

Images