Vgg-based external key point matching and tracking trajectory calculation method and device

Abstract

The application discloses a kind of external key point matching and tracking trajectory calculation method and device based on VGGT.The method comprises: loading configuration file and initializing model;Load external key point coordinates and carry out standardization and quantity alignment;First, input image is converted into tensor format, then image features are extracted, and finally L2 normalization is carried out;Depth feature vector of external key point is extracted from feature map;Calculate cosine similarity to evaluate matching quality;The average amplitude and standard deviation of feature vector are calculated to evaluate feature quality, and the matching threshold is dynamically adjusted;The coordinate difference of matching point is calculated to eliminate the matching that does not meet geometric constraint, and tracking trajectory is generated;Tracking quality analysis and visualization are carried out.The matching success rate of the application is greatly improved, the false matching rate is obviously reduced, the time consumption of processing image is obviously shortened, the efficiency is also greatly improved, the tracking stability is obviously improved under dynamic interference, and automatic high-precision inspection without manual intervention is realized.

Description

Technical Field

[0001] This invention relates to a calculation method in the field of power line inspection technology, and more particularly to a method for calculating external key point matching and tracking trajectory based on VGGT, and also to a device for calculating external key point matching and tracking trajectory based on VGGT. Background Technology

[0002] Traditional keypoint tracking methods face significant challenges in complex scenarios. First, when dealing with symmetrical structures such as power equipment, traditional feature matching algorithms like SIFT and ORB struggle to distinguish similar regions, leading to matching confusion and a success rate dropping below 68%. Second, these methods are extremely sensitive to dynamic disturbances such as swaying conductors and birds, causing keypoint loss and mismatches, with a mismatch rate as high as 15%. Furthermore, traditional processes require multiple geometric optimization steps, resulting in high computational costs and long processing times, relying heavily on post-processing optimization. For example, processing 10 frames of images takes more than 10 seconds, which is insufficient for the real-time requirements of power line inspections. Finally, they lack end-to-end capabilities; keypoint matching and 3D trajectory calculation are separated, causing errors to accumulate and propagate across multiple steps, ultimately affecting trajectory accuracy.

[0003] In the field of power equipment inspection, these problems are particularly prominent in power inspection scenarios. Power equipment is usually located in remote areas, the inspection environment is complex, and the requirements for safety and efficiency are extremely high. Traditional methods are not only inefficient but also require a lot of manual intervention, making it difficult to achieve automated, high-precision power equipment health monitoring and defect detection. Moreover, existing key point tracking methods also suffer from problems such as failure in matching key points of symmetrical structures, insufficient tracking stability under dynamic interference, lack of end-to-end processes, and low computational efficiency. Summary of the Invention

[0004] To address the technical problems of low tracking accuracy, low efficiency, and insufficient stability in existing key point tracking methods, this invention provides a method and apparatus for external key point matching and tracking trajectory calculation based on VGGT.

[0005] This invention is implemented using the following technical solution: a method for external key point matching and tracking trajectory calculation based on VGGT, which includes the following steps:

[0006] S1: Load the configuration file and initialize the VGGT model;

[0007] S2: Load the coordinates of external key points and perform standardization and quantity alignment to obtain normalized coordinates;

[0008] S3: First, adjust the color channel order of the input image, normalize it, and convert it to tensor format. Then, extract image features through the backbone network of the VGGT framework to generate a feature map including depth information, camera parameters, and tracking features. Finally, perform L2 normalization on the feature map.

[0009] S4: Based on the normalized coordinates, use bilinear interpolation to extract the depth feature vectors of external key points from the normalized feature map; calculate the cosine similarity of the feature vectors of key points in two frames to evaluate the matching quality; calculate the average magnitude and standard deviation of the feature vectors to evaluate the feature quality, and dynamically adjust the matching threshold according to the feature quality. The higher the feature quality, the higher the matching threshold, and the lower the feature quality, the lower the matching threshold; combine camera parameters and depth map to project 2D key points into 3D space, and calculate the coordinate difference of matching points to eliminate matches that do not meet geometric constraints, generating the tracking trajectory of key points;

[0010] S5: Perform tracking quality analysis and generate visualizations.

[0011] This invention effectively solves the technical problems of existing key point tracking methods, such as the failure of matching in symmetrical structures, low tracking accuracy due to sensitivity to dynamic interference, low computational efficiency caused by multi-step optimization, and insufficient stability caused by the separate process, by introducing the VGGT framework and deep integration with external key points, combined with an adaptive threshold matching strategy and an end-to-end trajectory calculation process. It achieves significant technical effects, such as a significant increase in matching success rate and a significant decrease in mismatch rate in symmetrical structure scenarios of power equipment, a significant reduction in processing time for 10 frames of images and a significant increase in efficiency, and a significant improvement in tracking stability under dynamic interference. At the same time, it realizes automated high-precision inspection without human intervention.

[0012] As a further improvement to the above scheme, the method for extracting the deep feature vector includes the following steps:

[0013] The keypoint coordinates are mapped to the feature map scale, and the mapping formula is as follows:

[0014]

[0015] In the formula, x and y represent the coordinates of the key points, and W and H represent the width and height of the feature map, respectively;

[0016] The depth feature vector is obtained by performing sub-pixel level feature sampling using the F.grid_sample function.

[0017] As a further improvement to the above scheme, the formula for calculating the cosine similarity is:

[0018] similarity = cos(feat1, feat2)

[0019] In the formula, similarity is the cosine similarity, and feat1 and feat2 are the feature vectors of the same key point in the two frames, respectively.

[0020] As a further improvement to the above scheme, the adjustment formula for the matching threshold is:

[0021] adaptive_threshold=base_threshold * (1+mean_magnitude * 0.1)

[0022] In the formula, adaptive_threshold is the matching threshold, base_threshold is the preset base threshold, and mean_magnitude is the average magnitude of the key point feature vector.

[0023] As a further improvement to the above scheme, the formula for calculating coordinates when projected into 3D space is:

[0024]

[0025] In the formula, K is the camera intrinsic parameter matrix, u and v are the keypoint coordinates, and d is the depth value.

[0026] As a further improvement to the above scheme, the standardization includes converting the key point coordinates into a normalized coordinate format; the quantity alignment includes aligning the quantities through interpolation or deletion strategies when the number of key points in two frames is inconsistent.

[0027] As a further improvement to the above scheme, the tracking quality analysis includes calculating at least one of the following quality indicators:

[0028] Feature statistics: The effectiveness of feature representation is evaluated by calculating the mean magnitude, standard deviation, and diversity of the feature vectors;

[0029] Spatial distribution range: Analyze the distribution range and center location of key points in the image;

[0030] Uniqueness score: The uniqueness of key points in a local region is evaluated by calculating the similarity of local features.

[0031] As a further improvement to the above scheme, the generated visualization results include at least one of the following charts:

[0032] Histogram of characteristic amplitude distribution;

[0033] A scatter plot showing the spatial distribution of key point uniqueness scores is displayed using color coding.

[0034] Key point quality score ranking bar chart;

[0035] A summary table of statistical information including the number of key points, feature dimensions, and matching rate.

[0036] As a further improvement to the above scheme, in step S1, system parameters are managed through a YAML configuration file. The system parameters include the camera model, feature extraction layer, and adaptive threshold base value. When selecting a device, the optimal computing device is automatically detected and selected, and the VGGT model is loaded only when needed.

[0037] The present invention also provides a VGGT-based external keypoint matching and tracking trajectory calculation device, which applies any of the above-described VGGT-based external keypoint matching and tracking trajectory calculation methods. The device includes:

[0038] The system initialization and configuration module is used to load configuration files and initialize the VGGT model.

[0039] The keypoint loading and coordinate transformation module is used to load the coordinates of external keypoints and perform standardization and quantity alignment to obtain normalized coordinates.

[0040] The feature map calculation and caching module is used to first adjust the color channel order of the input image, normalize it and convert it into tensor format, then extract image features through the backbone network of the VGGT framework to generate a feature map including depth information, camera parameters and tracking features, and finally perform L2 normalization on the feature map.

[0041] The depth feature extraction and matching module is used to extract depth feature vectors of external key points from the normalized feature map based on the normalized coordinates; the depth feature extraction and matching module is also used to calculate the cosine similarity of the feature vectors of key points in two frames to evaluate the matching quality; the depth feature extraction and matching module is also used to calculate the average magnitude and standard deviation of the feature vectors to evaluate the feature quality, and dynamically adjust the matching threshold based on the feature quality; the depth feature extraction and matching module is also used to combine camera parameters and depth map to project 2D key points into 3D space, and calculate the coordinate difference of matching points to eliminate matches that do not meet geometric constraints, and generate the tracking trajectory of the key points;

[0042] The tracking quality analysis and visualization module is used to perform tracking quality analysis and generate visualizations.

[0043] Compared with existing keypoint tracking technologies, the external keypoint matching and tracking trajectory calculation method and apparatus based on VGGT of the present invention have the following advantages:

[0044] 1. This VGGT-based external keypoint matching and tracking trajectory calculation method effectively solves the technical problems of existing keypoint tracking methods, such as symmetrical structure matching failure, low tracking accuracy due to dynamic interference sensitivity, low computational efficiency due to multi-step optimization, and insufficient stability caused by the separate process, by introducing the VGGT framework and deep fusion with external keypoints, combined with an adaptive threshold matching strategy and an end-to-end trajectory calculation process. It achieves significant technical effects, such as greatly improved matching success rate and significantly reduced mismatch rate in symmetrical structure scenarios of power equipment, significantly shortened processing time for 10 frames of images and greatly improved efficiency, and significantly improved tracking stability under dynamic interference. At the same time, it realizes automated high-precision inspection without manual intervention.

[0045] 2. The VGGT-based external key point matching and tracking trajectory calculation method improves the matching success rate from 68% to 92% in the symmetrical structure scenario of transmission towers, and reduces the false matching rate from 15% to below 5%. Moreover, through the adaptive threshold strategy, the matching accuracy is improved by more than 30%, reducing the need for manual intervention.

[0046] 3. The VGGT-based external key point matching and tracking trajectory calculation method significantly enhances its real-time performance. Specifically, the end-to-end process for 10 frames of images takes only 0.2 seconds, 50 times faster than the traditional BA method. Furthermore, in UAV power line inspection, the efficiency of a single inspection is increased by 1.9 times, and the number of power towers inspected per day is doubled.

[0047] 4. The robustness of this VGGT-based external keypoint matching and tracking trajectory calculation method is improved. In dynamic scenarios with conductor swaying or bird interference, tracking stability is improved by 40%, and the keypoint loss rate is reduced to less than 10%. Moreover, the global self-attention mechanism effectively suppresses interference from symmetrical structures, solving the matching failure problem of traditional methods in symmetrical structure scenarios such as transmission towers.

[0048] 5. This VGGT-based external keypoint matching and tracking trajectory calculation method optimizes computational efficiency and has wide applicability. Specifically, the feature caching mechanism reduces redundant calculations and improves processing speed. The memory management strategy optimizes GPU resource utilization and avoids memory overflow issues. Furthermore, this method supports keypoint matching and tracking for various power equipment (transmission towers, insulators, transformers, etc.) and can be extended to keypoint tracking tasks in other complex scenarios (such as building inspection and autonomous driving).

[0049] 6. The external key point matching and tracking trajectory calculation device based on VGGT has the same beneficial effects as the above-mentioned method, and will not be described in detail here. Attached Figure Description

[0050] Figure 1This is a flowchart of the external key point matching and tracking trajectory calculation method based on VGGT according to Embodiment 1 of the present invention.

[0051] Figure 2 for Figure 1 The overall framework diagram of the VGGT-based external key point matching and tracking trajectory calculation method is shown in the figure. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0053] Example 1

[0054] Please see Figure 1 as well as Figure 2 This embodiment provides a method for external keypoint matching and tracking trajectory calculation based on VGGT. This method is based on the Visual Geometry Guided Transformer (VGGT) framework and uses a feedforward neural network to directly infer camera intrinsic and extrinsic parameters, depth maps, point maps, and tracking features from the input image, achieving end-to-end external keypoint matching and 3D trajectory calculation. The main steps of this method are as follows (S1-S6).

[0055] S1. System Initialization and Configuration: Load the configuration file and initialize the VGGT model. In this embodiment, system parameters are managed through a YAML configuration file. These parameters include the camera model, feature extraction layer, and adaptive threshold base value (e.g., base_threshold=0.1). ） The system automatically detects and selects the optimal computing device (GPU / CPU) during device selection to ensure efficient computation. The VGGT model is loaded only when needed, avoiding unnecessary resource consumption. For example, the model is initialized only when processing an image for the first time, reducing memory usage.

[0056] S2. External Keypoint Loading and Coordinate Transformation: Load the coordinates of external keypoints and perform standardization and quantity alignment to obtain normalized coordinates. In this embodiment, the coordinates of external keypoints are loaded from a predefined file (such as JSON format). These keypoints are typically determined manually or by previous detection algorithms. Standardization involves converting the keypoint coordinates into a unified normalized coordinate format (such as [u, v] format) to ensure independence from image resolution. Quantity alignment involves aligning the number of keypoints between two frames using interpolation or deletion strategies when the number of keypoints is inconsistent, ensuring consistency in subsequent matching.

[0057] S3. Feature Map Calculation and Caching: First, the color channel order of the input image is adjusted, normalized, and converted to tensor format. Then, image features are extracted through the backbone network of the VGGT framework to generate a multi-dimensional feature map including depth information, camera parameters, and tracking features. Finally, L2 normalization is performed on the feature map to improve the stability and comparability of feature representation. In this embodiment, during image preprocessing, the input image is converted from BGR to RGB, normalized to the [0,1] range, and converted to tensor format. Furthermore, the feature map is cached to avoid redundant calculations, significantly improving processing efficiency.

[0058] S4. Deep Feature Extraction and Matching: (1) Key Point Feature Extraction: Based on the normalized coordinates, the deep feature vectors of external key points are extracted from the normalized feature map using bilinear interpolation. In this embodiment, the method for extracting the deep feature vectors includes the following steps:

[0059] (1.1) Map the keypoint coordinates to the feature map scale, and the mapping formula is:

[0060]

[0061] In the formula, x and y represent the coordinates of the key points, and W and H represent the width and height of the feature map, respectively.

[0062] (1.2) Sub-pixel level feature sampling is performed using the F.grid_sample function to improve accuracy and thus obtain depth feature vectors.

[0063] (2) Feature Similarity Calculation: The cosine similarity of the feature vectors of key points in two frames is calculated to evaluate the matching quality. In this embodiment, the formula for calculating the cosine similarity is:

[0064] similarity = cos(feat1, feat2)

[0065] In the formula, similarity is the cosine similarity, and feat1 and feat2 are the feature vectors of the same key point in the two frames, respectively.

[0066] (3) Adaptive threshold strategy: Calculate the average magnitude and standard deviation of the feature vector to evaluate feature quality, and dynamically adjust the matching threshold based on feature quality. In this embodiment, the adjustment formula for the matching threshold is:

[0067] adaptive_threshold=base_threshold * (1+mean_magnitude * 0.1)

[0068] In the formula, adaptive_threshold is the matching threshold, base_threshold is the preset base threshold (e.g., 0.1), and mean_magnitude is the average magnitude of the key point feature vector.

[0069] Feature quality assessment involves calculating the average magnitude and standard deviation of the feature vectors to evaluate the strength and diversity of feature representation. Dynamic adjustment is applied: higher feature quality results in a higher threshold, reducing false matches. Conversely, lower feature quality leads to a lower threshold, preventing missed matches.

[0070] (4) Spatial Consistency Verification: Combining camera parameters (intrinsic and extrinsic parameters) and depth map, 2D keypoints are projected into 3D space, and the coordinate differences of matching points are calculated to eliminate matches that do not meet geometric constraints, generating the tracking trajectory of the keypoints. In this embodiment, the coordinate calculation formula when projecting into 3D space is:

[0071]

[0072] In the formula, K is the camera intrinsic parameter matrix, u and v are the keypoint coordinates, and d is the depth value.

[0073] Difference threshold: Set the maximum allowable value for 3D coordinate differences (e.g., 5mm). Matches exceeding this range are considered invalid.

[0074] S5: Perform tracking quality analysis and generate visualizations. In this embodiment, tracking quality analysis includes calculating at least one of the following quality metrics.

[0075] (1) Feature statistics: The effectiveness of feature representation is evaluated by calculating the average magnitude, standard deviation and diversity of the feature vector.

[0076] (2) Spatial distribution range: Analyze the distribution range and central position of key points in the image to ensure coverage of key areas.

[0077] (3) Uniqueness score: The uniqueness of key points in local regions is evaluated by calculating the local feature similarity to avoid interference from symmetrical structures.

[0078] When generating visualizations, enhanced charts are created to visually display matching results and quality analysis. The generated visualizations include at least one of the following chart types:

[0079] (1) Feature amplitude distribution histogram: The histogram shows the distribution of feature vector amplitudes and identifies low-quality features.

[0080] (2) Spatial distribution scatter plot: The uniqueness score of key points is displayed by color coding, with red indicating high uniqueness and blue indicating low uniqueness.

[0081] (3) Key point quality score ranking bar chart: The bar chart shows the ranking of key point quality scores, and identifies key points that need to be focused on.

[0082] (4) A summary table of statistical information including the number of key points, feature dimensions, and matching rate. This table format displays statistical information such as the number of key points, feature dimensions, and matching rate, making it easy to quickly evaluate the overall effect.

[0083] S6: Result Saving and Output. During result processing, matching results, quality metrics, and visualization charts are organized into structured data. When saving the results, they are saved as JSON files, containing keypoint matching information, quality metrics, and visualization paths. During performance monitoring, performance metrics such as feature extraction time and matching time are recorded to evaluate system efficiency.

[0084] Compared to traditional methods that rely on feature point detection algorithms such as SIFT and ORB and cannot directly utilize externally labeled keypoints, this embodiment extracts deep feature vectors of external keypoints from the VGGT feature map using bilinear interpolation. It then combines this with cosine similarity to calculate matching quality, achieving seamless integration of external keypoints with the VGGT framework. Unlike traditional methods that use a fixed threshold (e.g., 0.7) for feature matching, which cannot adapt to different scenes and varying keypoint quality, this embodiment dynamically adjusts the matching threshold based on the average amplitude of the feature vectors, avoiding missed matches and improving overall matching accuracy. Furthermore, unlike traditional methods that rely on local feature matching and struggle with symmetrical structures and dynamic interference, this embodiment utilizes VGGT's global self-attention mechanism to integrate multi-view geometric relationships and combines deep features to enhance the geometric constraints of matching, effectively solving the problems of matching failures with symmetrical structures and sensitivity to dynamic interference. Compared to traditional methods that require multiple geometric optimization steps (such as BA), which are computationally complex and time-consuming, this embodiment directly outputs camera parameters, depth maps, and tracking features through the feedforward neural network of VGGT, achieving end-to-end keypoint matching and 3D trajectory calculation. It only takes 0.2 seconds to process 10 frames of images, which is 50 times faster than traditional methods.

[0085] The method in this embodiment can solve the problem of key point matching failure in symmetrical structures. Specifically, it solves the matching confusion problem of traditional feature matching algorithms in the scenario of symmetrical power equipment structures (such as transmission towers and insulators), and improves the accuracy and reliability of key point matching. If this problem is not solved, it will lead to the failure of key point tracking during power equipment inspection, making it impossible to accurately monitor equipment displacement, deformation and other health indicators, and increasing safety hazards.

[0086] The method in this embodiment can solve the problem of insufficient tracking stability under dynamic interference. Specifically, it addresses the issues of key point mismatch and loss caused by interference from moving objects such as swaying wires and birds, thereby improving the robustness of tracking. Without solving this problem, key points will be frequently interrupted during inspections, making continuous monitoring of equipment status impossible and affecting inspection efficiency and the reliability of results.

[0087] The method in this embodiment can solve the problems of missing end-to-end processes and low computational efficiency. Specifically, it addresses the high computational complexity and long processing time caused by traditional methods relying on multi-step geometric optimization (such as Business Analog), achieving efficient and accurate external key point matching and 3D trajectory calculation. Without solving this problem, the power inspection system will be unable to process large amounts of image data in real time, failing to meet the efficiency requirements of automated inspection and increasing labor costs.

[0088] In summary, compared with existing keypoint tracking methods, the VGGT-based external keypoint matching and tracking trajectory calculation method in this embodiment has the following advantages:

[0089] 1. This VGGT-based external keypoint matching and tracking trajectory calculation method effectively solves the technical problems of existing keypoint tracking methods, such as symmetrical structure matching failure, low tracking accuracy due to dynamic interference sensitivity, low computational efficiency due to multi-step optimization, and insufficient stability caused by the separate process, by introducing the VGGT framework and deep fusion with external keypoints, combined with an adaptive threshold matching strategy and an end-to-end trajectory calculation process. It achieves significant technical effects, such as greatly improved matching success rate and significantly reduced mismatch rate in symmetrical structure scenarios of power equipment, significantly shortened processing time for 10 frames of images and greatly improved efficiency, and significantly improved tracking stability under dynamic interference. At the same time, it realizes automated high-precision inspection without manual intervention.

[0090] 2. The VGGT-based external key point matching and tracking trajectory calculation method improves the matching success rate from 68% to 92% in the symmetrical structure scenario of transmission towers, and reduces the false matching rate from 15% to below 5%. Moreover, through the adaptive threshold strategy, the matching accuracy is improved by more than 30%, reducing the need for manual intervention.

[0091] 3. The VGGT-based external key point matching and tracking trajectory calculation method significantly enhances its real-time performance. Specifically, the end-to-end process for 10 frames of images takes only 0.2 seconds, 50 times faster than the traditional BA method. Furthermore, in UAV power line inspection, the efficiency of a single inspection is increased by 1.9 times, and the number of power towers inspected per day is doubled.

[0092] 4. The robustness of this VGGT-based external keypoint matching and tracking trajectory calculation method is improved. In dynamic scenarios with conductor swaying or bird interference, tracking stability is improved by 40%, and the keypoint loss rate is reduced to less than 10%. Moreover, the global self-attention mechanism effectively suppresses interference from symmetrical structures, solving the matching failure problem of traditional methods in symmetrical structure scenarios such as transmission towers.

[0093] 5. This VGGT-based external keypoint matching and tracking trajectory calculation method optimizes computational efficiency and has wide applicability. Specifically, the feature caching mechanism reduces redundant calculations and improves processing speed. The memory management strategy optimizes GPU resource utilization and avoids memory overflow issues. Furthermore, this method supports keypoint matching and tracking for various power equipment (transmission towers, insulators, transformers, etc.) and can be extended to keypoint tracking tasks in other complex scenarios (such as building inspection and autonomous driving).

[0094] Example 2

[0095] This embodiment provides a VGGT-based external keypoint matching and tracking trajectory calculation device, which applies the VGGT-based external keypoint matching and tracking trajectory calculation method in Embodiment 1. The device includes a system initialization and configuration module, a keypoint loading and coordinate transformation module, a feature map calculation and caching module, a depth feature extraction and matching module, and a tracking quality analysis and visualization module.

[0096] The system initialization and configuration module loads the configuration file and initializes the VGGT model. The keypoint loading and coordinate transformation module loads the coordinates of external keypoints and performs normalization and numerical alignment to obtain normalized coordinates. The feature map calculation and caching module first adjusts the color channel order of the input image, normalizes it, and converts it to tensor format. Then, it extracts image features through the backbone network of the VGGT framework to generate feature maps including depth information, camera parameters, and tracking features. Finally, it performs L2 normalization on the feature maps.

[0097] The depth feature extraction and matching module extracts depth feature vectors of external keypoints from the normalized feature map based on normalized coordinates. It also calculates the cosine similarity of keypoint feature vectors between two frames to evaluate matching quality. Furthermore, it calculates the mean magnitude and standard deviation of the feature vectors to assess feature quality and dynamically adjusts the matching threshold based on this quality. Finally, it combines camera parameters and the depth map to project 2D keypoints into 3D space, calculates the coordinate differences of matching points to eliminate matches that do not meet geometric constraints, and generates tracking trajectories for the keypoints. The tracking quality analysis and visualization module performs tracking quality analysis and generates visualizations.

[0098] Example 3

[0099] This embodiment provides a computer terminal, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the VGGT-based external keypoint matching and tracking trajectory calculation method of Embodiment 1.

[0100] The method in Example 1 can be applied in software form, such as by designing it as a standalone program and installing it on a computer terminal, which can be a computer, smartphone, control system, or other IoT device. Alternatively, the method in Example 1 can be designed as an embedded program and installed on a computer terminal, such as on a microcontroller.

[0101] Example 4

[0102] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it implements the steps of the VGGT-based external keypoint matching and tracking trajectory calculation method of Embodiment 1.

[0103] When applying the method of Example 1, it can be applied in the form of software, such as by designing it as a program that can run independently on a computer-readable storage medium. The computer-readable storage medium can be a USB flash drive, designed as a USB security token, and the program can be designed to start the entire method through an external trigger.

[0104] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for external key point matching and tracking trajectory calculation based on VGGT, characterized in that, It includes the following steps: S1: Load the configuration file and initialize the VGGT model; S2: Load the coordinates of external key points and perform standardization and quantity alignment to obtain normalized coordinates; S3: First, adjust the color channel order of the input image, normalize it, and convert it to tensor format. Then, extract image features through the backbone network of the VGGT framework to generate a feature map including depth information, camera parameters, and tracking features. Finally, perform L2 normalization on the feature map. S4: Based on the normalized coordinates, use bilinear interpolation to extract the depth feature vectors of external key points from the normalized feature map; calculate the cosine similarity of the feature vectors of key points in two frames to evaluate the matching quality; calculate the average magnitude and standard deviation of the feature vectors to evaluate the feature quality, and dynamically adjust the matching threshold according to the feature quality. The higher the feature quality, the higher the matching threshold, and the lower the feature quality, the lower the matching threshold; combine camera parameters and depth map to project 2D key points into 3D space, and calculate the coordinate difference of matching points to eliminate matches that do not meet geometric constraints, generating the tracking trajectory of key points; S5: Perform tracking quality analysis and generate visualizations.

2. The method for external key point matching and tracking trajectory calculation based on VGGT as described in claim 1, characterized in that, The method for extracting the deep feature vector includes the following steps: The keypoint coordinates are mapped to the feature map scale, and the mapping formula is as follows: ； In the formula, x and y represent the coordinates of the key points, and W and H are the width and height of the feature map, respectively; The depth feature vector is obtained by performing sub-pixel level feature sampling using the F.grid_sample function.

3. The method for external key point matching and tracking trajectory calculation based on VGGT as described in claim 1, characterized in that, The formula for calculating the cosine similarity is: similarity=cos(feat1, feat2); In the formula, similarity is the cosine similarity, and feat1 and feat2 are the feature vectors of the same key point in the two frames, respectively.

4. The method for external key point matching and tracking trajectory calculation based on VGGT as described in claim 1, characterized in that, The formula for adjusting the matching threshold is: adaptive_threshold=base_threshold * (1+mean_magnitude * 0.1); In the formula, adaptive_threshold is the matching threshold, base_threshold is the preset base threshold, and mean_magnitude is the average magnitude of the key point feature vector.

5. The method for external key point matching and tracking trajectory calculation based on VGGT as described in claim 1, characterized in that, The formula for calculating coordinates when projected into 3D space is: ； In the formula, K is the camera intrinsic parameter matrix, u and v are the keypoint coordinates, and d is the depth value.

6. The method for external key point matching and tracking trajectory calculation based on VGGT as described in claim 1, characterized in that, The standardization includes converting keypoint coordinates into a normalized coordinate format; the quantity alignment includes aligning the quantities through interpolation or deletion strategies when the number of keypoints in two frames is inconsistent.

7. The method for external key point matching and tracking trajectory calculation based on VGGT as described in claim 1, characterized in that, The tracking quality analysis includes calculating at least one of the following quality indicators: Feature statistics: The effectiveness of feature representation is evaluated by calculating the mean magnitude, standard deviation, and diversity of the feature vectors; Spatial distribution range: Analyze the distribution range and center location of key points in the image; Uniqueness score: The uniqueness of key points in a local region is evaluated by calculating the similarity of local features.

8. The method for external key point matching and tracking trajectory calculation based on VGGT as described in claim 1, characterized in that, The generated visualizations include at least one of the following charts: Histogram of characteristic amplitude distribution; A scatter plot showing the spatial distribution of key point uniqueness scores is displayed using color coding. Key point quality score ranking bar chart; A summary table of statistical information including the number of key points, feature dimensions, and matching rate.

9. The method for external key point matching and tracking trajectory calculation based on VGGT as described in claim 1, characterized in that, In step S1, system parameters are managed through a YAML configuration file. These system parameters include the camera model, feature extraction layer, and adaptive threshold base value. When selecting a device, the system automatically detects and selects the optimal computing device, and loads the VGGT model only when needed.

10. A device for external key point matching and tracking trajectory calculation based on VGGT, characterized in that, Its application is based on the VGGT-based external keypoint matching and tracking trajectory calculation method as described in any one of claims 1-9, wherein the apparatus comprises: The system initialization and configuration module is used to load configuration files and initialize the VGGT model. The keypoint loading and coordinate transformation module is used to load the coordinates of external keypoints and perform standardization and quantity alignment to obtain normalized coordinates. The feature map calculation and caching module is used to first adjust the color channel order of the input image, normalize it and convert it into tensor format, then extract image features through the backbone network of the VGGT framework to generate a feature map including depth information, camera parameters and tracking features, and finally perform L2 normalization on the feature map. The depth feature extraction and matching module is used to extract depth feature vectors of external key points from the normalized feature map based on the normalized coordinates; the depth feature extraction and matching module is also used to calculate the cosine similarity of the feature vectors of key points in two frames to evaluate the matching quality; the depth feature extraction and matching module is also used to calculate the average magnitude and standard deviation of the feature vectors to evaluate the feature quality, and dynamically adjust the matching threshold based on the feature quality; the depth feature extraction and matching module is also used to combine camera parameters and depth map to project 2D key points into 3D space, and calculate the coordinate difference of matching points to eliminate matches that do not meet geometric constraints, and generate the tracking trajectory of the key points; The Tracking Quality Analysis and Visualization module is used to perform tracking quality analysis and generate visualizations.

Images