Data analysis method for six-degree-of-freedom visual tracking measurement based on feature points

Abstract

This invention relates to the fields of visual measurement and computer vision technology, and discloses a data analysis method for six-DOF visual tracking measurement based on feature points. This method extracts multiple types of feature points from the current frame image and matches them with a 3D model feature database to establish an initial correspondence. It constructs a descriptor confidence-weighted pose observation information matrix and identifies the degradation direction and the contribution of each feature point component through singular value decomposition. The method then scores the tracking reliability prediction network and adds candidate points in high-value regions of the degradation sensitivity map. Finally, it calculates the six-DOF pose and uncertainty by weighting the descriptor confidence, reliability confidence, and component contribution, and then uses extended Kalman filtering to form a tracking trajectory with a quality score. This invention can locate the degradation direction, actively compensate for effective features, and output quantifiable pose and quality information when feature distribution is uneven or some degrees of freedom constraints are insufficient.

Description

Technical Field

[0001] This invention relates to the fields of visual measurement and computer vision technology, and more specifically, to a data analysis method for six-degree-of-freedom visual tracking measurement based on feature points. Background Technology

[0002] In operations such as online inspection of precision parts, monocular or multi-view vision schemes based on feature points are widely used to perform six-degree-of-freedom pose tracking and measurement of workpieces. The system acquires continuous images through an industrial camera, extracts features such as corner points, coded marks, and textures, and matches them with an offline 3D model feature database to calculate the spatial pose of the workpiece relative to the camera. This provides a measurement basis for robotic arm guidance and assembly positioning, and is widely used in industrial settings due to its flexible hardware configuration.

[0003] Existing tracking methods, after establishing feature correspondences, often use the number of feature points, the proportion of RANSAC inliers, or the overall reprojection error as quality criteria. During pose optimization, they often apply the same weight to each feature point or only weight it based on the descriptor matching confidence. There is a lack of unified quantification of the constraint strength of the current frame feature set in each of the six degrees of freedom directions. It is impossible to clearly identify the direction of the degree of freedom that tends to degenerate, and it is also difficult to distinguish the constraint contribution of each feature point to a specific degenerate direction. Therefore, when the constraint distribution is uneven, there is a lack of targeted feature supplementation and differential weighting basis.

[0004] When the workpiece rotates at a large angle, self-occludes, or features within the field of view converge to one side of the image, the pose observation information matrix experiences constraint collapse in some directions, resulting in insufficient observability of several degrees of freedom in translation and rotation, and increased pose estimation bias. Existing processes often struggle to synchronously output the uncertainty of each degree of freedom, and there is insufficient time-series identification of single-frame anomaly estimation and low-quality frames in the degradation stage. As a result, the assembly control system struggles to detect and track quality degradation in a timely manner, thus restricting the continuity of the tracking trajectory and the traceability of the assembly process quality. Summary of the Invention

[0005] This invention provides a data analysis method for six-degree-of-freedom visual tracking measurement based on feature points, which solves the technical problems in related technologies such as difficulty in quantitatively identifying the direction of degradation when feature point constraints degrade, lack of active compensation and quantification of degree-of-freedom uncertainty, and insufficient assessment of time-series tracking quality.

[0006] This invention provides a data analysis method for six-degree-of-freedom visual tracking measurements based on feature points, comprising the following steps: S1. Extract multiple types of feature points from the current frame image, match them with the 3D model feature database to establish an initial feature correspondence, and assign descriptor confidence to each corresponding point to obtain the initial feature correspondence set; S2, constructs a descriptor confidence weighted pose observation information matrix from the initial feature correspondence set, quantifies the constraint strength of each degree of freedom through singular value decomposition, identifies the degradation direction and calculates the contribution of each feature point component; S3, track the reliability prediction network score of the feature points in the initial feature correspondence set; when there is a degradation direction, add and screen candidate points in the high value area of ​​the comprehensive degradation sensitivity map to form an enhanced feature correspondence set with reliability confidence. S4 combines the descriptor confidence and reliability confidence in the enhanced feature correspondence set, and jointly weights them according to the contribution of the degradation direction component, to solve the six-degree-of-freedom pose of the current frame and quantify the estimation uncertainty of each degree of freedom; S5 uses the pose estimation and the uncertainty of each component of the six degrees of freedom as observations to perform extended Kalman filter time series consistency analysis, and updates the feature point reliability history database to generate a six-degree-of-freedom tracking trajectory and accuracy analysis report with quality score.

[0007] Furthermore, in S1, the multi-type feature points include corner features, coded marker features, and texture features; corner features are extracted based on image gradient structure tensor analysis and are used to calculate ORB descriptors after non-maximum suppression filtering; texture features are extracted by the FAST detector in regions where the average gradient magnitude exceeds a preset texture response threshold, and BRIEF descriptors are calculated. The coded marker features are extracted by adaptive threshold binarization and contour detection. Subpixel-level image coordinates are determined by ellipse fitting. After parsing the unique number of the coded region, a three-dimensional correspondence is directly established with the three-dimensional model feature database. The initial value of the descriptor confidence is set to the highest preset level of the three types of features. When the ellipse fitting residual or the contrast of the coded region is lower than the first quality threshold, a weight reduction correction is performed. Corner-type and texture-type features are indexed by KD-tree to perform nearest neighbor retrieval on the 3D model feature database. Matching pairs whose ratio of nearest neighbor to second nearest neighbor descriptor distance is lower than a preset matching discriminative threshold are determined to be valid, and the confidence of the descriptor is calculated by subtracting this ratio from the value of the nearest neighbor.

[0008] Furthermore, in S2, for each feature point in the initial feature corresponding set, the partial derivative of the six-degree-of-freedom pose parameters is obtained with the current initial pose as a reference to obtain the pose Jacobian matrix; the Jacobian matrix contributions of all feature points are accumulated with the descriptor confidence as the weight to construct a six-degree-of-freedom weighted pose observation information matrix. Singular value decomposition is performed on the weighted pose observation information matrix to obtain six singular values ​​and their corresponding right singular vectors arranged in descending order. Each singular value quantifies the constraint strength of the current feature set on the corresponding direction. The introduction of descriptor confidence weights compresses the contribution of feature points with low matching reliability to the information matrix, making the information matrix more realistically reflect the distribution of effective constraints in the current frame.

[0009] Furthermore, in S2, when the singular value is lower than the preset observability threshold, the corresponding right singular vector direction is identified as a degradation direction and included in the degradation direction set. The degradation severity score is defined by the ratio of the preset observability threshold to the singular value. The larger the ratio, the more severe the degradation. For each degradation direction in the set of degradation directions, the pose Jacobian matrix of each feature point in the set of initial features is multiplied by the right singular vector corresponding to the degradation direction. The magnitude of the product vector is taken as the component contribution of the feature point to the corresponding degradation direction. The larger the component contribution, the higher the constraint value of the feature point for the corresponding degradation direction of correction.

[0010] Furthermore, in S3, the tracking reliability prediction network is a three-branch fusion network; The first branch processes local image patches centered on candidate points, extracts local texture quality and occlusion signal features through multiple cascaded convolutional layers, and flattens them to form visual feature vectors. The second branch processes the recent motion history sequence of the feature point. The input is the frame-by-frame optical flow displacement sequence of the feature point in the history frames of a preset length. After being encoded by a double-layer gated cyclic unit, the output is a motion state feature vector. The third branch takes the concatenation of the component contribution vectors in each of the six degrees of freedom and the mean of the local gradient magnitude as input, and then fuses them through a fully connected layer to output the geometric constraint feature vector. When the set of degenerate directions is empty, the component contribution vectors are all zero. The three-branch outputs are spliced ​​together and processed by a fully connected layer, resulting in a reliability confidence level between zero and one.

[0011] Furthermore, the sub-process of S3 performs a tracking reliability prediction network score on all feature points in the initial feature correspondence set, which is unconditionally triggered in any frame; For feature points with accumulated tracking history, the actual optical flow displacement sequence of the most recent frame with a preset length is directly taken as the input of the second branch; for feature points that appear for the first time in the current frame and have not yet accumulated tracking history, the reverse optical flow tracking method is used to locate and predict the position frame by frame in the previous frame to construct the motion history sequence. Feature points whose reliability confidence is lower than a preset reliability screening threshold are removed from the set, and the retained feature points are passed to step S4 with their reliability confidence.

[0012] Furthermore, sub-process two of S3 is triggered only when the set of degradation directions is not empty; In the image, the contribution estimate of each location to the degradation direction is calculated using uniform grid points. The contribution estimates of each degradation direction are superimposed with the degradation severity score as a weight to form a comprehensive degradation sensitivity map. In the region of the comprehensive degradation sensitivity map that exceeds the preset candidate detection threshold, the FAST detector is densely run to extract candidate feature points, and the three-dimensional model feature database is queried to establish a three-dimensional correspondence. The newly detected candidate feature points are traced frame by frame in reverse using the Lucas-Kanade optical flow algorithm to construct a motion history sequence. The feature point reliability history database is queried, and candidate points whose historical reliability average value of the image region is lower than the preset historical pre-screening threshold are directly excluded. After the pre-screened candidate points are scored by the tracking reliability prediction network, they are sorted from high to low according to the product of reliability confidence and component contribution. Within the dynamic compensation upper limit of each degradation direction, the best candidates are selected and included in the set corresponding to the enhanced features.

[0013] Furthermore, in S4, the participating weights are constructed by weighting and summing the three components: sub-confidence, normalized degradation contribution, and reliability confidence. The normalized degradation contribution is obtained by summing the product of the component contribution of each feature point to each degradation direction in the degradation direction set and the corresponding degradation severity score, and then dividing by the maximum value of the corresponding values ​​of all feature points in the set. When the degradation direction set is empty, the normalized degradation contribution is zero, and the participation in the weighted degradation is a binary weighted sum of the descriptor confidence and the reliability confidence. The balance coefficients of the three components satisfy the normalization constraint and are calibrated by minimizing the pose estimation error on a representative tracking dataset.

[0014] Furthermore, in S4, the weighted pose calculation is divided into two stages: initialization and iterative refinement. In the initialization phase, the weighted EPnP method is adopted, which represents the three-dimensional coordinates of the feature points in the enhanced feature set as a weighted linear combination of virtual control points. The corresponding rows of the equation system are weighted by the weights of each feature point, and the camera coordinates of the control points are recovered by singular value decomposition to obtain the initial rotation matrix and translation vector. In the iterative refinement stage, Levenberg-Marquardt optimization is performed based on the initial estimate. The Huber function is applied to the reprojection error to suppress the influence of outliers. The result is multiplied by the participation weights to form a weighted objective function. The process stops when the change in the objective function is lower than the preset convergence threshold or when the upper limit of the number of iterations is reached. After the pose optimization converges, the weighted pose information matrix is ​​reconstructed at the converged pose using the participation weights and then inverted. The positive square root of each diagonal element is used as the estimated uncertainty quantification value of each component of the six degrees of freedom.

[0015] Furthermore, in S5, the extended Kalman filter state vector includes the workpiece's position in the camera coordinate system, rotational attitude represented by quaternions, linear velocity, and angular velocity, with uniform motion as the prediction model; the uncertainty of each component of the six degrees of freedom directly participates in the formation of the innovation covariance matrix, and the component with larger uncertainty allows for a larger innovation deviation; The innovation vector is composed of translational innovation and rotational innovation; the translational innovation is obtained by the difference between the observed translation vector and the predicted translation vector, and the rotational innovation is obtained by converting the relative rotation between the observed rotation matrix and the predicted rotation matrix into a rotation vector using the Rodrigues formula; The Mahalanobis distance of the innovation vector is calculated under the innovation covariance matrix to quantify the consistency between the current frame and the motion model. When the Mahalanobis distance exceeds the preset anomaly detection threshold, the current frame is marked as an anomaly frame, and the filter prediction value is used instead of the measurement value for time-series output. The comprehensive quality score of each degree of freedom component is obtained by weighting and fusing the ratio of the uncertainty of the current frame to the mean uncertainty of the same component within the historical sliding window and the normalized amplitude of the Kalman innovation vector in the corresponding component direction. The score is then averaged by a sliding window of a preset length to form a smooth quality score time series. The historical reliability database of feature points updates the reliability statistics of the corresponding image region by feeding back the reprojection error using an exponential moving average, providing historical priors for the pre-screening of candidate points in the next frame.

[0016] The beneficial effects of this invention are as follows: By constructing a pose observation information matrix weighted by descriptor confidence and performing singular value decomposition, the constraint strength of the current frame feature set in each of the six degrees of freedom, the set of degradation directions, and the contribution of each feature point component are quantitatively output. This clarifies the spatial distribution of directions with insufficient constraints and feature points that still have correction value, providing a unified basis for candidate point supplementation and joint weighted pose calculation guided by the comprehensive degradation sensitivity map. When the workpiece rotation causes uneven feature distribution, compensation feature points can be actively detected and screened in areas with greater potential for constraint in degradation directions on the image. Participation weights are constructed using descriptor confidence, reliability confidence, and component contribution, so that the weight of each feature point in the pose calculation during the degradation stage matches its geometric contribution, suppressing estimation jumps and bias accumulation caused by simply relying on locally visible features or uniform weights. The pose estimation uncertainty of each degree of freedom is incorporated into the innovation covariance of the extended Kalman filter. Components with larger uncertainties allow for greater innovation deviations without misjudging anomalies. Based on motion patterns, anomaly estimations in a single frame are identified, and the filtered predicted values ​​replace the anomaly measurement outputs. Simultaneously, quality scores for each degree of freedom and accuracy analysis reports for assembly tasks are generated. The feature point reliability history database accumulates the reliability patterns of each image region using an exponential moving average method during multiple assembly processes and feeds them back for pre-screening candidate points in the next frame, forming a closed loop between tracking, pose calculation, and historical priors. This helps maintain the temporal continuity of the six-degree-of-freedom tracking trajectory and provides the assembly control system with quality metadata beyond single pose values ​​to support decision-making responses. Attached Figure Description

[0017] Figure 1 This is a flowchart of the data analysis method for six-degree-of-freedom visual tracking measurement based on feature points according to the present invention; Figure 2 This is a flowchart of the data analysis method for six-degree-of-freedom visual tracking measurement based on feature points according to the present invention. Detailed Implementation

[0018] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0019] At least one embodiment of the present invention discloses a data analysis method for six-degree-of-freedom visual tracking measurement based on feature points. This method constructs an observation information matrix weighted by descriptor confidence for the current frame's feature point set and performs singular value decomposition to quantify the constraint strength in each direction of the six degrees of freedom into calculable values. Simultaneously, it calculates the contribution of each feature point to each degradation direction, indicating where the problem exists and providing spatial guidance on where it is meaningful to find compensation feature points in the image. Based on this, regardless of the existence of degradation directions, the reliability of existing feature points is evaluated, and compensation candidate points are actively detected and screened when degradation directions exist. Finally, pose optimization is driven by the joint weight of descriptor confidence, degradation direction contribution, and reliability confidence, ensuring that feature points with effective constraints in degradation directions obtain weights matching their actual contributions in the solution. Figures 1 to 2 As shown, it includes the following steps: S1. Extract multiple types of feature points from the current frame image, match them with the 3D model feature database to establish an initial feature correspondence, and assign descriptor confidence to each corresponding point to obtain the initial feature correspondence set; In one embodiment of the present invention, the sources of feature points in the workpiece image are divided into three categories: corner features are generated based on local curvature changes in the workpiece geometry; coded marker features are derived from circular coded patterns pre-pasted at key locations on the workpiece surface; and texture features are selected from local areas on the workpiece surface rich in gradient information. The three types of features each have their own emphasis in terms of spatial distribution, extraction stability, and geometric constraint characteristics. Coded marker features have the highest positioning accuracy because their unique numbers eliminate the need for descriptor matching; corner features are next; and texture features are the most numerous but have relatively low cross-viewpoint matching stability. These reliability differences will be reflected in the descriptor confidence score and ultimately affect the weight allocation of each feature point in pose calculation. Corner feature extraction is based on image gradient structure tensor analysis. After suppressing sensor noise by Gaussian kernel filtering, the gradient structure tensor of each pixel in its local neighborhood is calculated. The ratio of the smaller to the larger of the two feature values ​​of the structure tensor is used as the discriminant. When the ratio is higher than the corner response threshold, there are significant gradient changes in multiple directions around the pixel, which is then identified as a corner candidate. Non-maximum suppression is performed on the corner response map of the entire image to retain the candidate points with the strongest responses in the local neighborhood and remove overly dense redundant candidates. For each corner candidate that passes the screening, a local image patch is extracted centered on it, and an ORB (Oriented Fast and Rotated BRIEF) descriptor is calculated. The ORB descriptor incorporates the main direction of the feature into the positional arrangement of the binary description, so that the descriptors generated by the same corner point can still maintain relatively stable matching characteristics when the workpiece is rotated to different viewpoints, and has a certain adaptability to viewpoint shifts caused by changes in workpiece posture during tracking. The extraction of coded marker features is based on contour detection and code parsing. An adaptive thresholding method is used to generate a binary image of the current frame. All connected contours are extracted from the binary image, and preliminary screening is performed based on contour area and roundness to eliminate interference contours that clearly do not conform to the geometric features of the coded marker. Ellipse fitting is performed on the retained candidate contours, and the geometric center coordinates of the fitted ellipse are used as the sub-pixel-level image coordinates of the coded marker. Subsequently, the brightness and darkness patterns of the annular coded region within the marker contour are read to parse out the unique numerical code of the marker. The coded marker directly establishes the correspondence between its image coordinates and the 3D coordinates in the workpiece coordinate system by uniquely matching its number with the corresponding record in the 3D model feature database, without relying on descriptor matching. Therefore, even under the condition that any feature descriptor is unlikely to provide a high-quality match, the coded marker can still provide accurate 3D constraint information. The initial confidence value of the coded marker's descriptor is directly set to the highest level of the three types of features, with a default value of 0.95. When the ellipse fitting residual or the contrast of the coded region of a certain coded marker in the current frame is lower than the corresponding first quality threshold, the confidence value of the marker is subject to corresponding weighting correction to reflect the decrease in its extraction quality in that frame.

[0020] Texture features are extracted from areas with rich local texture, such as workpiece nameplates, engravings, or rough surface areas. First, the mean gradient magnitude of local image patches is calculated for the current frame image. Regions with a mean exceeding the texture response threshold are marked as texture-rich areas. Within these regions, a FAST corner detector is run at a high density to extract candidate texture feature points. For each candidate point, a BRIEF (Binary Robust Independent Elementary Features) descriptor is calculated, representing the local appearance features of the candidate point as a compact binary string. The cross-view matching stability of individual descriptors for texture features is relatively low, and mismatches are more likely to occur under conditions of large-angle viewpoint changes. This characteristic will be reflected in the descriptor confidence scoring stage.

[0021] After the three types of feature points are extracted, a matching relationship is established between the image features and the workpiece 3D model feature database. The 3D model feature database is built offline during the system deployment phase: based on the accurate 3D digital model of the workpiece, the 3D positions of various feature points are sampled on the model surface, their 3D coordinates in the workpiece coordinate system are recorded, and a reference descriptor set covering multiple viewpoints is generated for each feature point, enabling the database to match the appearance changes of the workpiece under different postures. During the online tracking phase, the matching of corner-type and texture-type features uses a KD-tree index to perform nearest neighbor retrieval on the database reference descriptor set. The ratio of the distance between the current frame descriptor and the retrieved nearest and second nearest neighbors is calculated. Matches with a ratio lower than the matching discriminative threshold are considered valid, and the initial confidence value of the descriptor for the match is "1 minus this ratio". The smaller the ratio, the stronger the discriminative power between the best and second-best matches, and the higher the confidence score. In cases where multiple overlapping feature points are repeatedly detected in the same workpiece area, the one with the highest confidence is retained, and the rest are removed.

[0022] Each successfully established feature point forms a corresponding record, which includes its image coordinates, three-dimensional coordinates in the workpiece coordinate system, feature type identifier, and descriptor confidence score. All valid corresponding records constitute the initial feature correspondence set. This set serves as the direct input for subsequent observability analysis; its quantity, type distribution, and confidence level collectively determine the information basis for the information matrix analysis. In the vast majority of frames of workpiece motion, It contains dozens to hundreds of corresponding records from three types of features; in difficult frames where feature points degenerate, the number of initial correspondences may decrease, and the observability analysis results of step S2 will reveal the deficiencies in the constraint direction and trigger the candidate point compensation sub-process of step S3.

[0023] S2, constructs a descriptor confidence weighted pose observation information matrix from the initial feature correspondence set, quantifies the constraint strength of each degree of freedom through singular value decomposition, identifies the degradation direction and calculates the contribution of each feature point component; The initial feature set provides image coordinates, 3D coordinates, and descriptor confidence for each corresponding record. However, these three pieces of information alone cannot answer a crucial question for tracking accuracy: which directions of the six degrees of freedom are sufficiently constrained by these feature points, and which are insufficiently constrained? This step answers this question by constructing a weighted observation information matrix and performing singular value decomposition. Its core output is the component contribution of each feature point to the degradation direction. This quantity directly tells step S3 "in which regions of the image should compensation feature points be found to effectively alleviate degradation," and is the information starting point of the entire active compensation link.

[0024] For sets The Middle Each corresponding point, its three-dimensional coordinates The predicted pixel coordinates projected onto the image at the current frame pose are given by the camera projection model. The projection function is applied to the six-DOF pose parameters (in Lie algebras). It means that, among them Let be the rotation vector. Taking the partial derivative of the translation vector, we obtain the first... The pose Jacobian matrix of each feature point ;Position Jacobian matrix The geometric meaning of the column is: if the pose follows the first... When a unit perturbation occurs in a direction of a degree of freedom, how much will the projected coordinates of that feature point change? When the magnitude of a column is close to zero, no matter how much deviation occurs in that degree of freedom, this feature point will not be aware of it and cannot provide effective correction constraints. During the tracking phase, the refined pose of the previous frame (the first frame is the initial pose) is used as the initial pose estimate for the current frame to calculate the pose Jacobian matrix of each feature point.

[0025] Descriptor confidence Using these weights, the contributions of all feature points in the set are summed to construct a weighted pose observation information matrix for the leaf tracking scene:

[0026] in This is a six-degree-of-freedom weighted observation information matrix of the current frame feature point set in the blade assembly scene, with dimension [missing information]. ; For the first The descriptor confidence score for each feature point, with a value range of [value range missing]. The value is calculated from the matching distance ratio in step S1, and the default value of the encoding tag is 0.95. For the first The pose Jacobian matrix of each feature point in the current initial pose; for The positive semi-definite matrix reflects the distribution of the constraint capability of the feature point in each of the six degrees of freedom directions. Introducing... The weighting reduces the contribution of feature points with low matching reliability to the information matrix, allowing the information matrix to more accurately reflect the distribution of effective constraints in the current frame.

[0027] Information matrix Perform singular value decomposition to obtain ,in Arranged from largest to smallest Each column Let be the corresponding right singular vector, representing the principal direction of the constraint strength in the six-degree-of-freedom parameter space. Singular values The larger the feature set is, the more directional it is. The more sufficient the constraints; When the value approaches zero, the current feature set has almost no constraint on that direction. When the blade rotation causes the feature points to concentrate on one side of the image, the singular values ​​corresponding to the rotational degrees of freedom in the mounting axis normal direction first show a phased collapse, while the singular values ​​in other directions remain at the normal order of magnitude. It is this non-uniform degradation feature that enables singular value decomposition to accurately locate the degradation direction, which cannot be achieved by simply relying on the decrease in the global confidence level.

[0028] Let the observability threshold be... ,when Time, direction It is identified as a degradation direction and included in the degradation direction set. The severity score for degradation in this direction is defined as follows: The larger the ratio, the more severe the degradation. The value of is determined during the calibration phase before system deployment: A large amount of workpiece tracking data is collected under various postures; the distribution range of singular values ​​in each frame's information matrix when the tracking accuracy meets process requirements is statistically analyzed; and the lower quartile of the singular value distribution during the stable tracking phase is multiplied by a safety factor (default value is 0.6) as . The calibration value. Through this calibration process, It is directly related to the accuracy requirements and workpiece geometry of the specific deployment scenario, avoiding the insufficient scenario adaptability caused by fixed thresholds. During the assembly verification of multiple sets of blades, the inventors observed that the singular values ​​corresponding to the rotational degrees of freedom around the mounting axis typically begin to decrease significantly after the blade rotation angle exceeds approximately 55 degrees. This pattern can be used as a reference during the calibration phase. Experimental reference benchmark for initial estimates.

[0029] While identifying the degradation direction, the contribution of each feature point to the components of each degradation direction is calculated. For each degradation direction... ( ), No. The contribution of each feature point The calculation method is as follows: the pose Jacobian matrix of the feature point is... Singular vectors with degenerate direction Multiply, and take the L2 norm of the product vector, i.e.

[0030] in Reflects the pose at the 1st When a deviation occurs in the direction of degradation, the first The larger the reprojection residual response generated by a feature point, the higher its value in correcting the degradation direction. Based on component contribution, two types of feature points that are treated equally in traditional methods can be distinguished from the existing feature set: feature points with acceptable descriptor confidence but extremely low contribution to the degradation direction, and feature points with slightly lower descriptor confidence but high contribution to the degradation direction. Their actual constraint values ​​under degradation conditions are drastically different, and the latter will receive priority in the active compensation in step S3 and the joint weighting in step S4.

[0031] Output the six singular values ​​and their corresponding right singular vectors and the set of degenerate directions. and severity scores for each direction of degradation. And the component contribution matrix of each feature point in the initial feature set to the degradation direction. .like If the value is empty, it means that the current frame feature point distribution has sufficient constraints on all six degrees of freedom in each direction, and the candidate point active detection and compensation sub-process in step S3 can be skipped directly; however, the RPPN reliability scoring sub-process for the original feature points in step S3 must still be executed unconditionally to ensure that each feature point in step S4 carries a reliability confidence level. This ensures the integrity of the joint weighted pose calculation.

[0032] S3, track the reliability prediction network score of the feature points in the initial feature correspondence set; when there is a degradation direction, add and screen candidate points in the high value area of ​​the comprehensive degradation sensitivity map to form an enhanced feature correspondence set with reliability confidence. Step S3 handles two independent but closely related sub-processes: Sub-process one evaluates the tracking reliability of all original feature points in the initial set of step S1, providing the reliability confidence level of each feature point for step S4. This sub-process is executed unconditionally in any frame; the second sub-process is executed when the set of degradation directions... When the image is not empty, new candidate feature points with high constraint potential on the degradation direction are actively searched in the image and included in the set through reliability screening. This sub-process is triggered only when the degradation direction is identified in step S2. The two sub-processes share the same Tracking Reliability Prediction Network (RPPN) to complete the scoring. Therefore, the architecture and training strategy of RPPN are described first, and then its application in the two sub-processes is explained.

[0033] RPPN is a lightweight multi-branch fusion network designed specifically for current tracing tasks (see attached manual). Figure 2The diagram shows the three-branch RPPN network structure, which integrates three types of information: local visual quality, recent motion history, and geometric constraint characteristics for reliability prediction. The first branch processes a 32×32 pixel local image patch centered on a candidate point. This patch is processed through three cascaded convolutional layers to extract texture quality and occlusion signal features: the first convolutional layer uses 16 5×5 kernels to calculate the distribution statistics of local gradient magnitudes, assessing the sharpness of the local texture; the second convolutional layer uses 32 5×5 kernels to calculate the uniformity of directional gradient distribution with a larger receptive field, capturing whether the texture has sufficient directional diversity to support stable matching; the third convolutional layer uses 32 3×3 kernels to focus on whether there are obvious occlusion boundaries or motion blur signs in the edge regions of the image patch. The feature maps from the three convolutional layers are compressed by max pooling and flattened to form a 64-dimensional visual feature vector.

[0034] The second branch processes the recent motion history sequence of feature points. The default sequence length is 8 frames. The input is the frame-by-frame optical flow displacement vector of the feature point over the past 8 frames (2 dimensions per frame, 16 dimensions in total). The temporal motion data is encoded using a two-layer GRU (Gated Recurrent Unit, 64 dimensions per layer of hidden states), outputting a 32-dimensional motion state feature vector. The 8-frame sequence length covers approximately 0.27 seconds of history at a frame rate of 30 frames per second, sufficient to cover the typical transition period from initial exposure to gradual stabilization of a single feature point during the blade rotation degradation stage. This allows the GRU to distinguish between "unstable points just exposed from the occlusion area" and "stable points that remain visible." The third branch processes geometric constraint information. The input is designed as a concatenation of a fixed 6-dimensional contribution vector and a 1-dimensional gradient magnitude mean, for a total of 7 dimensions: the 6-dimensional contribution vector... dimension( (Corresponding to the six-degree-of-freedom parameter space) A principal direction, if that direction belongs to the set of degradation directions of the current frame. Then fill in the direction of the feature point. Contribution of the component If the direction is not If the middle is filled with zero; when When empty, the entire 6-dimensional contribution vector is zero. The RPPN score is mainly driven by the first two branches, while the output of the third branch is biased towards neutrality. The fixed 6-dimensional design ensures that the RPPN architecture is completely determined before deployment and does not change with variations in the number of inter-frame degradation directions. The third branch fuses the 7-dimensional input into a 32-dimensional geometric constraint feature vector through a fully connected layer (output dimension 32). The output vectors of the three branches (a total of 128 dimensions) are concatenated and then passed through two fully connected layers with ReLU activation functions (the first layer reduces the 128-dimensional input to 64-dimensional, and the second layer reduces the 64-dimensional input to 1-dimensional via a sigmoid output), ultimately providing a reliability confidence score between 0 and 1. A higher value indicates that the tracking reliability of the feature point in the current frame is stronger.

[0035] Before system deployment, RPPN was trained offline using historical tracking sequences of similar scenarios. Self-supervised annotation was based on a baseline tracking method containing RANSAC (Random Sample Consensus Algorithm). The baseline method provides reliable in-situ / out-of-situ point classification results in normal frames with sufficient feature point constraints. RANSAC in-situ / out-of-situ point labels served as the basis for reliability annotation. Training data was collected only from frames with normal observability (i.e.,... Collection from empty frames: The set of interior points in RANSAC of observable degraded frames is unreliable due to insufficient constraints. Using the RANSAC results of degraded frames as the basis for labeling would introduce biased training signals, so they are excluded. Normal frames usually account for 75% to 85% of the entire sequence in actual tracking sequences, which can provide sufficient and reliable training samples. The feature reliability judgment rules learned by RPPN can be naturally generalized to the evaluation of candidate points in degraded frames. Positive samples are feature points whose reprojection error is consistently below the stability threshold and have not been removed by RANSAC. Negative samples are feature points with abrupt changes in reprojection error or that have been removed by RANSAC in multiple consecutive frames. Training uses binary classification cross-entropy loss, and oversampling of negative samples is performed to alleviate the problem of imbalance between positive and negative samples. After training, the RPPN weights are fixed in a read-only state, and only forward inference is performed during the online tracking phase.

[0036] In one embodiment of the present invention, sub-process one is (unconditionally executed): initializing the corresponding set for step S1. For all feature points in the dataset, calculate the reliability confidence level of each point using RPPN. The original feature points have complete historical tracking trajectories. The input of the second branch directly takes the actual optical flow displacement sequence of the most recent 8 frames. For feature points that appear for the first time in the current frame and have not yet accumulated tracking history (such as newly exposed corner points after blade rotation), the same reverse optical flow tracking method as the candidate points in this step is used to construct the motion history sequence of the past few frames. Feature points whose reliability confidence of the RPPN output is lower than the reliability screening threshold are removed from the set to avoid feature points with significantly insufficient reliability interfering with subsequent pose calculations. After screening, the retained original feature points constitute a sequence with... The base set of annotations, regardless of Whether it is empty or not, directly pass it to step S4; Subprocess 2 is (only when) (Execute when not empty): for the set of degenerate directions Each direction in In the image, uniformly spaced grid points are used as candidate locations. For each grid point, temporary 3D coordinates are obtained using its image coordinates and the current frame depth estimate (approximated by the intersection of the workpiece surface geometry model and the camera's line of sight, using the refined pose of the previous frame as the initial estimate for the current frame). The pose Jacobian matrix row corresponding to this temporary 3D point is then calculated, and its position relative to the degradation direction is obtained. Contribution estimate The spatial distribution characteristics of the sensitivity map mainly depend on the geometric distribution of currently visible feature points and the camera projection geometry. Even if there is an estimation error of less than 2 degrees in the initial pose, the spatial position of the high-value region will not shift significantly. The conclusion that "the focus is on detection on the side opposite to the blade section projection" during the degradation stage remains stable within this error range. The contribution estimates of multiple degradation directions are divided according to the severity of each direction. By weighting and summing, we obtain the overall degradation sensitivity map for the current frame:

[0037] in Represents image coordinates The potential feature points provide comprehensive constraint compensation value for mitigating all current degradation directions. For the first The severity of the degradation direction is scored, with more severe degradation directions having a higher weight in the overall graph. Within the region exceeding the candidate detection threshold, the FAST corner detector is run intensively to extract candidate feature points. For each candidate point, an ORB descriptor is calculated and the 3D model database is queried to establish a 3D correspondence with the same matching process as in step S1. For candidate points that cannot find a nearest neighbor in the database, the approximate 3D coordinates are estimated based on their image position and workpiece surface geometry interpolation. The candidate point is marked as an interpolation candidate, and its subsequent confidence is subject to additional weighting.

[0038] For newly detected candidate feature points, starting from the current frame image coordinates, the Lucas-Kanade optical flow algorithm is used to reverse-locate the predicted position of the point frame by frame in the preceding frames, extracting the displacement vector of each frame. This constructs a frame-by-frame motion history sequence of the candidate point over the past 8 frames, which serves as the input to the second branch of the RPPN. This reverse tracking method has additional filtering value: candidate points that have just been exposed from the occluded area during blade rotation will show jumps or mismatches in the reverse tracking of the occluded preceding frames, and the GRU will output a lower motion stability score accordingly; candidate points that are smoothly and stably reverse-tracked in the preceding frames indicate that the area is continuously observable, and the tracking reliability of the current frame is higher. If the reverse tracking fails consecutively, the remaining sequence is repeatedly filled with the displacement vector of the last successfully tracked point, and a failure flag is recorded to apply additional weighted correction to the output of the second branch.

[0039] Before performing RPPN scoring on candidate points, the historical reliability average of the corresponding image region is retrieved from the feature point reliability history database. If the historical average of the image region containing a candidate point is lower than the historical pre-screening threshold, the candidate point is directly excluded, and RPPN inference is not performed. The history database accumulates reliability patterns of different image regions during repeated system assembly, enabling the system to automatically skip regions on the blade that have long exhibited unstable performance (such as near the thin edge of the blade leading edge), concentrating RPPN inference resources on historically stable and usable regions, thus improving the overall efficiency of candidate point screening. Candidate points that pass the pre-screening enter the RPPN database, and the reliability confidence score is output. Simultaneously, the component contribution of each candidate point to each direction of the degradation direction set is calculated (using the same calculation method as in step S2, used for the input of the third branch and the subsequent candidate point selection and sorting). The inclusion of candidate feature points is constrained by the dynamic compensation upper limit of each degradation direction. The maximum number of candidate points allowed to be included in each degradation direction is:

[0040] in The basic compensation amount (default value is 8). To score the severity of degradation, This is the absolute upper limit (default value is 25). Overall reliability confidence level. and component contribution Sort the two by their product from high to low, and select the top-ranked candidate points within the upper limit of each degradation direction to include in the set.

[0041] Output augmented feature correspondence set This includes the original feature points after filtering by subprocess one, and in New candidate feature points are included by subprocess 2 when the space is not empty. Regardless of Whether it is empty, each feature point carries image coordinates, 3D coordinates, feature type identifier, and descriptor confidence. and RPPN reliability confidence .

[0042] S4 combines the descriptor confidence and reliability confidence in the enhanced feature correspondence set, and jointly weights them according to the contribution of the degradation direction component, to solve the six-degree-of-freedom pose of the current frame and quantify the estimation uncertainty of each degree of freedom; Each feature point in the enhanced feature set carries both a descriptor confidence score and an RPPN reliability confidence score. These two scores evaluate the quality of the feature point from different dimensions. The descriptor confidence score reflects the discriminativeness and stability of the point's descriptor matching in the current frame, and is an assessment of "whether the current match is accurate." The reliability confidence score integrates local visual quality, motion history, and geometric constraint characteristics, and is a comprehensive prediction of "the overall credibility of the point's participation in the solution." The core design of this step is to fuse the two into a unified participation weight, and to additionally superimpose the contribution of the degradation direction component into the weight.

[0043] The construction of the weights involves two steps. The first step is to calculate the set. Original degradation contribution of each feature point The sum of the product of the contribution of this feature point to each direction of the degradation direction set and the corresponding degradation severity score is calculated.

[0044] Then take the set All feature points maximum value , to each point Divide by Obtain normalized degradation contribution Map it to Scope; when When empty ,make To avoid division by zero, at this time Normalization eliminates the dimensional differences between the pose Jacobian matrix norm and the descriptor confidence and reliability confidence, bringing the three components to the same numerical scale. Only then do the default values ​​of the balance coefficients have a real meaning of "weight balancing." The second step is to construct the final participating weights using the weighted sum of the three components:

[0045] in For the first The final participation weight of each feature point in the blade assembly pose calculation; The confidence level of the descriptor serves as the source of the base weights; To normalize the degradation contribution, this component ensures that feature points with high potential to constrain the current degradation direction can receive a corresponding increase in weight. When empty, this item is zero, and the weight automatically degenerates into a binary weighted average of the descriptor confidence and the reliability confidence; The reliability confidence level of RPPN is used as a stability correction term. , , Let each of the three components represent the balance coefficients, satisfying... , The default value is 0.20. The default value is 0.30. The default value is 0.50, and the specific value is determined by system calibration on a representative tracking dataset by minimizing the pose estimation error.

[0046] Weighted pose solving is divided into two stages: initialization and iterative refinement. These two stages work together to ensure the global rationality and high local accuracy of the solution results. The initialization stage uses the weighted EPNP method: the set... The 3D coordinates of the feature points are represented as a weighted linear combination of four virtual control points. The reference coordinate system position of the control points is determined by performing singular value decomposition on the weighted 3D coordinate matrix. In the camera coordinate system, a system of linear equations is established using the feature point image coordinates and projection constraints. The equation row corresponding to each feature point in the system of equations is multiplied by the weight. The positive square root of the value makes the contribution of high-weight feature points to the linear system greater. Singular value decomposition is performed on the weighted linear system to recover the camera coordinate representation of the control points from the solution corresponding to the minimum singular value, thereby restoring the camera coordinates of all feature points. The initial rotation matrix and translation vector are calculated through Procrustes alignment. Weighted EPnP can provide a globally reasonable initial pose estimate within linear time complexity, providing a reliable convergence starting point for subsequent LM refinement and reducing the risk of LM getting trapped in local extrema.

[0047] Based on the initial EPNP estimate, Levenberg-Marquardt (LM) iterative refinement is performed. For the set... For each feature point in the image, calculate the difference between its 3D coordinates in the reprojected pixel coordinates under the current pose estimation and the measured image coordinates to obtain the 2D reprojection error vector. Calculate the square norm of the reprojection error. ;right Applying Huber functions: when Below the Huber threshold, the loss is equivalent to squared loss; above this threshold, it switches to linear loss to reduce the impact of outliers. The default value for the Huber threshold is 9.0 (corresponding to the square of a 3-pixel reprojection error), and it is calibrated based on the camera resolution and workpiece texture density during actual deployment. The Huber function output is then multiplied by the participating weights. This forms the weighted contribution of each feature point to the objective function. In each iteration, the LM algorithm calculates the pose Jacobian matrix of all feature points with weighted projection errors against the pose parameters, forming a normal equation with a damped term. By dynamically switching the damping factor between gradient descent and Gauss-Newton descent, it ensures a large step size descent rate when far from the optimal solution and a high-precision second-order convergence rate when approaching the optimal solution. Refinement stops when the change in the objective function between adjacent iterations falls below the convergence threshold or the number of iterations reaches the upper limit, and the converged rotation matrix is ​​output. Translation vector .

[0048] After pose optimization convergence, the uncertainty of each component of the six degrees of freedom is quantified using the weighted information matrix at the convergence point. (This is in the context of the converged pose.) At this point, the information matrix is ​​recalculated using the weights of each feature point:

[0049] in The weighted pose information matrix at the convergent pose has dimension . ; To recalculate the first [value] in the convergent pose The pose Jacobian matrix of each feature point; The participation weights constructed for step S4 have incorporated descriptor confidence, normalized degradation contribution, and reliability confidence. Inverting the matrix yields an approximation of the covariance matrix for pose estimation. , its first The diagonal element corresponds to the first The estimated variance of each of the six degrees of freedom components is calculated by taking the positive square root of each diagonal element to obtain the quantified value of the uncertainty of each component. The first three components correspond to the three-axis translation uncertainty, and the last three components correspond to the three-axis rotation uncertainty. This step ultimately outputs the six-DOF pose estimation result for the current frame. and the vector of uncertainties for each of the six degrees of freedom components. .

[0050] S5, using the pose estimation and the uncertainty of each component of the six degrees of freedom as observations, performs extended Kalman filter time series consistency analysis, updates the feature point reliability history database, and generates a six-degree-of-freedom tracking trajectory and accuracy analysis report with quality score; Even after joint weighted optimization, the pose estimation results of a single frame cannot completely eliminate two types of problems: First, in a certain frame, abnormal image acquisition quality (such as instantaneous vibration or brief occlusion) leads to an overall low quality of the corresponding features, resulting in isolated abnormal estimates that deviate from the actual trajectory. Second, in several frames, insufficient constraints during the feature degradation stage lead to significantly higher uncertainties in certain degree-of-freedom components, but the solution results appear to be within a reasonable range, without triggering explicit erroneous judgments. This step, through temporal modeling of pose estimation results across multiple frames, evaluates the quality of each frame's estimation results from the perspective of consistency in motion patterns, thereby effectively identifying the above two types of problems.

[0051] The temporal analysis framework is built upon an extended Kalman filter. The filter's state vector includes the workpiece's position (3D) in the camera coordinate system, its rotational attitude (4D) represented by quaternions, linear velocity (3D), and angular velocity (3D), totaling 13 state variables. The motion prediction model assumes uniform motion: linear velocity drives position changes, and angular velocity drives attitude changes; both are considered constants during the prediction phase. The process noise covariance matrix is ​​determined before system deployment through the following calibration steps: The robotic arm is driven to execute a standard assembly trajectory under conditions free from external measurement noise interference. A high-precision external reference system (such as a laser tracker) records the actual position and attitude changes between consecutive frames. The variance of the inter-frame changes in each dimension is statistically calculated, and 1.5 times this variance is used as the default value for the corresponding process noise covariance component. The coefficient 1.5 provides an additional margin to cope with slight random jitter during actual assembly.

[0052] Step S4 outputs the rotation matrix Translation vector Before entering the EKF, representation transformation and innovation vector calculation must be completed. Rotation matrix After converting to a unit quaternion using the Shepperd method, the rotation component in the EKF state vector is updated to ensure that the quaternion normalization constraint is always satisfied. The innovation vector is calculated independently in two parts: the innovation vector of the translation component is directly derived from the observed translation vector. Subtract the EKF predicted translation vector The 3D difference is obtained; the innovative vector of the rotation component is calculated by measuring the observed rotation matrix. EKF Predicted Rotation Matrix Relative rotation between Then The Rodrigues formula is used to convert the innovation into a corresponding rotation vector (axis-angle representation), resulting in a 3D rotational innovation component. The rotational innovation and translational innovation are then concatenated to form a 6D innovation vector, which is used in subsequent Mahalanobis distance calculations.

[0053] The Mahalanobis distance of the innovation vector is calculated under the innovation covariance matrix to quantify the consistency between the current frame estimation result and the motion model prediction. The uncertainty of each component output in step S4 is... The innovation covariance matrix is ​​directly involved in its construction. Components with higher uncertainty allow for greater innovation deviations without triggering anomaly detection, while components with lower uncertainty are more sensitive to deviations of the same magnitude. When the Mahalanobis distance exceeds the anomaly detection threshold, the current frame is marked as an anomaly frame, and the time-series output replaces the current frame's measurement value with the filter's predicted value to prevent anomaly measurements from polluting the prediction basis of subsequent frames through Kalman updates. The default value for the anomaly detection threshold is 11.07, corresponding to the chi-square distribution critical value of the 6-dimensional innovation vector at a significance level of 0.05. Through time series analysis, a comprehensive quality score is generated for each degree of freedom component in the current frame: the ratio of the quantified value of the uncertainty of the component in the current frame to the mean uncertainty of the same component within the historical sliding window is used as the first factor (reflecting the absolute quality level), and the normalized amplitude of the Kalman innovation vector in the direction of the component is used as the second factor (reflecting the degree of conformity with the motion law). The two are weighted and fused at a fixed ratio of 0.4 and 0.6, and then converted into a quality score by subtracting the fused value from 1, with a value between 0 and 1. The quality score sequence is averaged using a fixed-length sliding window, with the default value of the sliding window length being 10 frames, to obtain a smooth quality score time series for each degree of freedom component.

[0054] The feature point reliability history database is updated after each frame is processed. For each feature point involved in the pose calculation of the current frame, the actual stability feedback for the current frame is calculated based on its reprojection error at the converged pose: the feedback value is 1 (stable) when the reprojection error is below the stability threshold, and linearly decays according to the degree of excess when it exceeds the threshold, resulting in an actual stability feedback value between 0 and 1. This feedback value is updated to the reliability statistics of the corresponding feature type and image region in the history database using an exponential moving average method, giving higher update weight to recent data while retaining the influence of long-term historical trends. In step S3 sub-process two of the next frame, the history database provides image region-level historical reliability priors for newly detected candidate points in high-value areas of the sensitivity map, serving as a pre-screening threshold before candidate points enter the RPPN. This allows the system to accumulate knowledge of the reliability patterns of each image region during repeated assembly processes, forming a virtuous cycle of feedback between the tracking process and the history database.

[0055] The output includes two types of data: a six-DOF tracking trajectory with temporal quality assessment, where each frame's trajectory points carry six-DOF pose values ​​(in the form of rotation matrices and translation vectors) and a corresponding 6-dimensional quality score vector; and an accuracy analysis report for the current tracking task, covering the average quality score of each degree of freedom across the entire tracking trajectory, the number of frames with quality scores below a preset quality threshold, the temporal distribution and occurrence patterns of abnormal frames, and the actual contribution statistics of three types of feature points to the overall tracking accuracy (the statistical method is to calculate the average participation weight of each type of feature point in all frames in which each type of feature point participates in the calculation). Compared with the average reprojection error, the weighted participation rate measures the actual contribution ratio of various feature points to the pose solution. This information provides the assembly control system with quality metadata beyond a single pose value, enabling the control system to make corresponding decision responses when faced with frames that have degraded tracking quality, ensuring the traceability of assembly process accuracy and safety.

[0056] In one embodiment of the present invention, applied to the field of precision assembly of parts, the automated assembly measurement of the first-stage compressor blade of a certain type of engine is used as a specific application example, and the tracking data on the real assembly line is simulated for illustration.

[0057] In this application example, after the robotic arm picks up the blade from the storage rack, it performs a three-axis spatial translation of approximately 300 mm and a rotation of approximately 90 degrees around the blade mounting axis along a predetermined trajectory. Combined with minor oscillating adjustments, the blade is then fed into the compressor casing and oriented into the blade slot. The tracking system uses a single-lens industrial camera with a resolution of 2448×2048 pixels and a frame rate of 30 frames per second, continuously acquiring images of the blade's motion from a fixed viewpoint above the casing. Six circular coded markers are pre-attached to the blade surface: two at the blade root, two in the middle, and two near the blade tip. Several stable corner points are provided on the blade's contour boundary, and local texture information is present in the platform area at the blade root. When the system starts, the robotic arm first moves the blade to the initial detection position. The initialization process described in step S1 utilizes all six visible coded markers to complete the initial pose estimation, and then enters the inter-frame tracking mode.

[0058] The tracking sequence consists of 90 frames, covering the complete motion process of the blade from material picking to positioning. Frames 1 to 29 represent the translation phase, where the blade attitude changes smoothly, all six coded markers are visible, the minimum singular value of the information matrix stabilizes between 90 and 150, all degrees of freedom constraints are sufficient, and the degradation direction set remains empty. Step S3 only executes a sub-process to score one pair of existing feature points. Frames 30 to 60 represent the main rotation phase around the mounting axis. As the rotation angle increases, the front coded markers gradually turn to the back, and by frame 45, only about 3 to 4 markers remain visible and are concentrated on one side of the blade cross-section projection. During this phase, the minimum singular value of the information matrix continuously decreases from approximately 65 in frame 30 to approximately 10.9 in frame 50. The minimum singular value drops below the observability threshold in frame 35, and the degradation direction set becomes non-empty for the first time. Step S3 triggers sub-process two to compensate for candidate points in the region opposite the blade profile. In actual verification, the inventors observed that around frame 45, the projected width of the blade cross-section decreased to approximately 20% of its initial width. At this point, the tracking error relying solely on frontal feature points exhibited a significant jump, and the compensation mechanism in step S3 played a crucial stabilizing role during this stage. Around frame 50, as the rotation angle further increased, the blade cross-section approached a side-view shape, and the lateral baseline of the visible feature points became extremely short. The constraints of the monocular camera along the camera's optical axis (depth direction) weakened synchronously, and the singular values ​​corresponding to the depth direction also dropped below the threshold, thus forming a second degradation direction. Rotational constraint degradation and depth constraint degradation coexisted, corresponding to a degradation degree of freedom direction of 2 in frame 50 of Table 2. Frames 61 to 90 represent the swing positioning stage. The blade rotation ended, and the frontal feature points gradually became visible again. The minimum singular value rebounded to above 88 around frame 65, the set of degradation directions returned to empty, and the tracking accuracy returned to the level of the translation stage. The entire time series of the minimum singular value of the information matrix exhibits a clear three-stage structure of "stable - phased collapse - recovery", which directly corresponds to the physical laws of blade rotation.

[0059] The number and confidence of features in the five frames (5, 20, 35, 50, and 65) corresponding to the translational stationary segment, degradation segment, and recovery period in the aforementioned tracking sequence were compared and statistically analyzed. The number of feature points of the three types, the number of initial correspondences, and the average confidence of descriptors are shown in Table 1.

[0060] Table 1 shows the feature point extraction and correspondence establishment records for five representative frames in the tracking sequence.

[0061] Note: Frames 35 and 50 correspond to the stage where blade rotation causes feature degradation, and the initial number of corresponding elements and the mean confidence of descriptors are significantly lower than those of normal frames; in frame 65, the blade attitude has basically recovered to the orientation, and the feature distribution has returned to normal levels.

[0062] Using the same 5 frames as above, the minimum singular value of the information matrix, the number of degenerate degrees of freedom directions, the size of the set after candidate point compensation, and the output of translation and rotation uncertainty measures are summarized side by side to compare the observability evolution and solution quality. The results are shown in Table 2.

[0063] Table 2 shows the observability analysis and quality assessment results for the five representative frames mentioned above.

[0064] Note: The degradation direction count in frame 35 is 1. After compensation in step S3, the set increases from 40 to 51, adding 11 candidate points for degradation severity scoring. With basic compensation amount Within the corresponding dynamic upper limit; the number of degradation directions in frame 50 is 2, and compensation is performed on both degradation directions, resulting in a total increase of 16 candidate points (from 26 to 42). The compensation points mainly come from the opposite side of the blade profile and the area with significant changes in blade edge depth. The pose uncertainty of each frame is consistent with the degree of observable degradation. The uncertainty of the degraded frames is higher than that of the normal frames, but under the effect of the compensation mechanism in step S3, the actual uncertainty of the degraded frames is significantly lower than the estimated level when no compensation is performed. The time-series trajectory does not show any interruption or large jump during the degradation stage. The quality score of frames 35 to 50 in the accuracy analysis report drops slightly and then gradually recovers as the blade attitude recovers.

[0065] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. A data analysis method for six degrees of freedom visual tracking measurements based on feature points, characterized in that, Includes the following steps: S1. Extract multiple types of feature points from the current frame image, match them with the 3D model feature database to establish an initial feature correspondence, and assign descriptor confidence to each corresponding point to obtain the initial feature correspondence set; S2, constructs a descriptor confidence weighted pose observation information matrix from the initial feature correspondence set, quantifies the constraint strength of each degree of freedom through singular value decomposition, identifies the degradation direction and calculates the contribution of each feature point component; S3, track the reliability prediction network score of the feature points in the initial feature correspondence set; when there is a degradation direction, add and screen candidate points in the high value area of ​​the comprehensive degradation sensitivity map to form an enhanced feature correspondence set with reliability confidence. S4 combines the descriptor confidence and reliability confidence in the enhanced feature correspondence set, and jointly weights them according to the contribution of the degradation direction component, to solve the six-degree-of-freedom pose of the current frame and quantify the estimation uncertainty of each degree of freedom; S5 uses the pose estimation and the uncertainty of each component of the six degrees of freedom as observations to perform extended Kalman filter time series consistency analysis, and updates the feature point reliability history database to generate a six-degree-of-freedom tracking trajectory and accuracy analysis report with quality score.

2. The data analysis method of feature point based six degree of freedom visual tracking measurement according to claim 1, characterized in that, In S1, the multiple types of feature points include corner features, coded marker features, and texture features; Corner features are extracted based on image gradient structure tensor analysis and then ORB descriptors are calculated after nonmaximum suppression filtering; texture features are extracted by the FAST detector in regions where the average gradient magnitude exceeds a preset texture response threshold and BRIEF descriptors are calculated. The coded marker features are extracted by adaptive threshold binarization and contour detection to extract candidate contours. Subpixel-level image coordinates are determined by ellipse fitting. After parsing the unique number of the coded region, a three-dimensional correspondence is directly established with the three-dimensional model feature database. The initial value of the descriptor confidence is set to the highest preset level of the three types of features. When the ellipse fitting residual or the contrast of the coded region is lower than the first quality threshold, a weight reduction correction is performed. Corner-type and texture-type features are indexed by KD-tree to perform nearest neighbor retrieval on the 3D model feature database. Matching pairs whose ratio of nearest neighbor to second nearest neighbor descriptor distance is lower than a preset matching discriminative threshold are determined to be valid, and the confidence of the descriptor is calculated by subtracting this ratio from the value of the nearest neighbor.

3. The data analysis method for six-degree-of-freedom visual tracking measurement based on feature points according to claim 1, characterized in that, In S2, for each feature point in the initial feature corresponding set, the partial derivative of the six-degree-of-freedom pose parameters is obtained with the current initial pose as the reference to obtain the pose Jacobian matrix; the Jacobian matrix contributions of all feature points are accumulated with the descriptor confidence as the weight to construct a six-degree-of-freedom weighted pose observation information matrix. Singular value decomposition is performed on the weighted pose observation information matrix to obtain six singular values ​​and their corresponding right singular vectors arranged in descending order. Each singular value quantifies the constraint strength of the current feature set on the corresponding direction. The introduction of descriptor confidence weights compresses the contribution of feature points with low matching reliability to the information matrix, making the information matrix more realistically reflect the distribution of effective constraints in the current frame.

4. The data analysis method for six-degree-of-freedom visual tracking measurement based on feature points according to claim 3, characterized in that, In S2, when the singular value is lower than the preset observability threshold, the corresponding right singular vector direction is identified as the degradation direction and included in the degradation direction set. The degradation severity score is defined by the ratio of the preset observability threshold to the singular value. The larger the ratio, the more severe the degradation. For each degradation direction in the set of degradation directions, the pose Jacobian matrix of each feature point in the set of initial features is multiplied by the right singular vector corresponding to the degradation direction. The magnitude of the product vector is taken as the component contribution of the feature point to the corresponding degradation direction. The larger the component contribution, the higher the constraint value of the feature point for the corresponding degradation direction of correction.

5. The data analysis method for six-degree-of-freedom visual tracking measurement based on feature points according to claim 1, characterized in that, In S3, the tracking reliability prediction network is a three-branch fusion network; The first branch processes local image patches centered on candidate points, extracts local texture quality and occlusion signal features through multiple cascaded convolutional layers, and flattens them to form visual feature vectors. The second branch processes the recent motion history sequence of the feature point. The input is the frame-by-frame optical flow displacement sequence of the feature point in the history frames of a preset length. After being encoded by a double-layer gated cyclic unit, the output is a motion state feature vector. The third branch takes the concatenation of the component contribution vectors in each of the six degrees of freedom and the mean of the local gradient magnitude as input, and then fuses them through a fully connected layer to output the geometric constraint feature vector. When the set of degenerate directions is empty, the component contribution vectors are all zero. The three-branch outputs are spliced ​​together and processed by a fully connected layer, resulting in a reliability confidence level between zero and one.

6. The data analysis method for six-degree-of-freedom visual tracking measurement based on feature points according to claim 5, characterized in that, The sub-process of S3 performs a tracking reliability prediction network score on all feature points in the initial feature correspondence set, which is unconditionally triggered in any frame. For feature points that have been accumulated and tracked historically, the actual optical flow displacement sequence of their most recent preset length frames is directly taken as the input of the second branch; For feature points that appear for the first time in the current frame and have not yet accumulated a tracking history, the reverse optical flow tracing method is used to locate and predict their positions frame by frame in the previous frames to construct a motion history sequence. Feature points whose reliability confidence is lower than a preset reliability screening threshold are removed from the set, and the retained feature points are passed to step S4 with their reliability confidence.

7. The data analysis method for six-degree-of-freedom visual tracking measurement based on feature points according to claim 5, characterized in that, The second sub-process of S3 is triggered only when the set of degeneration directions is not empty; In the image, the contribution estimate of each location to the degradation direction is calculated using uniform grid points. The contribution estimates of each degradation direction are superimposed with the degradation severity score as a weight to form a comprehensive degradation sensitivity map. In the region of the comprehensive degradation sensitivity map that exceeds the preset candidate detection threshold, the FAST detector is densely run to extract candidate feature points, and the three-dimensional model feature database is queried to establish a three-dimensional correspondence. The newly detected candidate feature points are traced frame by frame in reverse using the Lucas-Kanade optical flow algorithm to construct a motion history sequence. The feature point reliability history database is queried, and candidate points whose historical reliability average value of the image region is lower than the preset historical pre-screening threshold are directly excluded. After the pre-screened candidate points are scored by the tracking reliability prediction network, they are sorted from high to low according to the product of reliability confidence and component contribution. Within the dynamic compensation upper limit of each degradation direction, the best candidates are selected and included in the set corresponding to the enhanced features.

8. The data analysis method for six-degree-of-freedom visual tracking measurement based on feature points according to claim 1, characterized in that, In S4, the participating weights are constructed by weighting and summing the three components: sub-confidence, normalized degradation contribution, and reliability confidence. The normalized degradation contribution is obtained by summing the product of the component contribution of each feature point to each degradation direction in the degradation direction set and the corresponding degradation severity score, and then dividing by the maximum value of the corresponding values ​​of all feature points in the set. When the degradation direction set is empty, the normalized degradation contribution is zero, and the participation in the weighted degradation is a binary weighted sum of the descriptor confidence and the reliability confidence. The balance coefficients of the three components satisfy the normalization constraint and are calibrated by minimizing the pose estimation error on a representative tracking dataset.

9. The data analysis method for six-degree-of-freedom visual tracking measurement based on feature points according to claim 1, characterized in that, In S4, the weighted pose calculation is divided into two stages: initialization and iterative refinement. In the initialization phase, the weighted EPnP method is adopted, which represents the three-dimensional coordinates of the feature points in the enhanced feature set as a weighted linear combination of virtual control points. The corresponding rows of the equation system are weighted by the weights of each feature point, and the camera coordinates of the control points are recovered by singular value decomposition to obtain the initial rotation matrix and translation vector. In the iterative refinement stage, Levenberg-Marquardt optimization is performed based on the initial estimate. The Huber function is applied to the reprojection error to suppress the influence of outliers. The result is multiplied by the participation weights to form a weighted objective function. The process stops when the change in the objective function is lower than the preset convergence threshold or when the upper limit of the number of iterations is reached. After the pose optimization converges, the weighted pose information matrix is ​​reconstructed at the converged pose using the participation weights and then inverted. The positive square root of each diagonal element is used as the estimated uncertainty quantification value of each component of the six degrees of freedom.

10. The data analysis method for six-degree-of-freedom visual tracking measurement based on feature points according to claim 1, characterized in that, In S5, the extended Kalman filter state vector includes the workpiece's position in the camera coordinate system, rotational attitude represented by quaternions, linear velocity, and angular velocity, with uniform motion as the prediction model; the uncertainty of each component of the six degrees of freedom directly participates in the formation of the innovation covariance matrix, and the component with larger uncertainty allows for greater innovation deviation; The innovation vector is composed of translational innovation and rotational innovation; the translational innovation is obtained by the difference between the observed translation vector and the predicted translation vector, and the rotational innovation is obtained by converting the relative rotation between the observed rotation matrix and the predicted rotation matrix into a rotation vector using the Rodrigues formula; The Mahalanobis distance of the innovation vector is calculated under the innovation covariance matrix to quantify the consistency between the current frame and the motion model. When the Mahalanobis distance exceeds the preset anomaly detection threshold, the current frame is marked as an anomaly frame, and the filter prediction value is used instead of the measurement value for time-series output. The comprehensive quality score of each degree of freedom component is obtained by weighting and fusing the ratio of the uncertainty of the current frame to the mean uncertainty of the same component within the historical sliding window and the normalized amplitude of the Kalman innovation vector in the corresponding component direction. The score is then averaged through a sliding window of a preset length to form a smooth quality score time series. The feature point reliability history database updates the reliability statistics of the corresponding image region by feeding back the reprojection error using an exponential moving average, providing historical priors for the pre-screening of candidate points in the next frame.

Images