Image target intelligent tracking positioning method and system under compass position constraint

Abstract

The application provides an image target intelligent tracking positioning method and system under Beidou position constraint, relates to the Beidou positioning technical field, and comprises the following steps: acquiring multi-source heterogeneous data streams, constructing a multi-modal representation model, and establishing a coordinate system dynamic projection transformation relationship; generating a search mask to limit the tracking range by using Beidou position information; performing Bayesian correction on the tracking result by using double-domain consistency measurement; and dynamically updating the projection transformation parameters and feature weights based on residual distribution.The application realizes the deep fusion of vision and Beidou information, and improves the accuracy and robustness of target tracking in a complex scene.

Description

Technical Field

[0001] This invention relates to BeiDou positioning technology, and more particularly to an intelligent tracking and positioning method and system for image targets under BeiDou position constraints. Background Technology

[0002] With the rapid development of computer vision and positioning navigation technologies, target tracking and positioning have become key technologies in fields such as intelligent monitoring, autonomous driving, and emergency rescue. Traditional image target tracking mainly relies on visual feature extraction and pattern matching algorithms to continuously track targets by analyzing their appearance, shape, color, and other features. Meanwhile, the BeiDou Navigation Satellite System, as my country's independently developed global satellite positioning system, can provide high-precision position, velocity, and time services.

[0003] In target tracking and localization applications, single visual tracking technologies are easily affected by factors such as target occlusion, changes in lighting, and changes in viewing angle, leading to tracking failure or drift. Relying solely on BeiDou positioning technology also suffers from signal blockage and multipath effects in complex environments, making it difficult to meet the requirements for high-precision and high-reliability tracking. Therefore, effectively integrating visual tracking and BeiDou positioning technologies to construct a target tracking and localization method based on multi-source heterogeneous data fusion has become an important research direction for solving these problems.

[0004] Existing technologies lack an effective dynamic projection transformation mechanism between image coordinate systems and geographic coordinate systems, making it difficult to accurately handle the nonlinear changes in the two coordinate systems in practical applications, resulting in insufficient accuracy in cross-domain information fusion.

[0005] Traditional methods for fusing BeiDou positioning information with visual tracking results typically employ simple weighted averaging or Kalman filtering, which cannot effectively model the probabilistic relationship between the BeiDou positioning accuracy factor and the target motion state, and are difficult to adapt to uncertainties in complex scenarios.

[0006] Existing technologies lack adaptive learning mechanisms and cannot dynamically adjust the weight allocation and feature representation strategies of multi-source heterogeneous data according to the actual tracking effect. This results in insufficient robustness of the system during long-term tracking and makes it difficult to cope with complex and ever-changing environmental conditions. Summary of the Invention

[0007] This invention provides an intelligent image target tracking and positioning method and system under BeiDou position constraints, which can solve the problems in the prior art.

[0008] A first aspect of this invention provides an intelligent image target tracking and positioning method under BeiDou position constraints, comprising:

[0009] Acquire multi-source heterogeneous data streams of the target to be tracked, construct a multi-modal representation model of the target that integrates visual semantic features and spatial topological features, and establish a dynamic projection transformation relationship between the image coordinate system and the geographic coordinate system based on the multi-source heterogeneous data streams;

[0010] Based on the dynamic projection transformation relationship, the BeiDou position information is back-projected into the image space. A search region mask is generated by probabilistic modeling based on the covariance propagation of the BeiDou positioning accuracy factor and the target motion state. Within the image range defined by the search region mask, the target multimodal representation model is applied to extract target candidates and match features to obtain preliminary tracking results and their corresponding matching confidence.

[0011] The preliminary tracking results are mapped to the geographic coordinate system through the dynamic projection transformation relationship. The consistency between the results and the BeiDou position prediction is measured in terms of both spatial distance and movement trend. Based on the measurement results, the matching confidence is corrected using Bayesian methods to obtain the final tracking and positioning result that integrates dual-domain information.

[0012] Based on the residual distribution of the final tracking and positioning results in the image domain and the geographic domain, the error compensation parameters in the projection transformation relationship and the feature weight allocation strategy in the target multimodal representation model are dynamically updated to achieve closed-loop adaptive optimization.

[0013] Constructing a target multimodal representation model that integrates visual semantic features and spatial topological features, and establishing a dynamic projection transformation relationship between the image coordinate system and the geographic coordinate system based on the multi-source heterogeneous data stream, includes:

[0014] Multi-scale sliding window convolution operations are applied to a continuous sequence of image frames to analyze the local texture features and global contour features of the target, forming a set of visual semantic features that describe the appearance of the target.

[0015] The geometric center position of the target in the image plane is determined by the set of visual semantic features. The relative distance vector and angle distribution between neighboring targets are constructed based on the geometric center position. The topological feature map reflecting the spatial relationship between the targets is generated by the relative distance vector and the angle distribution.

[0016] The visual semantic feature set and the topological feature map are mapped to a unified semantic space through nonlinear dimensionality reduction to generate a fused feature representation. A weight learning strategy is used to model the importance of each modality feature in the fused feature representation. The modeling results are used to optimize the fused feature representation to obtain the target multimodal representation model.

[0017] The real-time three-dimensional coordinates and attitude angle parameters in the carrier position and attitude information are analyzed. The real-time three-dimensional coordinates, attitude angle parameters and camera intrinsic and extrinsic parameter calibration information are subjected to matrix decomposition operation to obtain the initial projection transformation matrix at the current moment. At the same time, feature projection calibration is performed based on the target multimodal representation model.

[0018] The parameter drift of the initial projection transformation matrix within a continuous time window is established, and the initial projection transformation matrix is ​​adaptively corrected based on the parameter drift and the rate of change of the carrier position and attitude information to form a dynamic projection transformation relationship.

[0019] The visual semantic feature set and the topological feature map are mapped to a unified semantic space through nonlinear dimensionality reduction to generate a fused feature representation. A weight learning strategy is then used to model the importance of each modality feature in the fused feature representation, including:

[0020] A feature space transformation matrix is ​​constructed based on the difference in feature dimensions between the visual semantic feature set and the topological feature map. Nonlinear dimensionality reduction mapping is performed through the feature space transformation matrix to transform the visual semantic feature set and the topological feature map to a unified semantic space of a preset dimension, thereby generating visual semantic mapping features and topological structure mapping features.

[0021] Feature integration operations are performed using the similarity distribution of the visual semantic mapping features and the topological structure mapping features in a unified semantic space to form a fused feature representation; a feature weight evaluation strategy is constructed based on the response intensity differences of each modality feature in the fused feature representation, and visual weight coefficients and topological weight coefficients are generated through the feature weight evaluation strategy;

[0022] The visual semantic mapping features are optimized using the visual weight coefficients, and the topological structure mapping features are optimized using the topological weight coefficients. The optimized features are then combined and reconstructed to obtain a fusion feature representation that completes importance modeling.

[0023] Based on the dynamic projection transformation relationship, the BeiDou position information is back-projected into the image space. A probabilistic model is then performed based on the covariance propagation of the BeiDou positioning accuracy factor and the target motion state to generate a search region mask, including:

[0024] Nonlinear projection calculations are performed on the position coordinates of the target in the geographic coordinate system based on the BeiDou position information. The coordinates of the projection center point in the image space are constructed by the inverse transformation matrix of the dynamic projection transformation relationship. At the same time, the uncertainty distribution of the position coordinates in the geographic coordinate system is established based on the positioning accuracy factor in the BeiDou position information.

[0025] The uncertainty distribution is reconstructed into the image space using the inverse transformation matrix to form the position error ellipse corresponding to the coordinates of the projection center point. Optimization calculations are then applied to the velocity vector and acceleration vector in the target motion state to generate the covariance matrix of the velocity vector and the acceleration vector.

[0026] The uncertainty of the target's current state is quantitatively evaluated based on the covariance matrix. The evaluation result of the uncertainty of the state is then probabilistically fused with the position error ellipse to construct the probability distribution region of the target's appearance in the image space.

[0027] The set of confidence pixels in the probability distribution region is filtered according to a preset probability threshold, and the set of confidence pixels is reconstructed into a search region mask with boundary smoothness through spatial neighborhood constraints.

[0028] Nonlinear projection calculations are performed on the target's position coordinates in the geographic coordinate system based on BeiDou positioning information. The coordinates of the projection center point in the image space are constructed through the inverse transformation matrix of the dynamic projection transformation relationship. Simultaneously, the uncertainty distribution of the position coordinates in the geographic coordinate system is established based on the positioning accuracy factor in BeiDou positioning information, including:

[0029] The position coordinates of the target in the geographic coordinate system in the BeiDou position information are calculated and the inverse transformation matrix corresponding to the dynamic projection transformation relationship is constructed; based on the position coordinates and the inverse transformation matrix, a nonlinear projection space transformation is performed to generate the mapping relationship of the position coordinates from the geographic coordinate system to the image coordinate system;

[0030] Based on the mapping relationship, a spatial projection transformation operation is performed on the position coordinates to obtain the coordinates of the projection center point of the target in the image space; the positioning accuracy factor in the BeiDou position information is quantitatively analyzed, wherein the positioning accuracy factor characterizes the measurement error amplitude characteristics of the position coordinates;

[0031] The positioning accuracy factor is used to describe the measurement error amplitude characteristics, and the uncertainty distribution of the location coordinates is generated in the geographic coordinate system.

[0032] The preliminary tracking results are mapped to the geographic coordinate system through the dynamic projection transformation relationship. A dual consistency measurement of spatial distance and motion trend is performed between the results and the BeiDou position prediction. Based on the measurement results, the matching confidence is corrected using Bayesian methods to obtain the final tracking and positioning results fused from the dual-domain information, including:

[0033] The position coordinates and velocity vector of the target in the image space are optimized and analyzed in the preliminary tracking results, and a forward transformation matrix corresponding to the dynamic projection transformation relationship is constructed.

[0034] By using the location coordinates and the velocity vector to perform spatial projection transformation through the forward transformation matrix, the mapping position and mapping velocity of the preliminary tracking results in the geographic coordinate system are established.

[0035] Based on BeiDou position prediction, a predicted position and predicted velocity are generated in a geographic coordinate system. A spatial Euclidean distance metric is constructed based on the mapped position and the predicted position. At the same time, a motion trend metric is generated based on the difference between the directional angle and the velocity magnitude using the mapped velocity and the predicted velocity.

[0036] By adaptively combining the spatial distance metric and the motion trend metric, a dual consistency metric is constructed. The dual consistency metric is then used as input to update the matching confidence of the preliminary tracking results using Bayesian posterior probability.

[0037] Based on the matching confidence level and the corresponding image spatial location of the preliminary tracking result, an optimized selection is made, and the final tracking and positioning result fused with dual-domain information is reconstructed according to the magnitude of the matching confidence level.

[0038] Based on the residual distribution of the final tracking and positioning results in the image domain and the geographic domain, the error compensation parameters in the projection transformation relationship and the feature weight allocation strategy in the target multimodal representation model are dynamically updated as follows:

[0039] A dual-domain accuracy evaluation is performed on the final tracking and positioning results, and the positioning error distribution in the image space and the position residual distribution in the geographic space of the final tracking and positioning results are constructed.

[0040] A spatial compensation vector is established based on the positioning error distribution. The transformation parameters in the projection transformation relationship are adaptively adjusted using the spatial compensation vector to obtain the error compensation parameters.

[0041] A spatial similarity index is constructed based on the location residual distribution. The spatial similarity index is then used to evaluate the importance of feature components in the target multimodal representation model, and a feature weight allocation strategy is generated.

[0042] The error compensation parameters are applied to the projection transformation relationship, and the feature weight allocation strategy is integrated into the target multimodal representation model to achieve dynamic iterative optimization based on dual-domain residuals.

[0043] A second aspect of this invention provides an intelligent image target tracking and positioning system under BeiDou position constraints, comprising:

[0044] Module 1 is used to acquire multi-source heterogeneous data streams of the target to be tracked, construct a multi-modal representation model of the target that integrates visual semantic features and spatial topological features, and establish a dynamic projection transformation relationship between the image coordinate system and the geographic coordinate system based on the multi-source heterogeneous data streams.

[0045] Module 2 is used to back-project the BeiDou position information into the image space based on the dynamic projection transformation relationship, and generate a search region mask by probabilistic modeling based on the covariance propagation of the BeiDou positioning accuracy factor and the target motion state; within the image range defined by the search region mask, the target multimodal representation model is applied to extract target candidates and match features to obtain preliminary tracking results and their corresponding matching confidence.

[0046] Module 3 is used to map the preliminary tracking results to the geographic coordinate system through the dynamic projection transformation relationship, perform a dual consistency measurement of spatial distance and motion trend with the BeiDou position prediction, and perform Bayesian correction on the matching confidence based on the measurement results to obtain the final tracking and positioning result that integrates dual-domain information.

[0047] Module 4 is used to dynamically update the error compensation parameters in the projection transformation relationship and the feature weight allocation strategy in the target multimodal representation model based on the residual distribution of the final tracking and positioning results in the image domain and the geographic domain, so as to achieve closed-loop adaptive optimization.

[0048] A third aspect of the present invention provides an electronic device, comprising:

[0049] processor;

[0050] Memory used to store processor-executable instructions;

[0051] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0052] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0053] The beneficial effects of this application are as follows:

[0054] By constructing a multimodal target representation model that integrates visual semantic features and spatial topological features, the ability to recognize targets in complex scenes is improved, and the richness and robustness of feature representation are enhanced. Innovatively, BeiDou position information is back-projected into the image space and combined with accuracy factors and target motion states for probabilistic modeling to generate search region masks. This effectively narrows the target search range, reduces computational complexity, and decreases the probability of background interference and mismatches with similar targets.

[0055] By employing a dual consistency measurement mechanism based on spatial distance and motion trend, deep fusion of image domain and geographic domain information is achieved, and Bayesian correction is applied to the matching confidence, significantly improving the accuracy and reliability of tracking and positioning results.

[0056] Based on the residual distribution of tracking and positioning results in the image domain and geographic domain, a closed-loop adaptive optimization mechanism for projection transformation relationship error compensation and feature weight allocation is established, enabling the system to dynamically adjust according to the actual scenario, effectively cope with complex and ever-changing tracking environments, and improve the adaptability and long-term stability of the method. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating the intelligent image target tracking and positioning method under BeiDou position constraints according to an embodiment of the present invention.

[0058] Figure 2 This is a flowchart of the parameter update and adjustment process for dual-domain residual evaluation optimization in an embodiment of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0061] Figure 1 This is a flowchart illustrating the intelligent image target tracking and positioning method under BeiDou position constraints according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0062] Acquire multi-source heterogeneous data streams of the target to be tracked, construct a multi-modal representation model of the target that integrates visual semantic features and spatial topological features, and establish a dynamic projection transformation relationship between the image coordinate system and the geographic coordinate system based on the multi-source heterogeneous data streams;

[0063] Based on the dynamic projection transformation relationship, the BeiDou position information is back-projected into the image space. A search region mask is generated by probabilistic modeling based on the covariance propagation of the BeiDou positioning accuracy factor and the target motion state. Within the image range defined by the search region mask, the target multimodal representation model is applied to extract target candidates and match features to obtain preliminary tracking results and their corresponding matching confidence.

[0064] The preliminary tracking results are mapped to the geographic coordinate system through the dynamic projection transformation relationship. The consistency between the results and the BeiDou position prediction is measured in terms of both spatial distance and movement trend. Based on the measurement results, the matching confidence is corrected using Bayesian methods to obtain the final tracking and positioning result that integrates dual-domain information.

[0065] Based on the residual distribution of the final tracking and positioning results in the image domain and the geographic domain, the error compensation parameters in the projection transformation relationship and the feature weight allocation strategy in the target multimodal representation model are dynamically updated to achieve closed-loop adaptive optimization.

[0066] In one optional implementation, constructing a target multimodal representation model that integrates visual semantic features and spatial topological features, and establishing a dynamic projection transformation relationship between the image coordinate system and the geographic coordinate system based on the multi-source heterogeneous data stream includes:

[0067] Multi-scale sliding window convolution operations are applied to a continuous sequence of image frames to analyze the local texture features and global contour features of the target, forming a set of visual semantic features that describe the appearance of the target.

[0068] The geometric center position of the target in the image plane is determined by the set of visual semantic features. The relative distance vector and angle distribution between neighboring targets are constructed based on the geometric center position. The topological feature map reflecting the spatial relationship between the targets is generated by the relative distance vector and the angle distribution.

[0069] The visual semantic feature set and the topological feature map are mapped to a unified semantic space through nonlinear dimensionality reduction to generate a fused feature representation. A weight learning strategy is used to model the importance of each modality feature in the fused feature representation. The modeling results are used to optimize the fused feature representation to obtain the target multimodal representation model.

[0070] The real-time three-dimensional coordinates and attitude angle parameters in the carrier position and attitude information are analyzed. The real-time three-dimensional coordinates, attitude angle parameters and camera intrinsic and extrinsic parameter calibration information are subjected to matrix decomposition operation to obtain the initial projection transformation matrix at the current moment. At the same time, feature projection calibration is performed based on the target multimodal representation model.

[0071] The parameter drift of the initial projection transformation matrix within a continuous time window is established, and the initial projection transformation matrix is ​​adaptively corrected based on the parameter drift and the rate of change of the carrier position and attitude information to form a dynamic projection transformation relationship.

[0072] Multi-scale sliding window convolution operations are applied to a continuous sequence of image frames to analyze the target's features. Different sized convolution kernels, such as 3×3, 5×5, and 7×7, are used to perform convolution operations on the images to extract the target's texture details. Simultaneously, larger convolution kernels, such as 15×15 or larger, are designed to capture the overall contour information of the target. For the feature map output from each convolutional layer, max pooling and average pooling operations are further applied to reduce the feature dimensionality and retain salient features. Feature maps from different levels are fused through skip connections to form a feature pyramid containing multi-level information. Finally, these features are integrated into a single D-dimensional vector through a fully connected layer, forming a set of visual semantic features describing the target's appearance.

[0073] The geometric center position of the target in the image plane is determined using a set of visual semantic features. Specifically, the feature set obtained in the previous step is input into the target detection network to obtain the bounding box coordinates (x, y, w, h) of each target, where (x, y) are the coordinates of the top-left corner, and w and h are the width and height, respectively. The geometric center coordinates are calculated as (x+w / 2, y+h / 2). Based on these geometric center positions, a topological relationship graph is constructed. For each detected target i, the relationship between it and all other targets j in the scene is calculated, resulting in a relative distance vector dij and an angle θij. The distance vector is calculated as the Euclidean distance between the center points of two targets, and the angle is the angle between the line connecting them and the horizontal direction. This information is organized into an N×N relationship matrix (N is the number of targets), where each element of the matrix contains distance and angle information. This matrix is ​​further converted into a topological feature map, and a Gaussian kernel function is used to weight the distances to ensure that the relationships between nearby targets are more important.

[0074] The visual semantic feature set and the topological feature map are fused together. A t-SNE nonlinear dimensionality reduction algorithm is used to map the two heterogeneous features to the same D'-dimensional semantic space. Specifically, the visual feature vector and the topological feature vector are first concatenated, and then a nonlinear transformation is performed through a multilayer perceptron to generate preliminary fused features. To optimize the fusion effect, an attention mechanism is designed to model the importance of features from different modalities. The attention weights αv and αt for each modal feature are calculated, satisfying αv + αt = 1. The final fused feature F consists of the weighted sum of the two parts, i.e., F = αv·Fv + αt·Ft, where Fv and Ft are the dimensionality-reduced visual feature and the topological feature, respectively.

[0075] The vehicle's position and attitude information is analyzed to establish a projection relationship between the image coordinate system and the geographic coordinate system. Real-time 3D coordinates (X, Y, Z) and attitude angle parameters (roll, pitch, yaw) are obtained from the sensor. The projection matrix P = K[R|t] is calculated by combining the camera's intrinsic parameter matrix K (containing focal length and principal point coordinates) and extrinsic parameter matrix [R|t] (R is the rotation matrix, t is the translation vector). In practical applications, the camera's intrinsic parameters are obtained through offline calibration using a calibration board, while the extrinsic parameters are updated in real-time based on the vehicle's position and attitude. To improve projection accuracy, feature matching calibration is performed using a target multimodal characterization model. Several reference points with obvious features are selected, and the deviations between their projected positions and actual detection positions are compared. The least squares method is used to optimize the projection matrix parameters.

[0076] A dynamic projection transformation relationship is established, and the changes in projection matrix parameters within a continuous time window (e.g., the last 5 seconds) are recorded, calculating the parameter drift ΔP. Simultaneously, the rate of change v of the carrier's position and attitude information is monitored; when v exceeds a preset threshold, a projection matrix update is triggered. An adaptive correction factor λ = f(v) is designed, which increases with v to adjust the projection matrix update amplitude. The final corrected projection matrix P' = P + λ·ΔP. In practical applications, the time window size and correction factor calculation function can be dynamically adjusted according to scene complexity and carrier motion characteristics to balance computational efficiency and projection accuracy.

[0077] The target multimodal representation method proposed in this embodiment, which integrates visual semantic features and spatial topological features, can effectively combine target appearance information with spatial positional relationship information, thereby improving the accuracy of target recognition and tracking. Simultaneously, the established dynamic projection transformation relationship can adapt to parameter changes caused by carrier motion in complex environments, ensuring the accuracy of image coordinate to geographic coordinate conversion. This method has broad application prospects in fields such as intelligent surveillance, autonomous driving, and augmented reality.

[0078] In one optional implementation, the visual semantic feature set and the topological feature map are mapped to a unified semantic space through nonlinear dimensionality reduction to generate a fused feature representation, and a weight learning strategy is used to model the importance of each modality feature in the fused feature representation, including:

[0079] A feature space transformation matrix is ​​constructed based on the difference in feature dimensions between the visual semantic feature set and the topological feature map. Nonlinear dimensionality reduction mapping is performed through the feature space transformation matrix to transform the visual semantic feature set and the topological feature map to a unified semantic space of a preset dimension, thereby generating visual semantic mapping features and topological structure mapping features.

[0080] Feature integration operations are performed using the similarity distribution of the visual semantic mapping features and the topological structure mapping features in a unified semantic space to form a fused feature representation; a feature weight evaluation strategy is constructed based on the response intensity differences of each modality feature in the fused feature representation, and visual weight coefficients and topological weight coefficients are generated through the feature weight evaluation strategy;

[0081] The visual semantic mapping features are optimized using the visual weight coefficients, and the topological structure mapping features are optimized using the topological weight coefficients. The optimized features are then combined and reconstructed to obtain a fusion feature representation that completes importance modeling.

[0082] When constructing a feature space transformation matrix based on the difference in feature dimensions between a visual semantic feature set and a topological feature map, it is necessary to first obtain the dimensionality information of the two feature representations. The visual semantic feature set is typically extracted using a deep convolutional neural network, with feature dimensions ranging from 1024 to 4096, containing semantic representations at multiple scales. The topological feature map is stored in a graph structure, with node feature dimensions ranging from 64 to 256 and edge feature dimensions ranging from 32 to 128. By calculating the total dimensionality difference between the two features, the parameter configuration requirements of the transformation matrix are determined.

[0083] The feature space transformation matrix adopts a separate design, comprising a visual feature transformation module and a topological feature transformation module. The visual feature transformation module uses a multi-layer fully connected network structure, with 2048 neurons in the first layer, 1024 neurons in the intermediate layers, and an output layer with a pre-defined unified semantic space dimension. The topological feature transformation module first converts the graph-structured data into vector form, then uses a graph convolutional network for feature aggregation. The network contains three graph convolutional layers, each with 512 hidden units. The weight parameters of the transformation matrix are set using the Xavier initialization method to ensure the stability of the feature variance during forward propagation.

[0084] During the nonlinear dimensionality reduction mapping process, the visual semantic feature set is input to the visual feature transformation module, and after batch normalization, it enters the fully connected layer. The GELU activation function is used, which has better gradient propagation characteristics than the ReLU function. A dropout layer is added after each fully connected layer, with a dropout ratio set to 0.3 to prevent overfitting. The processing of the topological feature map requires first aggregating neighbor node information through a graph attention mechanism, with 8 attention heads, each with a dimension of 64.

[0085] The default unified semantic space dimension is set to 512, a choice that balances computational complexity and feature representation capability. After passing through their respective transformation networks, both types of features output a unified 512-dimensional dimension. The visual semantic mapping feature preserves the spatial structure information of the original visual features, maintaining spatial relevance through a positional encoding mechanism. The topological structure mapping feature maintains the topological relationships between nodes in the graph, constrained by adjacency matrix information.

[0086] When performing feature integration operations using the similarity distribution of visual semantic mapping features and topological structure mapping features in a unified semantic space, the cosine similarity metric is used for similarity calculation. The two mapping features are used as input vectors, and the similarity value is obtained by dividing the vector dot product by the product of the vector magnitudes. The similarity calculation result ranges from -1 to 1, with values ​​closer to 1 indicating greater similarity between the two features. To obtain statistical information on the similarity distribution, the similarity value range is divided into 20 equally spaced intervals, and the number of samples within each interval is counted.

[0087] Feature fusion is performed using a weighted fusion method based on similarity distribution results, with the weight allocation strategy dynamically adjusted according to similarity levels. Dimensions with higher similarity are assigned larger fusion weights, while dimensions with lower similarity are assigned smaller weights. The fusion process employs an attention mechanism, using the softmax function for normalization when calculating attention weights. The fused feature representation is generated through a weighted summation, where the value of each feature dimension is the weighted sum of the corresponding dimensions of the two mapped features.

[0088] When constructing a feature weight evaluation strategy based on the differences in response intensity among modal features in the fused feature representation, the response intensity is obtained through a statistical measure of feature activation values. For the visual modality, the response intensity is calculated based on the L2 norm of the feature vector, reflecting the overall intensity of feature activation. For the topological modality, the response intensity combines node degree and feature magnitude, reflecting the degree of influence of the graph structure. The difference in response intensity between the two modalities is represented by a normalized numerical difference; a larger difference value indicates a more significant difference in the contribution levels of the two modalities.

[0089] The feature weight evaluation strategy employs an adaptive gating mechanism, dynamically adjusting weight allocation based on differences in response intensity. The gating unit contains a sigmoid activation function with an output ranging from 0 to 1, serving as a weight modulation factor. Gating parameters are learned through a multilayer perceptron, which consists of two hidden layers: the first layer has 256 neurons, and the second layer has 128. The weight evaluation process considers historical information, using an exponential moving average method to update the weight coefficients, with a smoothing factor set to 0.95.

[0090] When generating visual and topological weight coefficients, ensure that the sum of the two coefficients is 1 to satisfy the normalization constraint. The value range of the visual weight coefficient is limited to 0.2 to 0.8 to avoid information loss due to excessively small weights for any particular modality. The topological weight coefficient is automatically determined as the complement of the visual weight coefficient. The weight coefficients are updated every 10 training batches to maintain the stability of weight adjustments.

[0091] When optimizing visual semantic mapping features using visual weight coefficients, the optimization operation is achieved through element-wise multiplication, where each feature dimension is multiplied by its corresponding weight coefficient. Residual connections are introduced during the optimization process, proportionally adding the weighted features to the original features while retaining some original information. The optimization of topological structure mapping features using topological weight coefficients employs the same mechanism to ensure consistency between the two modalities. The optimized features are then subjected to layer normalization to stabilize the feature distribution.

[0092] The optimized feature combination and reconstruction employs a multi-level fusion strategy, first performing feature dimensionality-level fusion, and then semantic-level fusion. Dimensionality-level fusion is achieved through feature concatenation, connecting the optimized visual and topological features along the feature dimension to form a 1024-dimensional combined feature vector. Semantic-level fusion is achieved through a cross-attention mechanism, enabling the two types of features to mutually influence and interact. Cross-attention comprises three components: query, key-value, and numerical value. The query originates from the visual features, and the key-value pair originates from the topological features.

[0093] The fused feature representation used for importance modeling integrates visual semantic information and topological structure information, with the feature dimension varying between 512 and 1024 depending on the fusion strategy. The numerical stability of the fused features is guaranteed through gradient clipping and weight regularization, with the gradient clipping threshold set to 1.0 and the weight regularization coefficient set to 0.01. Feature quality is evaluated using intra-class tightness and inter-class separation indices; a value greater than 0.8 indicates that the feature quality meets the requirements.

[0094] In image-based target tracking applications, the input visual semantic features have a dimension of 2048, derived from the last layer feature map of the ResNet network. The topological feature map contains 300 nodes and 1500 edges, representing the spatial adjacency relationships of the target region. After feature space transformation, both features are converted to a 512-dimensional representation. Similarity calculations show an average similarity of 0.72, a visual weight coefficient of 0.65, and a topological weight coefficient of 0.35. The final fused features are combined using a concatenation method, resulting in an output dimension of 1024. The feature values ​​are evenly distributed, providing an effective multimodal representation for subsequent target localization.

[0095] In one optional implementation, the BeiDou position information is back-projected into the image space based on the dynamic projection transformation relationship, and a search region mask is generated by probabilistic modeling based on the covariance propagation of the BeiDou positioning accuracy factor and the target motion state, including:

[0096] Nonlinear projection calculations are performed on the position coordinates of the target in the geographic coordinate system based on the BeiDou position information. The coordinates of the projection center point in the image space are constructed by the inverse transformation matrix of the dynamic projection transformation relationship. At the same time, the uncertainty distribution of the position coordinates in the geographic coordinate system is established based on the positioning accuracy factor in the BeiDou position information.

[0097] The uncertainty distribution is reconstructed into the image space using the inverse transformation matrix to form the position error ellipse corresponding to the coordinates of the projection center point. Optimization calculations are then applied to the velocity vector and acceleration vector in the target motion state to generate the covariance matrix of the velocity vector and the acceleration vector.

[0098] The uncertainty of the target's current state is quantitatively evaluated based on the covariance matrix. The evaluation result of the uncertainty of the state is then probabilistically fused with the position error ellipse to construct the probability distribution region of the target's appearance in the image space.

[0099] The set of confidence pixels in the probability distribution region is filtered according to a preset probability threshold, and the set of confidence pixels is reconstructed into a search region mask with boundary smoothness through spatial neighborhood constraints.

[0100] This involves performing nonlinear projection calculations on the target's position coordinates in the geographic coordinate system based on BeiDou positioning information. Assuming the target's geographic coordinates provided by the BeiDou system are (longitude, latitude, altitude), the inverse transformation matrix M is based on the dynamic projection transformation relationship. -1 The geographic coordinates are calculated and transformed into the projection center point coordinates (u0, v0) in image space. This inverse transformation matrix consists of the camera intrinsic parameter matrix K, the rotation matrix R, and the translation vector T, forming a mapping relationship from geographic coordinates to image coordinates. Simultaneously, based on the horizontal positioning accuracy factor HDOP and the vertical positioning accuracy factor VDOP from the BeiDou positioning information, the uncertainty distribution of the position coordinates in the geographic coordinate system is established. Typically, this uncertainty distribution can be represented as a three-dimensional Gaussian distribution with a covariance matrix Σ. geo It is proportional to HDOP and VDOP.

[0101] Using the inverse transformation matrix M -1 The uncertainty distribution is reconstructed into image space, and the error propagation during coordinate transformation is calculated using the Jacobian matrix J. The covariance matrix Σ in the geographic coordinate system is then used. geo Convert to covariance matrix Σ in image coordinate systemimg Based on this covariance matrix, a position error ellipse centered at the projection center point (u0, v0) is formed in the image space. Furthermore, optimization calculations are applied to the velocity vector v and acceleration vector a in the target's motion state. Using methods such as Kalman filtering or particle filtering, combined with historical observation data, the covariance matrix Σ of the velocity and acceleration vectors is generated. motion This process takes into account the dynamic changes in the target's motion pattern and the influence of environmental factors, thus improving the accuracy of state estimation.

[0102] Based on the covariance matrix Σ motion The uncertainty of the target's current state is quantitatively assessed by calculating the Mahalanobis distance or information entropy of the state estimate to evaluate the degree of uncertainty. The assessment result of the state uncertainty is then probabilistically fused with the position error ellipse using a Bayesian fusion framework to obtain the probability distribution P of the position error. position With the probability distribution P of the motion state motion By combining these methods, a joint probability distribution region P of target occurrence in image space is constructed. joint This distribution area reflects the probability of the target appearing at each pixel position in the image after comprehensively considering positioning errors and motion states.

[0103] The set of confidence pixels S in the probability distribution region is filtered according to a preset probability threshold τ (usually set to 0.8 or 0.9). For each pixel (u, v) in the image, when P joint When (u, v) > τ, the pixel is included in the confidence pixel set S. To improve the practicality and robustness of the search region, the confidence pixel set S is processed by spatial neighborhood constraints. Morphological operations such as dilation and closing are applied to fill the holes in the set and smooth the boundaries, ultimately generating a search region mask M with smooth boundaries. search In practical applications, this mask can be directly used to guide subsequent target detection and tracking algorithms, significantly improving computational efficiency and detection accuracy.

[0104] In urban traffic monitoring scenarios, surveillance cameras are installed at intersections to track specific vehicles equipped with BeiDou terminals. The BeiDou system provides the vehicle's real-time geographic coordinates and accuracy factor, which are projected onto the monitoring image using the method described above. Combined with the vehicle's motion status (including speed and steering information), a probabilistic search region is generated in the image. Even in heavy traffic, the system can accurately pinpoint the target vehicle's location range, reducing false detection rates and improving tracking efficiency.

[0105] The above technical solution successfully achieved deep integration of BeiDou positioning information and image monitoring systems, overcoming the limitations of single systems and providing efficient and reliable technical support for target detection and tracking in complex environments. The generation process of the search area mask fully considers the uncertainties of positioning errors and motion states, enabling the target detection system to achieve high-precision target locking under limited computing resources.

[0106] In one optional implementation, a nonlinear projection calculation is performed on the target's position coordinates in the geographic coordinate system based on the BeiDou location information. The coordinates of the projection center point in the image space are constructed through the inverse transformation matrix of the dynamic projection transformation relationship. Simultaneously, the uncertainty distribution of the position coordinates in the geographic coordinate system is established based on the positioning accuracy factor in the BeiDou location information, including:

[0107] The position coordinates of the target in the geographic coordinate system in the BeiDou position information are calculated and the inverse transformation matrix corresponding to the dynamic projection transformation relationship is constructed; based on the position coordinates and the inverse transformation matrix, a nonlinear projection space transformation is performed to generate the mapping relationship of the position coordinates from the geographic coordinate system to the image coordinate system;

[0108] Based on the mapping relationship, a spatial projection transformation operation is performed on the position coordinates to obtain the coordinates of the projection center point of the target in the image space; the positioning accuracy factor in the BeiDou position information is quantitatively analyzed, wherein the positioning accuracy factor characterizes the measurement error amplitude characteristics of the position coordinates;

[0109] The positioning accuracy factor is used to describe the measurement error amplitude characteristics, and the uncertainty distribution of the location coordinates is generated in the geographic coordinate system.

[0110] When calculating the position coordinates of a target in the geographic coordinate system from BeiDou positioning information, the calculation process requires establishing a set of BeiDou satellite observation equations. These equations are based on the geometric relationship between pseudorange observations and the satellite and receiver positions. Specifically, the pseudorange observation equals the true distance between the satellite and receiver plus the receiver clock bias, satellite clock bias, ionospheric delay, tropospheric delay, and multipath error. The true distance is calculated using the three-dimensional Euclidean distance formula, which is the square root of the sum of the squares of the differences between the satellite and receiver coordinates in the x, y, and z directions. The receiver clock bias is solved along with the position coordinates as an unknown parameter. The satellite clock bias is calculated using clock bias parameters from broadcast ephemeris data. The ionospheric delay is calculated using the Klobuchar model, which uses eight ionospheric parameters to describe the diurnal variation and geographic distribution of the delay. The tropospheric delay is calculated using the Hopfield model, which calculates the delay based on ground meteorological parameters and satellite elevation angle. Multipath error is identified and suppressed using signal strength and carrier-to-noise ratio (CNR) metrics.

[0111] When constructing the inverse transformation matrix corresponding to the dynamic projection transformation relationship, the establishment of the dynamic projection transformation relationship requires real-time acquisition of carrier attitude information and camera parameters. Carrier attitude includes pitch angle, roll angle, and yaw angle, acquired through an inertial measurement unit (IMU), with an angle measurement accuracy requirement better than 0.1 degrees. Camera parameters include an intrinsic parameter matrix and distortion coefficients. The intrinsic parameter matrix contains the components of the focal length in the x and y directions and the principal point coordinates. The distortion coefficients include the second and fourth order coefficients of radial distortion and two coefficients of tangential distortion. The construction of the dynamic projection transformation matrix adopts a chain transformation method. First, the geographic coordinates are converted to the camera coordinate system, and then converted to image coordinates through perspective projection. The transformation matrix from geographic coordinates to the camera coordinate system includes a rotation matrix and a translation vector. The rotation matrix is ​​calculated from the carrier attitude angles using the Euler angle transformation formula, and the translation vector represents the camera's position in the geographic coordinate system. The perspective projection transformation uses a pinhole camera model to project three-dimensional points onto a two-dimensional image plane. The inverse transformation matrix is ​​calculated by inverting the transformation step by step. First, the inverse transformation of perspective projection is calculated, then the inverse transformation of coordinate system transformation is calculated, and finally the two inverse transformation matrices are multiplied to obtain the total inverse transformation matrix.

[0112] When performing nonlinear projective spatial transformations based on position coordinates and inverse transformation matrices, handling nonlinear characteristics is a key technical aspect. The calculation of the Earth's curvature effect requires consideration of an Earth ellipsoid model, using WGS84 ellipsoid parameters. The calculation of the radius of curvature includes the meridional radius and the radii of curvature. The meridional radius of curvature is equal to the semi-major axis of the ellipsoid multiplied by (1 - eccentricity squared) divided by (1 - eccentricity squared multiplied by the square of the latitude sine) to the power of 1.5. The radii of curvature of the radii of the ellipsoid are equal to the semi-major axis of the ellipsoid divided by (1 - eccentricity squared multiplied by the square of the latitude sine) to the power of 0.5. Atmospheric refraction is modeled using a layered atmospheric model, dividing the atmosphere into multiple altitude layers, each with a different refractive index. The refractive index is directly proportional to atmospheric density, which decreases exponentially with altitude, with a decay constant of approximately 8.4 km. The angle of refraction is calculated using Snell's law; the ratio of the sine of the incident angle to the sine of the refraction angle is equal to the inverse ratio of the refractive index. Geometric distortion correction for perspective projection employs a polynomial model. The radial distortion correction formula is: the corrected coordinates equal to the original coordinates multiplied by (1 plus k1 multiplied by the square of the radial distance plus k2 multiplied by the fourth power of the radial distance), where k1 and k2 are the radial distortion coefficients, and the radial distance is the distance from the image point to the image center. Tangential distortion correction requires two additional parameters, p1 and p2, and the correction formula includes a cross term of the radial distance and the image point coordinates.

[0113] When generating the mapping relationship between location coordinates from the geographic coordinate system to the image coordinate system, accurate establishment of the mapping relationship requires solving the spatiotemporal synchronization problem. Time synchronization employs a timestamp interpolation method, aligning the timestamps of the BeiDou location information with the timestamps of the image frames. The interpolation method is chosen based on the data change rate: linear interpolation is used when location changes are slow, while cubic spline interpolation is used when location changes are rapid. Calculating cubic spline interpolation requires constructing a tridiagonal matrix equation system, with matrix elements determined by the time intervals between adjacent data points. Spatial synchronization needs to consider position changes caused by carrier motion, employing a motion compensation algorithm for correction. Motion compensation is based on the carrier's velocity and angular velocity information; velocity compensation uses a uniform linear motion model, while angular velocity compensation uses a uniform angular velocity rotation model. The mapping relationship is verified using a control point verification method. Control points are selected as easily identifiable feature points in the image, and the mapping error is calculated by measuring the coordinates of the control points in the geographic and image coordinate systems.

[0114] When performing spatial projection transformations on location coordinates based on mapping relationships, the implementation of these transformations requires addressing coordinate system one and numerical stability issues. Coordinate system one includes the selection of the datum for the geographic coordinate system and the determination of the projection method. Datum selection affects coordinate accuracy; the WGS84 datum is suitable for global applications, while local datums are suitable for regional applications. The choice of projection method is based on the survey area and accuracy requirements; planar projection is used for small survey areas, while spherical projection is used for large survey areas. Numerical stability issues mainly occur during matrix operations; when the transformation matrix approaches singularity, numerical instability can result. Stability improvement methods include singular value decomposition (SVD), QR decomposition, and Cholesky decomposition. SVD decomposes a matrix into the product of three matrices, including a singular value matrix; stability is improved by setting a singular value threshold to remove small singular values. QR decomposition decomposes a matrix into the product of an orthogonal matrix and an upper triangular matrix; the condition number of the orthogonal matrix is ​​1, exhibiting good numerical properties. Cholesky decomposition is suitable for positive definite matrices; the decomposed matrix is ​​symmetric and computationally efficient.

[0115] When obtaining the coordinates of the target's projection center point in image space, evaluating and optimizing coordinate accuracy is crucial for ensuring the effectiveness of subsequent processing. Coordinate accuracy is affected by multiple factors, including BeiDou positioning accuracy, vehicle attitude measurement accuracy, camera calibration accuracy, and transformation algorithm accuracy. BeiDou positioning accuracy is quantified using a position accuracy factor; standard single-point positioning accuracy is approximately 3-5 meters, differential positioning accuracy is approximately 1-3 meters, and carrier phase differential positioning accuracy can reach the centimeter level. Vehicle attitude measurement accuracy depends on the performance level of the inertial measurement unit (IMU); consumer-grade IMUs have an angular accuracy of approximately 1-3 degrees, industrial-grade IMUs have an angular accuracy of approximately 0.1-0.5 degrees, and navigation-grade IMUs can achieve angular accuracy below 0.01 degrees. Camera calibration accuracy is evaluated through reprojection error; a calibration error of less than 0.5 pixels is considered good calibration quality. Transform algorithm accuracy is calculated through error propagation analysis, using a first-order Taylor expansion method to propagate input errors to output coordinates. Coordinate optimization employs a multi-sensor fusion method, combining BeiDou positioning, inertial navigation, and visual odometry information to improve coordinate accuracy.

[0116] When quantifying and analyzing the positioning accuracy factor in BeiDou location information, the calculation of the positioning accuracy factor involves complex geometric and statistical analyses. The geometric accuracy factor is calculated based on the satellite geometric distribution matrix, whose elements contain the direction cosines from the satellite to the receiver. The direction cosines are calculated by dividing the coordinate difference between the satellite position and the receiver position by the distance. The geometric distribution matrix is ​​a 4×n matrix, where 4 corresponds to three position parameters and one time parameter, and n is the number of visible satellites. The geometric accuracy factor is equal to the square root of the trace of the transpose of the geometric distribution matrix multiplied by the inverse of the geometric distribution matrix. The calculation of the position accuracy factor needs to consider the user's equivalent distance error, which includes contributions from multiple error sources. Satellite clock error is estimated through the accuracy of clock error parameters, typically ranging from 2 to 5 nanoseconds, corresponding to a distance error of 0.6 to 1.5 meters. Ionospheric delay error is related to the solar activity cycle and geomagnetic activity; daytime errors are usually greater than nighttime errors, and errors in high-latitude regions are usually greater than those in low-latitude regions. Tropospheric delay error is related to meteorological conditions; the dry delay is relatively stable at approximately 2.3 meters, while the wet delay varies considerably, reaching 0-0.5 meters. Multipath error is environmentally related; the error is smaller in open environments, but can reach several meters in densely built-up areas.

[0117] When the positioning accuracy factor characterizes the magnitude characteristics of position coordinate measurement errors, a statistical error model needs to be established to describe these characteristics. Measurement errors consist of two components: systematic bias and random noise. Systematic bias has deterministic characteristics and can be eliminated through calibration, while random noise has statistical characteristics and needs to be described using statistical methods. The statistical characteristics of random errors include probability distribution type, mean, variance, and correlation. The probability distribution type is determined through the KS test or chi-square test, and in most cases conforms to a Gaussian distribution. The mean reflects the systematic shift of the error, ideally being zero. The variance reflects the dispersion of the error and is proportional to the positioning accuracy factor. Correlation describes the degree of association between errors at different times or in different directions, quantified by autocorrelation or cross-correlation functions. The temporal correlation of errors exhibits colored noise characteristics, with the correlation time constant related to the satellite's geometric change rate. The spatial correlation of errors is manifested as the coupling relationship between errors in the three directions (northeast, south, and east), described by the covariance matrix.

[0118] When using a positioning accuracy factor to describe the magnitude characteristics of measurement errors and generating a location coordinate uncertainty distribution in a geographic coordinate system, the generation of the uncertainty distribution employs a Bayesian inference framework. The prior distribution is established based on statistical analysis of historical positioning data, the likelihood function is established based on current observation conditions, and the posterior distribution is calculated using Bayes' theorem. The parameters of the prior distribution are estimated using maximum likelihood estimation or moment estimation methods, and the parameters include the distribution's location, scale, and shape parameters. The construction of the likelihood function needs to consider the statistical characteristics of observation noise and the influence of systematic errors. The posterior distribution is calculated using Markov chain Monte Carlo sampling, with commonly used algorithms including the Metropolis-Hastings algorithm and the Gibbs sampling algorithm. The Metropolis-Hastings algorithm generates candidate samples through a proposal distribution and decides whether to accept a candidate sample based on the acceptance probability. The Gibbs sampling algorithm samples through a conditional distribution and is suitable for parameter estimation in high-dimensional cases. The uncertainty distribution is represented in the form of a confidence region, and the boundaries of the confidence region are calculated using a quantile function.

[0119] In one optional implementation, the preliminary tracking results are mapped to the geographic coordinate system through the dynamic projection transformation relationship, and a dual consistency measurement of spatial distance and motion trend is performed between the results and the BeiDou position prediction. Based on the measurement results, the matching confidence is corrected using Bayesian methods to obtain the final tracking and positioning results that fuse dual-domain information, including:

[0120] The position coordinates and velocity vector of the target in the image space in the preliminary tracking results are optimized and analyzed to construct the forward transformation matrix corresponding to the dynamic projection transformation relationship;

[0121] By using the location coordinates and the velocity vector to perform spatial projection transformation through the forward transformation matrix, the mapping position and mapping velocity of the preliminary tracking results in the geographic coordinate system are established.

[0122] Based on BeiDou position prediction, a predicted position and predicted velocity are generated in a geographic coordinate system. A spatial Euclidean distance metric is constructed based on the mapped position and the predicted position. At the same time, a motion trend metric is generated based on the difference between the directional angle and the velocity magnitude using the mapped velocity and the predicted velocity.

[0123] By adaptively combining the spatial distance metric and the motion trend metric, a dual consistency metric is constructed. The dual consistency metric is then used as input to update the matching confidence of the preliminary tracking results using Bayesian posterior probability.

[0124] Based on the matching confidence level and the corresponding image spatial location of the preliminary tracking result, an optimized selection is made, and the final tracking and positioning result fused with dual-domain information is reconstructed according to the magnitude of the matching confidence level.

[0125] The target's position coordinates and velocity vector in image space are optimized and analyzed from the initial tracking results to construct a forward transformation matrix corresponding to the dynamic projection transformation relationship. In the specific implementation, the target's position coordinates in image space are assumed to be (u, v), and the velocity vector is (Δu, Δv), which can be obtained by differencing the target's position at multiple time points. Considering the dynamic changes in camera parameters and environmental factors, an adaptive optimization algorithm is used to correct the position coordinates and velocity vectors. By collecting camera intrinsic and extrinsic parameters and scene geometric structure information, a projection relationship from image space to the geographic coordinate system is established, forming a forward transformation matrix T that includes rotation, translation, and scaling transformations. This matrix can be represented as a comprehensive transformation relationship that includes information such as camera pose, intrinsic parameters, and external distortion correction, taking into account projection distortion caused by non-planar factors such as ground undulations.

[0126] Using the aforementioned position coordinates and velocity vectors, a spatial projection transformation is performed through a forward transformation matrix to establish the preliminary tracking results' mapped position and velocity in the geographic coordinate system. Specifically, the position coordinates (u, v) in image space are multiplied by the transformation matrix T to obtain the mapped position (x, y, z) in the geographic coordinate system; similarly, a Jacobian matrix transformation is applied to the velocity vector (Δu, Δv) to obtain the mapped velocity (vx, vy, vz) in the geographic coordinate system. To improve mapping accuracy, target height estimation is introduced during the projection process. The actual target height information is obtained through multi-view geometric constraints or deep learning methods, and the height value z is corrected, thereby optimizing the mapping results.

[0127] Based on BeiDou positioning prediction, predicted position and velocity are generated in a geographic coordinate system, and a dual consistency metric is constructed. Historical trajectory data of the target is obtained from the BeiDou system, and time-series prediction algorithms such as Kalman filtering or particle filtering are used to predict the target's current position (x', y', z') and velocity (vx', vy', vz') in the geographic coordinate system. A spatial Euclidean distance metric Ds is constructed based on the mapped and predicted positions, calculated as the three-dimensional spatial distance between the two points. Simultaneously, a motion trend metric Dt is generated using the mapped and predicted velocities, which comprehensively considers the difference between the direction angle and the velocity magnitude of the velocity vectors. The direction angle is calculated using the vector dot product, and the difference in velocity magnitude is represented by a normalized relative error. These two metrics together constitute the dual consistency metric.

[0128] A dual consistency metric is constructed by adaptively combining spatial distance and motion trend metrics, and the matching confidence of the initial tracking results is updated using Bayesian posterior probability. During the feature combination stage, adaptive weighting factors α and β are introduced, such that D = α·Ds + β·Dt, where α and β are dynamically adjusted according to the current tracking conditions. For example, the value of β is increased when the target moves quickly, and the weight of α is increased in stationary or low-speed moving scenarios. The combined metric D is used as the observation value, and the matching confidence P is updated using a Bayesian framework. Specifically, the prior probability P(match) represents the probability of a correct match in the initial tracking result, and the posterior probability P(match|D), i.e., the updated matching confidence, is calculated using the observed likelihood P(D|match) and the marginal probability P(D).

[0129] The system optimizes the selection based on the image spatial location corresponding to the matching confidence score and the initial tracking result, and reconstructs the final tracking and localization result by fusing dual-domain information based on the matching confidence score. In practical applications, the matching confidence score is calculated for multiple candidate targets, and a threshold Pth is set. When the highest confidence score exceeds the threshold, the target is selected as the final tracking result; if the confidence scores of all candidate targets are lower than the threshold, a re-detection mechanism is triggered or the tracking result of the previous frame is retained, but its reliability score is reduced. To enhance the robustness of the system, a temporal smoothing strategy can also be introduced, which filters out misjudgments caused by short-term interference by weighted averaging of the matching confidence scores of multiple consecutive frames.

[0130] In real-world scenarios, such as traffic monitoring systems, the above method effectively addresses vehicle tracking in complex environments. After the monitoring camera captures vehicle images and completes initial tracking, the vehicle's position is mapped to a geographic coordinate system through dynamic projection transformation, and then fused with location information provided by the vehicle's onboard BeiDou navigation equipment. Even under partial obstruction or changes in lighting conditions, the system maintains stable tracking performance, providing reliable location data support for traffic management and dispatching.

[0131] In one optional implementation, based on the residual distribution of the final tracking and positioning results in the image domain and the geographic domain, dynamically updating the error compensation parameters in the projection transformation relationship and the feature weight allocation strategy in the target multimodal representation model includes:

[0132] A dual-domain accuracy evaluation is performed on the final tracking and positioning results, and the positioning error distribution in the image space and the position residual distribution in the geographic space of the final tracking and positioning results are constructed.

[0133] A spatial compensation vector is established based on the positioning error distribution. The transformation parameters in the projection transformation relationship are adaptively adjusted using the spatial compensation vector to obtain the error compensation parameters.

[0134] A spatial similarity index is constructed based on the location residual distribution. The spatial similarity index is then used to evaluate the importance of feature components in the target multimodal representation model, and a feature weight allocation strategy is generated.

[0135] The error compensation parameters are applied to the projection transformation relationship, and the feature weight allocation strategy is integrated into the target multimodal representation model to achieve dynamic iterative optimization based on dual-domain residuals.

[0136] like Figure 2 As shown, the method includes:

[0137] A dual-domain accuracy evaluation is performed on the final tracking and positioning results, constructing the positioning error distribution in image space and the position residual distribution in geographic space. In image space, the positioning error is quantified by calculating the overlap rate between the target detection box and the tracking prediction box within the current frame, specifically using the Intersection over Union (IOU) metric to evaluate the matching degree between boxes. If the current frame tracking box coordinates are (x_t, y_t, w_t, h_t), and the corresponding detection box coordinates are (x_d, y_d, w_d, h_d), then the IOU value between the two is calculated, and (1-IOU) is taken as the positioning error in image space. By accumulating the positioning errors over multiple consecutive frames, an error sample set E_img={e_1, e_2, ..., e_n} is formed, and the kernel density estimation method is used to construct the positioning error distribution P_img(e) in the image domain.

[0138] In geospatial space, after mapping image coordinates to geographic coordinates through projection transformation, the Euclidean distance between the predicted location and the reference location is calculated as the location residual. Assuming the geographic predicted coordinates of the target in the current frame are (lon_p, lat_p), and the reference coordinates are (lon_r, lat_r), then the geographic location residual is the distance difference between the two. Similarly, by collecting residual samples from consecutive frames, a residual sample set R_geo={r_1, r_2, ..., r_m} is constructed, and a Gaussian mixture model is used to fit the geographic spatial location residual distribution P_geo(r).

[0139] A spatial compensation vector is constructed based on the positioning error distribution. This compensation vector is then used to adaptively adjust the transformation parameters in the projection transformation relationship to obtain the error compensation parameters. Specifically, statistical features, including the error mean μ_img and variance σ_img, are extracted from the image domain error distribution P_img(e). 2 And the skewness coefficient s_img. Based on these statistical characteristics, a two-dimensional spatial compensation vector C=(C_x, C_y) is constructed, where C_x represents the horizontal compensation amount and C_y represents the vertical compensation amount, calculated using the following formula:

[0140] C_x = μ_img·cos(θ)·f(σ_img, s_img);

[0141] C_y = μ_img·sin(θ)·f(σ_img, s_img);

[0142] Where θ is the principal direction angle of the error, and f(σ_img, s_img) is a weighting function based on variance and skewness.

[0143] The transformation matrix H in the projection transformation relation is decomposed into three parts: a rotation matrix R, a translation vector T, and a scale factor S. The translation vector T is adjusted according to the compensation vector C to obtain the corrected translation vector T' = T + λ·C, where λ is the adaptive learning rate, dynamically determined by the current tracking confidence. The adjusted translation vector T' is then recombinated with the original rotation matrix R and scale factor S to obtain the updated projection transformation matrix H', thus achieving error compensation in the projection transformation relation.

[0144] A spatial similarity index is constructed based on the location residual distribution. This index is then used to evaluate the importance of feature components in the target multimodal representation model, generating a feature weight allocation strategy. First, the residual mean μ_geo and residual variance σ_geo are extracted from the geospatial residual distribution P_geo(r). 2 Given the kurtosis k_geo, calculate the spatial stability score S_stable = (1 / (1+μ_geo))·exp(-σ_geo) 2 / α)·g(k_geo), where α is the scaling factor and g(k_geo) is the kurtosis adjustment function.

[0145] For each feature component f_i (such as color features, texture features, contour features, etc.) in the target multimodal representation model, its correlation with spatial stability ρ_i is calculated. This is done by analyzing the correlation between changes in feature f_i and changes in residuals. The larger the absolute value of the correlation ρ_i, the more significant the impact of that feature component on spatial localization. Based on the correlation and stability scores, the feature weight w_i = S_stable·|ρ_i| / Σ|ρ_j| is calculated to achieve adaptive weight allocation.

[0146] For multimodal feature fusion in complex scenes, a hierarchical weighting strategy is adopted, dividing features into three categories: appearance features (F_app), motion features (F_mot), and context features (F_ctx). First, the inter-class weights (w_class) are calculated, then the relative weights (w_rel) of each feature component within a class are determined, and the final feature weights are the product of the two. The weight ratios of each feature category are dynamically adjusted according to different scene conditions (such as changes in illumination and degree of occlusion). The weights of motion features are increased in scenes with drastic lighting changes, and the weights of context features are increased when occlusion is severe.

[0147] Error compensation parameters are applied to the projection transformation relationship, and the feature weight allocation strategy is integrated into the target multimodal representation model to achieve dynamic iterative optimization based on dual-domain residuals. In practical applications, a parameter update process is performed every fixed number of frames (e.g., every 20 frames) to achieve dynamic optimization of the projection transformation relationship and feature weights. For the projection transformation relationship, the updated transformation matrix H' is applied to the coordinate mapping of subsequent frames; for the feature weights, the updated weight vector {w_1, w_2, ..., w_k} is integrated into the feature fusion stage of the target representation model.

[0148] When implementing this solution in autonomous driving scenarios, image sequences captured by onboard cameras can be correlated with high-precision map data to accurately locate and track pedestrians, vehicles, and other traffic participants. A dynamic optimization mechanism based on dual-domain residuals effectively addresses various complex environments, such as sudden changes in tunnel lighting and GPS signal attenuation between tall buildings, ensuring tracking and positioning accuracy. Through iterative optimization, the system continuously learns environmental change patterns, improving the robustness and accuracy of target tracking.

[0149] A second aspect of this invention provides an intelligent image target tracking and positioning system under BeiDou position constraints, comprising:

[0150] Module 1 is used to acquire multi-source heterogeneous data streams of the target to be tracked, construct a multi-modal representation model of the target that integrates visual semantic features and spatial topological features, and establish a dynamic projection transformation relationship between the image coordinate system and the geographic coordinate system based on the multi-source heterogeneous data streams.

[0151] Module 2 is used to back-project the BeiDou position information into the image space based on the dynamic projection transformation relationship, and generate a search region mask by probabilistic modeling based on the covariance propagation of the BeiDou positioning accuracy factor and the target motion state; within the image range defined by the search region mask, the target multimodal representation model is applied to extract target candidates and match features to obtain preliminary tracking results and their corresponding matching confidence.

[0152] Module 3 is used to map the preliminary tracking results to the geographic coordinate system through the dynamic projection transformation relationship, perform a dual consistency measurement of spatial distance and motion trend with the BeiDou position prediction, and perform Bayesian correction on the matching confidence based on the measurement results to obtain the final tracking and positioning result that integrates dual-domain information.

[0153] Module 4 is used to dynamically update the error compensation parameters in the projection transformation relationship and the feature weight allocation strategy in the target multimodal representation model based on the residual distribution of the final tracking and positioning results in the image domain and the geographic domain, so as to achieve closed-loop adaptive optimization.

[0154] A third aspect of the present invention provides an electronic device, comprising:

[0155] processor;

[0156] Memory used to store processor-executable instructions;

[0157] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0158] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0159] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent tracking and positioning of image targets under BeiDou position constraints, characterized in that, include: Acquire multi-source heterogeneous data streams of the target to be tracked, construct a multi-modal representation model of the target that integrates visual semantic features and spatial topological features, and establish a dynamic projection transformation relationship between the image coordinate system and the geographic coordinate system based on the multi-source heterogeneous data streams; Based on the dynamic projection transformation relationship, the BeiDou position information is back-projected into the image space. A search region mask is generated by probabilistic modeling based on the covariance propagation of the BeiDou positioning accuracy factor and the target motion state. Within the image range defined by the search region mask, the target multimodal representation model is applied to extract target candidates and match features to obtain preliminary tracking results and their corresponding matching confidence. The preliminary tracking results are mapped to the geographic coordinate system through the dynamic projection transformation relationship. The consistency between the results and the BeiDou position prediction is measured in terms of both spatial distance and movement trend. Based on the measurement results, the matching confidence is corrected using Bayesian methods to obtain the final tracking and positioning result that integrates dual-domain information. Based on the residual distribution of the final tracking and positioning results in the image domain and the geographic domain, the error compensation parameters in the projection transformation relationship and the feature weight allocation strategy in the target multimodal representation model are dynamically updated to achieve closed-loop adaptive optimization.

2. The method according to claim 1, characterized in that, Constructing a target multimodal representation model that integrates visual semantic features and spatial topological features, and establishing a dynamic projection transformation relationship between the image coordinate system and the geographic coordinate system based on the multi-source heterogeneous data stream, includes: Multi-scale sliding window convolution operations are applied to a continuous sequence of image frames to analyze the local texture features and global contour features of the target, forming a set of visual semantic features that describe the appearance of the target. The geometric center position of the target in the image plane is determined by the set of visual semantic features. The relative distance vector and angle distribution between neighboring targets are constructed based on the geometric center position. The topological feature map reflecting the spatial relationship between the targets is generated by the relative distance vector and the angle distribution. The visual semantic feature set and the topological feature map are mapped to a unified semantic space through nonlinear dimensionality reduction to generate a fused feature representation. A weight learning strategy is used to model the importance of each modality feature in the fused feature representation. The modeling results are used to optimize the fused feature representation to obtain the target multimodal representation model. The real-time three-dimensional coordinates and attitude angle parameters in the carrier position and attitude information are analyzed. The real-time three-dimensional coordinates, attitude angle parameters and camera intrinsic and extrinsic parameter calibration information are subjected to matrix decomposition operation to obtain the initial projection transformation matrix at the current moment. At the same time, feature projection calibration is performed based on the target multimodal representation model. The parameter drift of the initial projection transformation matrix within a continuous time window is established, and the initial projection transformation matrix is ​​adaptively corrected based on the parameter drift and the rate of change of the carrier position and attitude information to form a dynamic projection transformation relationship.

3. The method according to claim 2, characterized in that, The visual semantic feature set and the topological feature map are mapped to a unified semantic space through nonlinear dimensionality reduction to generate a fused feature representation. The importance modeling of each modality feature in the fused feature representation is performed using a weighted learning strategy, including: A feature space transformation matrix is ​​constructed based on the difference in feature dimensions between the visual semantic feature set and the topological feature map. Nonlinear dimensionality reduction mapping is performed through the feature space transformation matrix to transform the visual semantic feature set and the topological feature map to a unified semantic space of a preset dimension, thereby generating visual semantic mapping features and topological structure mapping features. Feature integration operations are performed using the similarity distribution of the visual semantic mapping features and the topological structure mapping features in a unified semantic space to form a fused feature representation; a feature weight evaluation strategy is constructed based on the response intensity differences of each modality feature in the fused feature representation, and visual weight coefficients and topological weight coefficients are generated through the feature weight evaluation strategy; The visual semantic mapping features are optimized using the visual weight coefficients, and the topological structure mapping features are optimized using the topological weight coefficients. The optimized features are then combined and reconstructed to obtain a fusion feature representation that completes importance modeling.

4. The method according to claim 1, characterized in that, Based on the dynamic projection transformation relationship, the BeiDou position information is back-projected into the image space. A probabilistic model is then performed based on the covariance propagation of the BeiDou positioning accuracy factor and the target motion state to generate a search region mask, including: Nonlinear projection calculations are performed on the position coordinates of the target in the geographic coordinate system based on the BeiDou position information. The coordinates of the projection center point in the image space are constructed by the inverse transformation matrix of the dynamic projection transformation relationship. At the same time, the uncertainty distribution of the position coordinates in the geographic coordinate system is established based on the positioning accuracy factor in the BeiDou position information. The uncertainty distribution is reconstructed into the image space using the inverse transformation matrix to form the position error ellipse corresponding to the coordinates of the projection center point. Optimization calculations are then applied to the velocity vector and acceleration vector in the target motion state to generate the covariance matrix of the velocity vector and the acceleration vector. The uncertainty of the target's current state is quantitatively evaluated based on the covariance matrix. The evaluation result of the uncertainty of the state is then probabilistically fused with the position error ellipse to construct the probability distribution region of the target's appearance in the image space. The set of confidence pixels in the probability distribution region is filtered according to a preset probability threshold, and the set of confidence pixels is reconstructed into a search region mask with boundary smoothness through spatial neighborhood constraints.

5. The method according to claim 4, characterized in that, Nonlinear projection calculations are performed on the target's position coordinates in the geographic coordinate system based on BeiDou positioning information. The coordinates of the projection center point in the image space are constructed through the inverse transformation matrix of the dynamic projection transformation relationship. Simultaneously, the uncertainty distribution of the position coordinates in the geographic coordinate system is established based on the positioning accuracy factor in BeiDou positioning information, including: The position coordinates of the target in the geographic coordinate system in the BeiDou position information are calculated and the inverse transformation matrix corresponding to the dynamic projection transformation relationship is constructed; based on the position coordinates and the inverse transformation matrix, a nonlinear projection space transformation is performed to generate the mapping relationship of the position coordinates from the geographic coordinate system to the image coordinate system; Based on the mapping relationship, a spatial projection transformation operation is performed on the position coordinates to obtain the coordinates of the projection center point of the target in the image space; the positioning accuracy factor in the BeiDou position information is quantitatively analyzed, wherein the positioning accuracy factor characterizes the measurement error amplitude characteristics of the position coordinates; The positioning accuracy factor is used to describe the measurement error amplitude characteristics, and the uncertainty distribution of the location coordinates is generated in the geographic coordinate system.

6. The method according to claim 1, characterized in that, The preliminary tracking results are mapped to the geographic coordinate system through the dynamic projection transformation relationship. A dual consistency measurement of spatial distance and motion trend is performed between the results and the BeiDou position prediction. Based on the measurement results, the matching confidence is corrected using Bayesian methods to obtain the final tracking and positioning results fused from the dual-domain information, including: The position coordinates and velocity vector of the target in the image space are optimized and analyzed in the preliminary tracking results, and a forward transformation matrix corresponding to the dynamic projection transformation relationship is constructed. By using the location coordinates and the velocity vector to perform spatial projection transformation through the forward transformation matrix, the mapping position and mapping velocity of the preliminary tracking results in the geographic coordinate system are established. Based on BeiDou position prediction, a predicted position and predicted velocity are generated in a geographic coordinate system. A spatial Euclidean distance metric is constructed based on the mapped position and the predicted position. At the same time, a motion trend metric is generated based on the difference between the directional angle and the velocity magnitude using the mapped velocity and the predicted velocity. By adaptively combining the spatial Euclidean distance metric and the motion trend metric, a dual consistency metric is constructed. The dual consistency metric is then used as input to update the matching confidence of the preliminary tracking results using Bayesian posterior probability. Based on the matching confidence level and the corresponding image spatial location of the preliminary tracking result, an optimized selection is made, and the final tracking and positioning result fused with dual-domain information is reconstructed according to the magnitude of the matching confidence level.

7. The method according to claim 1, characterized in that, Based on the residual distribution of the final tracking and positioning results in the image domain and the geographic domain, the error compensation parameters in the projection transformation relationship and the feature weight allocation strategy in the target multimodal representation model are dynamically updated as follows: A dual-domain accuracy evaluation is performed on the final tracking and positioning results, and the positioning error distribution in the image space and the position residual distribution in the geographic space of the final tracking and positioning results are constructed. A spatial compensation vector is established based on the positioning error distribution. The transformation parameters in the projection transformation relationship are adaptively adjusted using the spatial compensation vector to obtain the error compensation parameters. A spatial similarity index is constructed based on the location residual distribution. The spatial similarity index is then used to evaluate the importance of feature components in the target multimodal representation model, and a feature weight allocation strategy is generated. The error compensation parameters are applied to the projection transformation relationship, and the feature weight allocation strategy is integrated into the target multimodal representation model to achieve dynamic iterative optimization based on dual-domain residuals.

8. An intelligent image target tracking and positioning system under BeiDou position constraints, used to implement the method of any one of claims 1-7, characterized in that, include: Module 1 is used to acquire multi-source heterogeneous data streams of the target to be tracked, construct a multi-modal representation model of the target that integrates visual semantic features and spatial topological features, and establish a dynamic projection transformation relationship between the image coordinate system and the geographic coordinate system based on the multi-source heterogeneous data streams. Module 2 is used to back-project the BeiDou position information into the image space based on the dynamic projection transformation relationship, and generate a search region mask by probabilistic modeling based on the covariance propagation of the BeiDou positioning accuracy factor and the target motion state; within the image range defined by the search region mask, the target multimodal representation model is applied to extract target candidates and match features to obtain preliminary tracking results and their corresponding matching confidence. Module 3 is used to map the preliminary tracking results to the geographic coordinate system through the dynamic projection transformation relationship, perform a dual consistency measurement of spatial distance and motion trend with the BeiDou position prediction, and perform Bayesian correction on the matching confidence based on the measurement results to obtain the final tracking and positioning result that integrates dual-domain information. Module 4 is used to dynamically update the error compensation parameters in the projection transformation relationship and the feature weight allocation strategy in the target multimodal representation model based on the residual distribution of the final tracking and positioning results in the image domain and the geographic domain, so as to achieve closed-loop adaptive optimization.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images