Positioning and tracking linkage method and device, computer equipment and readable storage medium

Abstract

The invention relates to the technical field of mobile positioning and video tracking, and provides a positioning and tracking linkage method and device, computer equipment and a readable storage medium. The method comprises the following steps: fusing absolute position coordinates of a mobile platform and a tracking target at each moment obtained by a Beidou positioning unit, attitude and motion parameter data of the mobile platform at each moment obtained by an inertial navigation measurement unit, and feature information of the tracking target at each moment obtained by a video acquisition unit; obtaining a positioning prediction result of the tracking target at each moment and a position prediction relation of the mobile platform relative to the tracking target at each moment; and according to the positioning prediction result of the tracking target at each moment and the position prediction relation of the mobile platform relative to the tracking target at each moment, obtaining a video tracking control instruction at each moment so as to drive a video acquisition unit to adjust the shooting parameters of the camera holder according to the video tracking control instruction at each moment. The method can improve the tracking precision.

Description

Technical Field

[0001] This application relates to the field of mobile positioning and video tracking technology, and in particular to a positioning and tracking linkage method, device, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] With the widespread application of mobile platforms such as drones, mobile robots, and vehicle monitoring, the demand for automatic video tracking technology is increasing. Currently, traditional video tracking methods mostly rely on single video image analysis and target feature matching algorithms to track targets. However, in complex scenarios such as high-speed movement of mobile platforms, complex lighting, and occlusion, problems such as target loss and decreased tracking accuracy are prone to occur. Summary of the Invention

[0003] Therefore, it is necessary to provide a positioning and tracking linkage method, device, computer equipment, computer-readable storage medium, and computer program product to address the above-mentioned technical problems.

[0004] Firstly, this application provides a positioning, tracking, and linkage method, including:

[0005] The absolute position coordinates of the mobile platform and the tracking target at each moment are obtained through the Beidou positioning unit, and the attitude and motion parameter data of the mobile platform at each moment are obtained through the inertial navigation measurement unit.

[0006] The tracking target area images at each moment are acquired by the video acquisition unit, and the feature information of the tracking target at each moment is extracted from the tracking target area images at each moment;

[0007] The absolute position coordinates of the mobile platform and the tracking target at each moment, the attitude and motion parameter data of the mobile platform at each moment, and the feature information of the tracking target at each moment are fused to obtain the positioning prediction results of the tracking target at each moment and the position prediction relationship of the mobile platform relative to the tracking target at each moment.

[0008] Based on the positioning prediction results of the tracking target at each moment and the position prediction relationship between the mobile platform and the tracking target at each moment, video tracking control commands are obtained at each moment to drive the video acquisition unit to adjust the shooting parameters of the camera pan-tilt unit according to the video tracking control commands at each moment; the camera pan-tilt unit is a component in the video acquisition unit.

[0009] Secondly, this application also provides a positioning and tracking linkage device, including:

[0010] The positioning and inertial navigation data acquisition module is used to acquire the absolute position coordinates of the mobile platform and the tracking target at each moment through the Beidou positioning unit, and to acquire the attitude and motion parameter data of the mobile platform at each moment through the inertial navigation measurement unit.

[0011] The feature information acquisition module is used to acquire images of the tracking target area at each moment through the video acquisition unit, and extract feature information of the tracking target at each moment from the images of the tracking target area at each moment;

[0012] The data fusion module is used to fuse the absolute position coordinates of the mobile platform and the tracking target at each moment, the attitude and motion parameter data of the mobile platform at each moment, and the feature information of the tracking target at each moment, to obtain the positioning prediction results of the tracking target at each moment and the position prediction relationship of the mobile platform relative to the tracking target at each moment.

[0013] The shooting parameter adjustment module is used to obtain video tracking control commands at each moment based on the positioning prediction results of the tracking target at each moment and the position prediction relationship between the mobile platform and the tracking target at each moment, so as to drive the video acquisition unit to adjust the shooting parameters of the camera pan-tilt head according to the video tracking control commands at each moment; the camera pan-tilt head is a component in the video acquisition unit.

[0014] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the above-described method.

[0015] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, the computer program being executed by a processor using the methods described above.

[0016] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that is executed by a processor using the methods described above.

[0017] The aforementioned positioning and tracking linkage method, device, computer equipment, computer-readable storage medium, and computer program product acquire the absolute position coordinates of the mobile platform and the tracking target at each moment through the BeiDou positioning unit, and acquire the attitude and motion parameter data of the mobile platform at each moment through the inertial navigation measurement unit; acquire images of the tracking target area at each moment through the video acquisition unit, and extract the feature information of the tracking target at each moment from the images of the tracking target area at each moment; fuse the absolute position coordinates of the mobile platform and the tracking target at each moment, the attitude and motion parameter data of the mobile platform at each moment, and the feature information of the tracking target at each moment to obtain the positioning prediction results of the tracking target at each moment and the position prediction relationship of the mobile platform relative to the tracking target at each moment; based on the positioning prediction results of the tracking target at each moment and the position prediction relationship of the mobile platform relative to the tracking target at each moment, obtain the video tracking control command at each moment to drive the video acquisition unit to adjust the shooting parameters of the camera pan-tilt unit according to the video tracking control command at each moment; the camera pan-tilt unit is a component in the video acquisition unit. This application fuses the absolute position coordinates of the mobile platform and the tracking target at each moment, the attitude and motion parameter data of the mobile platform at each moment, and the feature information of the tracking target at each moment to obtain the positioning prediction results of the tracking target at each moment and the position prediction relationship of the mobile platform relative to the tracking target at each moment. This yields video tracking control commands at each moment, thereby driving the video acquisition unit to adjust the shooting parameters of the camera pan-tilt unit according to the video tracking control commands at each moment. This enables real-time dynamic coupling of positioning data and tracking actions, achieving a breakthrough in bidirectional assistance between positioning and tracking. It overcomes the limitations of traditional video tracking methods in complex scenarios such as high-speed movement of mobile platforms, complex lighting, and occlusion, and can improve tracking accuracy, stability, and anti-interference capabilities. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is an application environment diagram of the positioning and tracking linkage method in one embodiment;

[0020] Figure 2 This is a flowchart illustrating a positioning and tracking linkage method in one embodiment;

[0021] Figure 3 This is a flowchart illustrating the positioning and tracking linkage method in another embodiment;

[0022] Figure 4This is a structural block diagram of a positioning and tracking linkage device in one embodiment;

[0023] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0025] It should be noted that the terms "comprising" and "having," and any variations thereof, as used in this application, are intended to cover non-exclusive inclusion. The term "multiple" as used in this application refers to two or more. The term "and / or" as used in this application refers to one of the solutions, or any combination of multiple solutions.

[0026] The positioning and tracking linkage method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Terminal 102 obtains the absolute position coordinates of the mobile platform and the tracking target at each moment through the BeiDou positioning unit, and obtains the attitude and motion parameter data of the mobile platform at each moment through the inertial navigation measurement unit; it obtains images of the tracking target area at each moment through the video acquisition unit, and extracts the feature information of the tracking target at each moment from the images of the tracking target area at each moment; it fuses the absolute position coordinates of the mobile platform and the tracking target at each moment, the attitude and motion parameter data of the mobile platform at each moment, and the feature information of the tracking target at each moment to obtain the positioning prediction results of the tracking target at each moment and the position prediction relationship of the mobile platform relative to the tracking target at each moment, so as to obtain the video tracking control commands at each moment, thereby driving the video acquisition unit to adjust the shooting parameters of the camera pan-tilt unit according to the video tracking control commands at each moment. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. Server 104 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0027] In one exemplary embodiment, such as Figure 2 As shown, a positioning and tracking linkage method is provided, which is applied to... Figure 1Taking terminal 102 as an example, the explanation includes the following steps S201 to S204. Wherein:

[0028] Step S201: Obtain the absolute position coordinates of the mobile platform and the tracking target at each moment through the Beidou positioning unit, and obtain the attitude and motion parameter data of the mobile platform at each moment through the inertial navigation measurement unit.

[0029] The mobile platform includes a BeiDou positioning unit, an inertial navigation measurement unit, a video acquisition unit, and a core control unit. The mobile platform can be a drone, a mobile robot, or a vehicle-mounted monitoring system. Terminal 102 can be the core control unit within the mobile platform.

[0030] The mobile platform can capture relevant videos of the target to be tracked, thereby enabling the tracking of the target.

[0031] The BeiDou-3 dual-mode positioning technology can be used to obtain the absolute position coordinates of the mobile platform at various times by receiving satellite signals from the BeiDou positioning unit. and the absolute position coordinates of the target at each moment. sampling frequency Positioning accuracy ≤ 1m. Among them, and Indicates longitude. and Indicates latitude, and Indicates altitude.

[0032] The attitude and motion parameters of the mobile platform at various moments can be acquired using inertial measurement units, such as micro-electro-mechanical systems (MEMS) based inertial sensors. These attitude and motion parameter data include attitude angles. (Pitch angle, roll angle, yaw angle), angular velocity acceleration sampling frequency Attitude angle measurement accuracy . Indicates pitch angle, Indicates the roll angle. Indicates the heading angle. Represents the angular velocity along the x-axis. Represents the angular velocity along the y-axis. Represents the z-axis angular velocity. Represents the acceleration along the x-axis. Represents the y-axis acceleration. This represents the acceleration along the z-axis.

[0033] Step S202: Acquire images of the tracking target area at each moment through the video acquisition unit, and extract the feature information of the tracking target at each moment from the images of the tracking target area at each moment.

[0034] The tracking target area is captured in real time by a video acquisition unit, obtaining images of the target area at various times. The video acquisition unit can employ a high-definition camera with a resolution ≥1080P and a frame rate ≥30fps, paired with... Heading rotation, A motorized pan-tilt head with tilt adjustment; therefore, the camera pan-tilt head in the video acquisition unit is... Heading rotation, An electric pan-tilt head with pitch adjustment.

[0035] Step S203: The absolute position coordinates of the mobile platform and the tracking target at each time, the attitude and motion parameter data of the mobile platform at each time, and the feature information of the tracking target at each time are fused to obtain the positioning prediction results of the tracking target at each time and the position prediction relationship of the mobile platform relative to the tracking target at each time.

[0036] The absolute position coordinates of the mobile platform and the tracking target at each moment, the attitude and motion parameter data of the mobile platform at each moment, and the feature information of the tracking target at each moment can be incorporated into a unified fusion framework based on the Kalman filter fusion algorithm. By constructing coupled state equations and observation equations, the positioning noise, inertial navigation cumulative error and video pixel offset error are corrected, and the positioning prediction results of the tracking target at each moment and the position prediction relationship of the mobile platform relative to the tracking target at each moment are obtained.

[0037] Step S204: Based on the positioning prediction results of the tracking target at each moment and the position prediction relationship between the mobile platform and the tracking target at each moment, video tracking control commands are obtained at each moment to drive the video acquisition unit to adjust the shooting parameters of the camera pan-tilt unit according to the video tracking control commands at each moment; the camera pan-tilt unit is a component in the video acquisition unit.

[0038] Based on the predicted positioning results of the tracking target at each moment and the predicted position relationship between the mobile platform and the tracking target at each moment, as well as the differences between the actual positioning results of the tracking target at each moment and the actual position relationship between the mobile platform and the tracking target at each moment, a positioning-driven tracking control model can be constructed to obtain video tracking control commands at each moment.

[0039] It can drive the video acquisition unit to adjust the shooting parameters of the camera pan-tilt unit according to the video tracking control commands at each moment, thereby realizing real-time linkage between positioning information and tracking actions.

[0040] In the aforementioned positioning and tracking linkage method, the absolute position coordinates of the mobile platform and the tracking target at each moment, the attitude and motion parameter data of the mobile platform at each moment, and the feature information of the tracking target at each moment are fused to obtain the positioning prediction results of the tracking target at each moment and the position prediction relationship of the mobile platform relative to the tracking target at each moment. This allows for the generation of video tracking control commands at each moment, thereby driving the video acquisition unit to adjust the shooting parameters of the camera pan-tilt unit according to the video tracking control commands at each moment. This method enables real-time dynamic coupling of positioning data and tracking actions, achieving a breakthrough in bidirectional assistance between positioning and tracking. It overcomes the limitations of traditional video tracking methods in complex scenarios such as high-speed movement of the mobile platform, complex lighting, and occlusion, and can improve tracking accuracy, stability, and anti-interference capabilities.

[0041] In one embodiment, after obtaining the absolute position coordinates of the mobile platform and the tracking target at each moment through the BeiDou positioning unit and the attitude and motion parameter data of the mobile platform at each moment through the inertial navigation measurement unit, the method provided in this application further includes: based on the absolute position coordinates of the mobile platform and the tracking target at each moment, the attitude and motion parameter data of the mobile platform at each moment, and the least squares method, under the optimization objective of unifying the timestamps of the BeiDou positioning unit and the inertial navigation measurement unit to the same reference, fitting the clock drift coefficient and deviation of the BeiDou positioning unit and the inertial navigation measurement unit to calibrate the timestamps of the BeiDou positioning unit and the inertial navigation measurement unit; using the calibrated timestamp of the BeiDou positioning unit as the reference, performing linear interpolation on the attitude and motion parameter data of the mobile platform at each moment to align the absolute position coordinates of the mobile platform and the tracking target at each moment with the attitude and motion parameter data of the mobile platform at each moment from the time dimension.

[0042] The built-in clocks of the BeiDou positioning unit and the inertial navigation measurement unit have independent deviations and drifts, so the original timestamps of the BeiDou positioning unit and the inertial navigation measurement unit need to be calibrated to the same time reference.

[0043] The clock drift coefficients and deviations of the Beidou positioning unit and the inertial navigation measurement unit can be fitted based on the absolute position coordinates of the mobile platform and the tracking target at each moment, the attitude and motion parameter data of the mobile platform at each moment, and the least squares method, as shown in Equations (1) and (2).

[0044] (1)

[0045] (2)

[0046] in, This represents the timestamp of the k-th sampling by the BeiDou positioning unit. This represents the timestamp of the m-th sample taken by the inertial navigation measurement unit. This indicates that the BeiDou positioning unit is sampling the original timestamp for the first time. Indicates the initial timestamp of the inertial navigation measurement unit's first sampling; clock offset. , With drift coefficient , This can be achieved through synchronous calibration during the initialization phase. It indicates the starting zero point of the unified time base.

[0047] Under the optimization objective of unifying the timestamps of the BeiDou positioning unit and the inertial navigation measurement unit to the same reference, equations (1) and (2) are solved to obtain the clock drift coefficients and deviations of the BeiDou positioning unit and the inertial navigation measurement unit.

[0048] Specifically, select no fewer than 10 sets of synchronous sampling points (simultaneously triggering BeiDou and inertial navigation sampling), that is, based on the mobile platform and the tracking target. Time to Absolute position coordinates at time, mobile platform time At that time The attitude and motion parameter data are used to solve the optimization objectives shown in equations (1), (2) and (3) to obtain the clock drift coefficient and deviation of the Beidou positioning unit and the inertial navigation measurement unit.

[0049] (3)

[0050] in Indicates the relationship with the BeiDou positioning unit. The inertial navigation sampling index corresponding to each sampling.

[0051] The timestamps of the BeiDou positioning unit and the inertial navigation measurement unit can be calibrated based on the clock drift coefficient and deviation of the BeiDou positioning unit and the inertial navigation measurement unit.

[0052] After calibration, the distribution of BeiDou sampling points (sparse) and inertial navigation sampling points (dense) on the time axis is uneven, based on the timestamp of the calibrated BeiDou positioning unit. Based on this, linear interpolation is performed on the attitude and motion parameter data of the mobile platform at various times to ensure a one-to-one correspondence between the absolute position coordinates of the mobile platform and the tracking target at the same time point and the attitude and motion parameter data of the mobile platform at various times. The linear interpolation method is adopted to balance real-time performance and accuracy, meeting the dynamic requirements of the mobile platform.

[0053] For the BeiDou positioning unit Uniform time of the next sampling Find two adjacent sampling points in the inertial navigation measurement unit. and ,satisfy:

[0054] (4)

[0055] The attitude and motion parameter data of the mobile platform (which can be called inertial navigation data) are in The interpolation result at point is:

[0056] (5)

[0057] like (The BeiDou positioning unit samples first, but the inertial navigation measurement unit is not activated), using forward extrapolation, if (The inertial navigation measurement unit stops first, while the BeiDou positioning unit continues sampling), using a backward extrapolation approach:

[0058] (6)

[0059] in This is the derivative of the first sampled inertial navigation system (calculated by fitting the data from the first three sets of inertial navigation data).

[0060] To ensure alignment, the synchronization error can be defined as the difference between the time corresponding to the interpolated inertial navigation data and the BeiDou sampling time:

[0061] (7)

[0062] when When, synchronization is deemed valid; when If so, increase the inertial navigation sampling frequency (temporarily increase to 1000Hz) and re-execute the interpolation calculation.

[0063] In this embodiment, under the optimization objective of unifying the timestamps of the BeiDou positioning unit and the inertial navigation measurement unit to the same reference, the clock drift coefficients and deviations of the BeiDou positioning unit and the inertial navigation measurement unit are fitted to calibrate their timestamps. Using the calibrated timestamps of the BeiDou positioning unit as a reference, linear interpolation is performed on the attitude and motion parameter data of the mobile platform at each moment to align the absolute position coordinates of the mobile platform and the tracking target at each moment with the attitude and motion parameter data of the mobile platform at each moment from a time dimension. This two-stage synchronization mechanism of timestamp calibration and data interpolation can solve the problems of data asynchrony and fusion delay caused by the difference in sampling frequencies between the BeiDou positioning unit and the inertial navigation measurement unit.

[0064] In one embodiment, feature information of the tracking target at each moment is extracted from the tracking target area image at each moment. The specific steps are as follows: extract the contour features, color features, and motion trajectory features of the tracking target at each moment from the tracking target area image at each moment; obtain the target distance between the mobile platform and the tracking target at each moment based on the absolute position coordinates of the mobile platform and the tracking target at each moment; adjust the weights of the contour features and color features of the tracking target at each moment based on the target distance at each moment; perform weighted fusion of the contour features and color features of the tracking target at each moment based on the weights of the contour features and color features of the tracking target at each moment to obtain the fused features of the tracking target at each moment; obtain the feature information of the tracking target at each moment based on the fused features and motion trajectory features of the tracking target at each moment.

[0065] The distance between the mobile platform and the tracked target at each moment can be obtained based on their absolute position coordinates. As shown in equation (8).

[0066] (8)

[0067] in The unit is meters (m). At long distances, color feature weights are increased; at close distances, outline features are emphasized. To track the spatial coordinates of the target, These are the three-dimensional spatial coordinates of the mobile platform.

[0068] The weights of the contour features and color features of the target at each moment can be adjusted according to the target distance at each moment, as shown in Equations (9) and (10).

[0069] (9)

[0070] (10)

[0071] in, The weights representing color features The weights of the contour features are represented.

[0072] In this embodiment, the weight of color features is increased when the distance between the mobile platform and the tracking target is far, and the contour features are strengthened when the distance is close. The feature extraction strategy of the tracking target can be dynamically adjusted by integrating positioning distance information, thereby improving the robustness of tracking target recognition at different distances.

[0073] In one embodiment, the absolute position coordinates of the mobile platform and the tracking target at each moment, the attitude and motion parameter data of the mobile platform at each moment, and the feature information of the tracking target at each moment are fused to obtain the positioning prediction result of the tracking target at each moment and the position prediction relationship of the mobile platform relative to the tracking target at each moment. The specific steps are as follows: determine the state vector and the observation vector; the state vector includes the motion characteristic term of the mobile platform and the trend coupling term of the tracking target; obtain the prior state prediction value at the current moment based on the state vector and the posterior state prediction value at the previous moment; obtain the prior covariance matrix at the current moment based on the posterior covariance matrix at the previous moment; the posterior covariance matrix at the previous moment... The matrix is ​​obtained from the Kalman gain matrix and the prior covariance matrix of the previous time step. The Kalman gain matrix of the current time step is obtained from the prior covariance matrix of the current time step. The observation value of the current time step is obtained from the observation vector, the absolute position coordinates of the mobile platform and the tracking target at the current time step, the attitude and motion parameter data of the mobile platform at the current time step, and the feature information of the tracking target at the current time step. The posterior state prediction value of the current time step is obtained from the Kalman gain matrix of the current time step and the observation value of the current time step. The positioning prediction result of the tracking target at the current time step and the position prediction relationship of the mobile platform relative to the tracking target at the current time step are obtained from the posterior state prediction value of the current time step.

[0074] The motion characteristics of the mobile platform and the trend coupling term of the target motion can be incorporated into the state vector. The state vector can cover multiple dimensions such as position, velocity, attitude, and pixel coordinates. The recursive formula of the state vector is as shown in equation (11).

[0075] (11)

[0076] Wherein, the state vector , These are the absolute position coordinates of the target in three-dimensional space. It is the velocity of a target moving along each axis in three-dimensional space. It's the attitude and perspective of the mobile platform. To track the target pixel coordinates, This is the state transition matrix (including the prediction results of the motion mode of the mobile platform). For noise driving matrix, () represents process noise.

[0077] The three sources of data can be uniformly mapped to the world coordinate system. That is, the absolute position coordinates of the mobile platform and the tracking target at the current moment, the attitude and motion parameter data of the mobile platform at the current moment, and the feature information of the tracking target at the current moment can be uniformly mapped to the world coordinate system to achieve multi-dimensional observation complementarity and obtain the observation vector as shown in equation (12).

[0078] (12)

[0079] Among them, the observation vector , The observation matrix (including pixel coordinates to world coordinates transformation relationship) is used. This is process noise.

[0080] The prior state prediction value at the current time can be obtained based on the state vector and the posterior state prediction value at the previous time step, as shown in equation (13).

[0081] (13)

[0082] in, Let k be the predicted value of the prior state at time k. The posterior state prediction at time k-1. Let be the state transition matrix.

[0083] The prior covariance matrix at the current time can be obtained from the posterior covariance matrix at the previous time step, as shown in equation (14).

[0084] (14)

[0085] in, Let k be the prior covariance matrix. Let be the posterior covariance matrix at time k-1. This is the process noise driving matrix. Let be the process noise covariance matrix.

[0086] The posterior covariance matrix of the previous time step is obtained from the Kalman gain matrix and the prior covariance matrix of the previous time step, as shown in equation (15).

[0087] (15)

[0088] in, Let be the posterior covariance matrix at time k-1. It is an identity matrix.

[0089] The Kalman gain matrix at the current time can be obtained from the prior covariance matrix at the current time, as shown in equation (16).

[0090] (16)

[0091] in, Here is the Kalman gain matrix. The observation matrix; To observe the noise covariance matrix.

[0092] The predicted posterior state value at the current time can be obtained based on the Kalman gain matrix and the observed value at the current time, as shown in Equation (17).

[0093] (17)

[0094] in, Let k be the observation value at time k. Let be the posterior state prediction value at time k.

[0095] Based on the posterior state prediction value at the current moment, the positioning prediction result of the tracking target at the current moment is obtained. And the current position prediction relationship between the mobile platform and the target being tracked, where the position prediction relationship is the predicted azimuth angle of the mobile platform relative to the target being tracked. Predicted distance .

[0096] In this embodiment, the absolute position coordinates of the mobile platform and the tracking target at each moment, the attitude and motion parameter data of the mobile platform at each moment, and the feature information of the tracking target at each moment are incorporated into a unified fusion framework. By constructing state vectors and observation vectors and performing Kalman recursive calculation, the positioning prediction results of the tracking target at each moment and the position prediction relationship of the mobile platform relative to the tracking target at each moment are obtained. This enables real-time dynamic coupling of positioning data and tracking actions, achieving a breakthrough in bidirectional assistance between positioning and tracking. It solves the limitations of traditional video tracking methods in complex scenarios such as high-speed movement of mobile platforms, complex lighting, and occlusion, and can improve tracking accuracy, stability, and anti-interference ability.

[0097] In one embodiment, the method provided by this application further includes: obtaining the positioning error at each moment based on the positioning prediction results of the tracking target at each moment and the position prediction relationship between the mobile platform and the tracking target at each moment, as well as the actual positioning results of the tracking target at each moment and the actual position relationship between the mobile platform and the tracking target at each moment; obtaining the average positioning error corresponding to each sliding window through a sliding window algorithm and the positioning error at each moment; adjusting the fusion weights of the absolute position coordinates of the mobile platform and the tracking target at the current moment, the attitude and motion parameter data of the mobile platform at the current moment, and the feature information of the tracking target at the current moment according to the average positioning error corresponding to the current sliding window; and obtaining the posterior covariance matrix at the current moment based on the adjusted fusion weights of the absolute position coordinates of the mobile platform and the tracking target at the current moment, the attitude and motion parameter data of the mobile platform at the current moment, and the feature information of the tracking target at the current moment, the Kalman gain matrix of the previous moment, and the prior covariance matrix of the previous moment.

[0098] Based on the predicted positioning results of the tracking target at each time point and the predicted position relationship between the mobile platform and the tracking target at each time point, as well as the actual positioning results of the tracking target at each time point and the actual position relationship between the mobile platform and the tracking target at each time point, the positioning error at each time point is obtained.

[0099] For the k-th frame of the image tracking target region, the positioning error is... The formula is shown in equation (18).

[0100] (18)

[0101] in, To predict the location coordinates of the target at time k, To track the target's true coordinates at time k.

[0102] The positioning error of the target region image in the k-th frame can be used to track the target region image. This is called the positioning error at time k.

[0103] The average positioning error for each sliding window can be obtained using the sliding window algorithm and the positioning error at each time step. Specifically, let the sliding window be... (Total 10 frames, if) Then take the former. (frame), the first in the window The frame positioning error is N represents the number of sliding windows. The mean positioning error of all frames within a window can be averaged to obtain the mean positioning error for that sliding window. As shown in equation (19).

[0104] (19)

[0105] Based on the mean positioning error corresponding to the current sliding window, adjust the fusion weights of the absolute position coordinates of the mobile platform and the tracking target at the current moment, the attitude and motion parameter data of the mobile platform at the current moment, and the feature information of the tracking target at the current moment, as shown in Equations (20) to (22).

[0106] (20)

[0107] (twenty one)

[0108] (twenty two)

[0109] in, The weights are the absolute position coordinates of the mobile platform and the tracking target at the current moment (which can be called BeiDou positioning data), the attitude and motion parameter data of the mobile platform at the current moment (which can be called inertial navigation data), and the feature information of the tracking target at the current moment (which can be called video data). When the average positioning error increases, the inertial navigation weight is increased to enhance the anti-blocking capability.

[0110] The posterior covariance matrix at the current moment can be obtained based on the adjusted absolute position coordinates of the mobile platform and the tracking target at the current moment, the fusion weights of the mobile platform's current attitude and motion parameter data and the tracking target's current feature information, the Kalman gain matrix of the previous moment and the prior covariance matrix of the previous moment. This can increase the weight of inertial navigation data in the data fusion process when the mean positioning error increases, thereby enhancing the anti-occlusion capability.

[0111] In this embodiment, based on the average positioning error corresponding to the current sliding window, the fusion weights of the absolute position coordinates of the mobile platform and the tracking target at the current moment, the attitude and motion parameter data of the mobile platform at the current moment, and the feature information of the tracking target at the current moment are adjusted. The weights can be adjusted in real time according to the positioning error, thereby improving the anti-interference capability.

[0112] In one embodiment, based on the positioning prediction results of the tracking target at each moment and the position prediction relationship between the mobile platform and the tracking target at each moment, video tracking control commands for each moment are obtained to drive the video acquisition unit to adjust the shooting parameters of the camera pan-tilt unit according to the video tracking control commands at each moment. The specific steps are as follows: Based on the positioning prediction results of the tracking target at each moment and the position prediction relationship between the mobile platform and the tracking target at each moment, as well as the actual positioning results of the tracking target at each moment and the actual position relationship between the mobile platform and the tracking target at each moment, the azimuth error, pixel offset error, and other parameters for each moment are obtained. The system calculates the predicted target distance and the target's movement speed at each moment; it obtains the camera pan / tilt angle and yaw angle control information based on the azimuth error and pixel offset error at each moment; it obtains the camera pan / tilt angle focal length adjustment information based on the predicted target distance at each moment; it obtains the camera pan / tilt angle shooting frame rate adjustment information based on the target's movement speed at each moment; and it obtains the video tracking control commands at each moment based on the camera pan / tilt angle and yaw angle control information, focal length adjustment information, and shooting frame rate adjustment information, thereby driving the video acquisition unit to adjust the camera pan / tilt angle shooting parameters according to the video tracking control commands at each moment.

[0113] The azimuth error at each moment can be obtained from the predicted azimuth angle in the position prediction relationship between the mobile platform and the target at each moment, and the difference between the true azimuth angle in the actual position relationship between the mobile platform and the target at each moment, as shown in Equation (23).

[0114] (twenty three)

[0115] in, for Azimuth error at any time, for Predict the azimuth angle at all times. for Real azimuth angle at all times.

[0116] The pixel offset error at each time can be obtained based on the difference between the predicted pixel coordinates of the tracking target in the positioning prediction results at each time and the pixel coordinates of the center of the tracking target area image in the actual positioning results at each time, as shown in Equation (24).

[0117] (twenty four)

[0118] in, for Pixel offset error at any given time. for Continuously track the center pixel coordinates of the target region image. for Continuously track the target and predict pixel coordinates.

[0119] Based on the azimuth error at each time and pixel offset error at each time step The camera's pan and tilt angle control information is obtained. As shown in equation (25).

[0120] (25)

[0121] in, These are the pixel error weighting coefficients. These are the parameters in the Proportional-Integral-Derivative (PID) algorithm.

[0122] The distance to the target at each moment can be predicted based on the relationship between the mobile platform's relative position to the target at each moment. By adjusting the focal length through the positioning distance-focal length mapping model, the focal length adjustment information of the camera pan-tilt unit is obtained in order to maintain the stable proportion of the tracking target in the tracking target area image, as shown in Equation (26).

[0123] (26)

[0124] in, For reference distance At a base focal length of 50m, the target proportion remains stable at 10%-30%.

[0125] The camera pan-tilt-zoom (PTZ) frame rate adjustment information can be obtained based on the target's motion speed at various times. Specifically, this is based on the acceleration of the moving platform collected by the inertial navigation system. Predicting the target's movement speed at various times, the camera pan-tilt unit increases the shooting frame rate to 60fps when the target is moving at high speed, and maintains the shooting frame rate at 30fps when the target is moving at low speed, thus balancing smoothness and power consumption.

[0126] In this embodiment, video tracking control commands at each moment are obtained based on the camera pan-tilt angle and yaw angle control information, focal length adjustment information, and shooting frame rate adjustment information. This drives the video acquisition unit to adjust the shooting parameters of the camera pan-tilt according to the video tracking control commands at each moment. This enables real-time dynamic coupling of positioning data and tracking actions, achieving a breakthrough in bidirectional assistance between positioning and tracking. It overcomes the limitations of traditional video tracking methods in complex scenarios such as high-speed movement of mobile platforms, complex lighting, and occlusion, and can improve tracking accuracy, stability, and anti-interference capabilities.

[0127] In one embodiment, the method provided by this application further includes: obtaining the tracking target matching degree based on the positioning prediction results of the tracking target at each time and the actual positioning results of the tracking target at each time; when the tracking target matching degree is less than the matching degree threshold, estimating the motion trajectory of the tracking target in a set future time period based on the positioning prediction results of the tracking target in historical time periods and the Kalman prediction algorithm; driving the camera pan-tilt unit of the video acquisition unit to scan according to the motion trajectory of the tracking target in the set future time period, and restarting the target detection algorithm to lock the tracking target.

[0128] The target matching degree can be obtained by comparing the predicted positioning results of the target at each time point with the actual positioning results of the target at each time point.

[0129] You can set a matching threshold according to the actual situation. For example, you can set the matching threshold to 0.6.

[0130] When the matching degree of the tracking target is less than the matching degree threshold, the tracking target’s trajectory for the set future time period is estimated based on the location prediction results of the tracking target’s historical time and the Kalman prediction algorithm, as shown in Equations (27) and (28).

[0131] (27)

[0132] (28)

[0133] in, ( (corresponding to the number of predicted steps within 3 seconds) For the first The step-by-step tracking of the target's coordinates in three-dimensional space. This is a multi-step state transition matrix. For the first The target is predicted to be located in the next step.

[0134] The camera pan-tilt unit of the video acquisition unit can scan the motion trajectory of the target in the future time period as set, and restart the target detection algorithm, such as the YOLO (You Only Look Once) target detection algorithm, to lock onto the target. The recapture time is ≤3 seconds, and the recapture success rate is ≥98%.

[0135] In this embodiment, when the matching degree of the tracking target is less than the matching degree threshold, the motion trajectory of the tracking target in the set future time period is estimated based on the positioning prediction results of the tracking target in the historical time period and the Kalman prediction algorithm; the camera pan-tilt unit of the video acquisition unit is driven to scan according to the motion trajectory of the tracking target in the set future time period, and the target detection algorithm is restarted to lock the tracking target, which can improve the tracking accuracy.

[0136] To better understand the above method, an application embodiment of the positioning, tracking and linkage method of this application is described in detail below.

[0137] With the widespread application of mobile platforms such as drones, mobile robots, and vehicle monitoring, the demand for automatic video tracking technology is increasing. Currently, traditional video tracking methods mostly rely on single video image analysis and target feature matching to achieve tracking. However, in scenarios such as high-speed movement of mobile platforms, complex lighting, and occlusion, problems such as target loss and decreased tracking accuracy are prone to occur.

[0138] Meanwhile, the application of positioning technology in mobile platforms is relatively mature. BeiDou positioning technology can provide high-precision absolute position information, and inertial navigation measurement units can acquire platform attitude and motion parameters in real time. However, single positioning technologies have limitations: BeiDou positioning is susceptible to signal interruption due to obstruction, and inertial navigation measurement units have cumulative errors, which significantly reduce accuracy over long-term use. In addition, the core defect of traditional technologies lies in the independence of positioning data and video tracking, without forming an effective linkage mechanism: positioning data is only used for platform navigation and does not participate in video tracking strategy optimization; video tracking relies solely on image features and cannot use positioning information to compensate for target loss in obstructed or high-speed motion scenarios, resulting in insufficient stability and adaptability of the overall system.

[0139] Therefore, how to overcome the technical bottleneck of separating positioning and tracking, and organically combine the high-precision absolute position information of BeiDou positioning, the real-time attitude motion information of inertial navigation, and video tracking technology to build a two-way linkage and coordination mechanism has become an urgent technical problem to be solved.

[0140] To address the aforementioned technical issues, this embodiment provides a positioning and tracking linkage method, which can also be called a video automatic tracking and BeiDou / inertial navigation positioning linkage method for mobile platforms. This method can achieve deep coupling and bidirectional assistance between positioning and tracking, overcome the limitations of single technologies in mobile scenarios, and improve tracking accuracy, stability, and anti-interference capabilities.

[0141] The location tracking and linkage method, the specific process is as follows: Figure 3 As shown, the core of this method lies in building a closed-loop system of "positioning acquisition - data fusion - linkage control - dynamic optimization". Through the coordinated execution of five steps, deep linkage between positioning and video tracking is achieved. The specific steps are as follows.

[0142] Step S1: Synchronous Acquisition and Calibration of BeiDou / Inertial Navigation Asynchronous Data:

[0143] By collecting multi-dimensional data in parallel with the BeiDou positioning unit and the inertial navigation measurement unit, and adopting a two-stage synchronization mechanism of timestamp calibration and data interpolation, the problem of data asynchrony caused by the difference in sampling frequency between the BeiDou positioning unit and the inertial navigation measurement unit is solved.

[0144] The BeiDou-3 dual-mode positioning technology can be used to obtain the absolute position coordinates of the mobile platform at various times by receiving satellite signals from the BeiDou positioning unit. and the absolute position coordinates of the target at each moment. sampling frequency Positioning accuracy ≤ 1m. Among them, and Indicates longitude. and Indicates latitude, and Indicates altitude.

[0145] The attitude and motion parameters of the mobile platform at various moments can be acquired using inertial measurement units, such as micro-electro-mechanical systems (MEMS) based inertial sensors. These attitude and motion parameter data include attitude angles. (Pitch angle, roll angle, yaw angle), angular velocity acceleration sampling frequency Attitude angle measurement accuracy . Indicates pitch angle, Indicates the roll angle. Indicates the heading angle. Represents the angular velocity along the x-axis. Represents the angular velocity along the y-axis. Represents the z-axis angular velocity. Represents the acceleration along the x-axis. Represents the y-axis acceleration. This represents the acceleration along the z-axis.

[0146] The built-in clocks of the BeiDou positioning unit and the inertial navigation measurement unit have independent deviations and drifts, so the original timestamps of the BeiDou positioning unit and the inertial navigation measurement unit need to be calibrated to the same time reference.

[0147] The clock drift coefficients and deviations of the Beidou positioning unit and the inertial navigation measurement unit can be fitted based on the absolute position coordinates of the mobile platform and the tracking target at each moment, the attitude and motion parameter data of the mobile platform at each moment, and the least squares method, as shown in Equations (1) and (2).

[0148] (1)

[0149] (2)

[0150] in, This represents the timestamp of the k-th sampling by the BeiDou positioning unit. This represents the timestamp of the m-th sample taken by the inertial navigation measurement unit. This indicates that the BeiDou positioning unit is sampling the original timestamp for the first time. Indicates the initial timestamp of the inertial navigation measurement unit's first sampling; clock offset. , With drift coefficient , This can be achieved through synchronous calibration during the initialization phase. It indicates the starting zero point of the unified time base.

[0151] Under the optimization objective of unifying the timestamps of the BeiDou positioning unit and the inertial navigation measurement unit to the same reference, equations (1) and (2) are solved to obtain the clock drift coefficients and deviations of the BeiDou positioning unit and the inertial navigation measurement unit.

[0152] Specifically, select no fewer than 10 sets of synchronous sampling points (simultaneously triggering BeiDou and inertial navigation sampling), that is, based on the mobile platform and the tracking target. Time to Absolute position coordinates at time, mobile platform time At that time The attitude and motion parameter data are used to solve the optimization objectives shown in equations (1), (2) and (3) to obtain the clock drift coefficient and deviation of the Beidou positioning unit and the inertial navigation measurement unit.

[0153] (3)

[0154] in Indicates the relationship with the BeiDou positioning unit. The inertial navigation sampling index corresponding to each sampling.

[0155] The timestamps of the BeiDou positioning unit and the inertial navigation measurement unit can be calibrated based on the clock drift coefficient and deviation of the BeiDou positioning unit and the inertial navigation measurement unit.

[0156] After calibration, the distribution of BeiDou sampling points (sparse) and inertial navigation sampling points (dense) on the time axis is uneven, based on the timestamp of the calibrated BeiDou positioning unit. Based on this, linear interpolation is performed on the attitude and motion parameter data of the mobile platform at various times to ensure a one-to-one correspondence between the absolute position coordinates of the mobile platform and the tracking target at the same time point and the attitude and motion parameter data of the mobile platform at various times. The linear interpolation method is adopted to balance real-time performance and accuracy, meeting the dynamic requirements of the mobile platform.

[0157] For the BeiDou positioning unit Uniform time of the next sampling Find two adjacent sampling points in the inertial navigation measurement unit. and ,satisfy:

[0158] (4)

[0159] The attitude and motion parameter data of the mobile platform (which can be called inertial navigation data) are in The interpolation result at point is:

[0160] (5)

[0161] like (The BeiDou positioning unit samples first, but the inertial navigation measurement unit is not activated), using forward extrapolation, if (The inertial navigation measurement unit stops first, while the BeiDou positioning unit continues sampling), using a backward extrapolation approach:

[0162] (6)

[0163] in This is the derivative of the first sampled inertial navigation system (calculated by fitting the data from the first three sets of inertial navigation data).

[0164] To ensure alignment, the synchronization error can be defined as the difference between the time corresponding to the interpolated inertial navigation data and the BeiDou sampling time:

[0165] (7)

[0166] when When, synchronization is deemed valid; when If so, increase the inertial navigation sampling frequency (temporarily increase to 1000Hz) and re-execute the interpolation calculation.

[0167] Step S2: Dynamic extraction and optimization of target features:

[0168] The video acquisition unit captures images of the area where the tracking target is located in real time, obtaining images of the tracking target area at various times. It can dynamically adjust the target feature extraction strategy by fusing positioning distance information, improving the robustness of target recognition at different distances.

[0169] The contour features, color features, and motion trajectory features of the tracking target at each moment can be extracted from the tracking target area images at each moment.

[0170] The distance between the mobile platform and the tracked target at each moment can be obtained based on their absolute position coordinates. As shown in equation (8).

[0171] (8)

[0172] in The unit is meters (m). At long distances, color feature weights are increased; at close distances, outline features are emphasized. To track the spatial coordinates of the target, These are the three-dimensional spatial coordinates of the mobile platform.

[0173] The weights of the contour features and color features of the target at each moment can be adjusted according to the target distance at each moment, as shown in Equations (9) and (10).

[0174] (9)

[0175] (10)

[0176] in, The weights representing color features The weights of the contour features are represented.

[0177] Based on the weights of the contour and color features of the tracking target at each time step, a weighted fusion of these features is performed to obtain the fused features of the tracking target at each time step. Based on the fused features and motion trajectory features of the tracking target at each time step, the feature information of the tracking target at each time step is obtained. The pixel coordinates of the tracking target can then be extracted from this feature information. .

[0178] Step S3: BeiDou-Inertial Navigation-Video Three-Source Coupled Kalman Filter Fusion:

[0179] The core control unit can construct a three-source data fusion model based on the Kalman filter fusion algorithm, incorporating the BeiDou absolute position, inertial navigation attitude motion parameters, and video pixel coordinates into a unified fusion framework to correct errors in individual data.

[0180] The motion characteristics of the mobile platform and the motion trend of the target can be coupled into the state vector of the Kalman filter fusion algorithm, and the BeiDou positioning coordinates, inertial navigation attitude parameters and tracking target pixel coordinates can be included in the observation vector.

[0181] The state vector can encompass multiple dimensions such as position, velocity, attitude, and pixel coordinates. The recursive formula for the state vector is as shown in equation (11).

[0182] (11)

[0183] Wherein, the state vector , These are the absolute position coordinates of the target in three-dimensional space. It is the velocity of a target moving along each axis in three-dimensional space. It's the attitude and perspective of the mobile platform. To track the target pixel coordinates, This is the state transition matrix (including the prediction results of the motion mode of the mobile platform). For noise driving matrix, () represents process noise.

[0184] The three sources of data can be uniformly mapped to the world coordinate system. That is, the absolute position coordinates of the mobile platform and the tracking target at the current moment, the attitude and motion parameter data of the mobile platform at the current moment, and the feature information of the tracking target at the current moment can be uniformly mapped to the world coordinate system to achieve multi-dimensional observation complementarity and obtain the observation vector as shown in equation (12).

[0185] (12)

[0186] Among them, the observation vector , The observation matrix (including pixel coordinates to world coordinates transformation relationship) is used. This is process noise.

[0187] The prior state prediction value at the current time can be obtained based on the state vector and the posterior state prediction value at the previous time step, as shown in equation (13).

[0188] (13)

[0189] in, Let k be the predicted value of the prior state at time k. The posterior state prediction at time k-1. Let be the state transition matrix.

[0190] The prior covariance matrix at the current time can be obtained from the posterior covariance matrix at the previous time step, as shown in equation (14).

[0191] (14)

[0192] in, Let k be the prior covariance matrix. Let be the posterior covariance matrix at time k-1. This is the process noise driving matrix. Let be the process noise covariance matrix.

[0193] The posterior covariance matrix of the previous time step is obtained from the Kalman gain matrix and the prior covariance matrix of the previous time step, as shown in equation (15).

[0194] (15)

[0195] in, Let be the posterior covariance matrix at time k-1. It is an identity matrix.

[0196] The Kalman gain matrix at the current time can be obtained from the prior covariance matrix at the current time, as shown in equation (16).

[0197] (16)

[0198] in, Here is the Kalman gain matrix. The observation matrix; To observe the noise covariance matrix.

[0199] The predicted posterior state value at the current time can be obtained based on the Kalman gain matrix and the observed value at the current time, as shown in Equation (17).

[0200] (17)

[0201] in, Let k be the observation value at time k. Let be the posterior state prediction value at time k.

[0202] Based on the posterior state prediction value at the current moment, the positioning prediction result of the tracking target at the current moment is obtained. And the current position prediction relationship between the mobile platform and the target being tracked, where the position prediction relationship is the predicted azimuth angle of the mobile platform relative to the target being tracked. Predicted distance .

[0203] By using recursive calculations to achieve complementary correction of multi-dimensional data, the positioning fusion accuracy is improved by more than 40% compared with traditional dual-source fusion.

[0204] Step S4: Position-driven PID composite linkage tracking control:

[0205] Based on the fusion results of step S3, namely the positioning prediction results of the tracking target at each moment and the position prediction relationship of the mobile platform relative to the tracking target at each moment, a PID composite control model with dual input parameters of positioning error and visual error is constructed to drive the video acquisition unit to dynamically adjust the shooting parameters.

[0206] The azimuth error at each moment can be obtained from the predicted azimuth angle in the position prediction relationship between the mobile platform and the target at each moment, and the difference between the true azimuth angle in the actual position relationship between the mobile platform and the target at each moment, as shown in Equation (23).

[0207] (twenty three)

[0208] in, for Azimuth error at any time, for Predict the azimuth angle at all times. for Real azimuth angle at all times.

[0209] The pixel offset error at each time can be obtained based on the difference between the predicted pixel coordinates of the tracking target in the positioning prediction results at each time and the pixel coordinates of the center of the tracking target area image in the actual positioning results at each time, as shown in Equation (24).

[0210] (twenty four)

[0211] in, for Pixel offset error at any given time. for Continuously track the center pixel coordinates of the target region image. for Continuously track the target and predict pixel coordinates.

[0212] Based on the azimuth error at each time and pixel offset error at each time step The camera's pan and tilt angle control information is obtained. As shown in equation (25).

[0213] (25)

[0214] in, These are the pixel error weighting coefficients. These are the parameters in the Proportional-Integral-Derivative (PID) algorithm.

[0215] The distance to the target at each moment can be predicted based on the relationship between the mobile platform's relative position to the target at each moment. By adjusting the focal length through the positioning distance-focal length mapping model, the focal length adjustment information of the camera pan-tilt unit is obtained in order to maintain the stable proportion of the tracking target in the tracking target area image, as shown in Equation (26).

[0216] (26)

[0217] in, For reference distance At a base focal length of 50m, the target proportion remains stable at 10%-30%.

[0218] The camera pan-tilt-zoom (PTZ) frame rate adjustment information can be obtained based on the target's motion speed at various times. Specifically, this is based on the acceleration of the moving platform collected by the inertial navigation system. Predicting the target's movement speed at various times, the camera pan-tilt unit increases the shooting frame rate to 60fps when the target is moving at high speed, and maintains the shooting frame rate at 30fps when the target is moving at low speed, thus balancing smoothness and power consumption.

[0219] By incorporating the target pixel offset and positioning distance error into the control model, and simultaneously adjusting the gimbal angle and focal length, the proportion of the target in the center area of ​​the image is kept stable, thus solving the problem of insufficient long-distance tracking accuracy caused by traditional single pixel offset control.

[0220] Step S5: Adaptive weight adjustment and dynamic optimization for target recapture:

[0221] Based on the predicted positioning results of the tracking target at each time point and the predicted position relationship between the mobile platform and the tracking target at each time point, as well as the actual positioning results of the tracking target at each time point and the actual position relationship between the mobile platform and the tracking target at each time point, the positioning error at each time point is obtained.

[0222] For the k-th frame of the image tracking target region, the positioning error is... The formula is shown in equation (18).

[0223] (18)

[0224] in, To predict the location coordinates of the target at time k, To track the target's true coordinates at time k.

[0225] The positioning error of the target region image in the k-th frame can be used to track the target region image. This is called the positioning error at time k.

[0226] The average positioning error for each sliding window can be obtained using the sliding window algorithm and the positioning error at each time step. Specifically, let the sliding window be... (Total 10 frames, if) Then take the former. (frame), the first in the window The frame positioning error is N represents the number of sliding windows. The mean positioning error of all frames within a window can be averaged to obtain the mean positioning error for that sliding window. As shown in equation (19).

[0227] (19)

[0228] Based on the mean positioning error corresponding to the current sliding window, adjust the fusion weights of the absolute position coordinates of the mobile platform and the tracking target at the current moment, the attitude and motion parameter data of the mobile platform at the current moment, and the feature information of the tracking target at the current moment, as shown in Equations (20) to (22).

[0229] (20)

[0230] (twenty one)

[0231] (twenty two)

[0232] in, The weights are the absolute position coordinates of the mobile platform and the tracking target at the current moment (which can be called BeiDou data), the attitude and motion parameter data of the mobile platform at the current moment (which can be called inertial navigation data), and the feature information of the tracking target at the current moment (which can be called video data). When the average positioning error increases, the inertial navigation weight is increased to enhance the anti-blocking capability.

[0233] The posterior covariance matrix at the current moment can be obtained based on the adjusted absolute position coordinates of the mobile platform and the tracking target at the current moment, the fusion weights of the mobile platform's current attitude and motion parameter data and the tracking target's current feature information, the Kalman gain matrix of the previous moment and the prior covariance matrix of the previous moment. This can increase the weight of inertial navigation data in the data fusion process when the mean positioning error increases, thereby enhancing the anti-occlusion capability.

[0234] The target matching degree can be obtained by comparing the predicted positioning results of the target at each time point with the actual positioning results of the target at each time point.

[0235] You can set a matching threshold according to the actual situation. For example, you can set the matching threshold to 0.6.

[0236] When the matching degree of the tracking target is less than the matching degree threshold, the tracking target’s trajectory for the set future time period is estimated based on the location prediction results of the tracking target’s historical time and the Kalman prediction algorithm, as shown in Equations (27) and (28).

[0237] (27)

[0238] (28)

[0239] in, ( (corresponding to the number of predicted steps within 3 seconds) For the first The step-by-step tracking of the target's coordinates in three-dimensional space. This is a multi-step state transition matrix. For the first The target is predicted to be located in the next step.

[0240] The camera pan-tilt unit of the video acquisition unit can scan the motion trajectory of the target in the future time period as set, and restart the target detection algorithm, such as the YOLO (You Only Look Once) target detection algorithm, to lock onto the target. The recapture time is ≤3 seconds, and the recapture success rate is ≥98%.

[0241] The positioning and tracking linkage method provided in this embodiment can be applied to mobile platform devices. These mobile platform devices may include a BeiDou positioning module, an inertial navigation measurement unit, a video acquisition unit, a core control unit, and a linkage execution unit. The linkage communication protocol between these units enables low-latency transmission (latency ≤ 100ms) of positioning data, video data, and control commands.

[0242] In this embodiment, a closed-loop architecture with deep linkage between positioning, video, and control is constructed. Through the collaborative design of the BeiDou positioning unit, inertial navigation measurement unit, video acquisition unit, core control unit, and linkage execution unit, real-time dynamic coupling of positioning data and tracking actions is achieved. The core control unit employs a multi-source heterogeneous data fusion algorithm to deeply integrate BeiDou absolute position data, inertial navigation attitude motion parameters, and tracking target features, generating accurate positioning prediction results and driving the video acquisition unit to adaptively adjust shooting parameters. Simultaneously, a dynamic weight optimization mechanism addresses the technical bottlenecks of single positioning or video tracking. This embodiment achieves a breakthrough in bidirectional assistance between positioning and tracking, significantly improving the stability and accuracy of tracking in complex environments, and is applicable to various mobile platforms such as drones, mobile robots, and vehicle-mounted equipment.

[0243] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0244] Based on the same inventive concept, this application also provides a positioning tracking linkage device for implementing the positioning tracking linkage method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more positioning tracking linkage device embodiments provided below can be found in the limitations of the positioning tracking linkage method described above, and will not be repeated here.

[0245] In one exemplary embodiment, such as Figure 4 As shown, a positioning and tracking linkage device is provided, wherein:

[0246] The positioning and inertial navigation data acquisition module 401 is used to acquire the absolute position coordinates of the mobile platform and the tracking target at each moment through the Beidou positioning unit, and to acquire the attitude and motion parameter data of the mobile platform at each moment through the inertial navigation measurement unit.

[0247] The feature information acquisition module 402 is used to acquire images of the tracking target area at each moment through the video acquisition unit, and extract feature information of the tracking target at each moment from the images of the tracking target area at each moment;

[0248] The data fusion module 403 is used to fuse the absolute position coordinates of the mobile platform and the tracking target at each moment, the attitude and motion parameter data of the mobile platform at each moment, and the feature information of the tracking target at each moment to obtain the positioning prediction result of the tracking target at each moment and the position prediction relationship of the mobile platform relative to the tracking target at each moment.

[0249] The shooting parameter adjustment module 404 is used to obtain video tracking control commands at each moment based on the positioning prediction results of the tracking target at each moment and the position prediction relationship between the mobile platform and the tracking target at each moment, so as to drive the video acquisition unit to adjust the shooting parameters of the camera pan-tilt head according to the video tracking control commands at each moment; the camera pan-tilt head is a component in the video acquisition unit.

[0250] In one embodiment, the device further includes a time synchronization module, configured to: based on the absolute position coordinates of the mobile platform and the tracking target at various times, the attitude and motion parameter data of the mobile platform at various times, and the least squares method, under the optimization objective of unifying the timestamps of the BeiDou positioning unit and the inertial navigation measurement unit to the same reference, fit the clock drift coefficients and deviations of the BeiDou positioning unit and the inertial navigation measurement unit to calibrate the timestamps of the BeiDou positioning unit and the inertial navigation measurement unit; and based on the calibrated timestamps of the BeiDou positioning unit, perform linear interpolation on the attitude and motion parameter data of the mobile platform at various times to align the absolute position coordinates of the mobile platform and the tracking target at various times with the attitude and motion parameter data of the mobile platform at various times from the time dimension.

[0251] In one embodiment, the feature information acquisition module 402 is further configured to: extract the contour features, color features, and motion trajectory features of the tracking target at each moment from the tracking target area image at each moment; obtain the target distance between the mobile platform and the tracking target at each moment based on the absolute position coordinates of the mobile platform and the tracking target at each moment; adjust the weights of the contour features and color features of the tracking target at each moment based on the target distance at each moment; perform weighted fusion of the contour features and color features of the tracking target at each moment based on the weights of the contour features and color features of the tracking target at each moment to obtain the fused features of the tracking target at each moment; and obtain the feature information of the tracking target at each moment based on the fused features and motion trajectory features of the tracking target at each moment.

[0252] In one embodiment, the data fusion module 403 is further configured to: determine a state vector and an observation vector; the state vector includes a motion characteristic term of the mobile platform and a trend coupling term of the tracking target; obtain a prior state prediction value at the current moment based on the state vector and the posterior state prediction value at the previous moment; obtain a prior covariance matrix at the current moment based on the posterior covariance matrix at the previous moment; the posterior covariance matrix at the previous moment is obtained based on the Kalman gain matrix at the previous moment and the prior covariance matrix at the previous moment; obtain a Kalman gain matrix at the current moment based on the prior covariance matrix at the current moment; obtain an observation value at the current moment based on the observation vector, the absolute position coordinates of the mobile platform and the tracking target at the current moment, the attitude and motion parameter data of the mobile platform at the current moment, and the feature information of the tracking target at the current moment; obtain a posterior state prediction value at the current moment based on the Kalman gain matrix at the current moment and the observation value at the current moment; and obtain the positioning prediction result of the tracking target at the current moment and the position prediction relationship of the mobile platform relative to the tracking target at the current moment based on the posterior state prediction value at the current moment.

[0253] In one embodiment, the data fusion module 403 is further configured to: obtain the positioning error at each moment based on the positioning prediction results of the tracking target at each moment and the position prediction relationship between the mobile platform and the tracking target at each moment, as well as the actual positioning results of the tracking target at each moment and the actual position relationship between the mobile platform and the tracking target at each moment; obtain the average positioning error corresponding to each sliding window through a sliding window algorithm and the positioning errors at each moment; adjust the fusion weights of the absolute position coordinates of the mobile platform and the tracking target at the current moment, the attitude and motion parameter data of the mobile platform at the current moment, and the feature information of the tracking target at the current moment, based on the average positioning error corresponding to the current sliding window; and obtain the posterior covariance matrix at the current moment based on the adjusted fusion weights of the absolute position coordinates of the mobile platform and the tracking target at the current moment, the attitude and motion parameter data of the mobile platform at the current moment, and the feature information of the tracking target at the current moment, the Kalman gain matrix of the previous moment, and the prior covariance matrix of the previous moment.

[0254] In one embodiment, the shooting parameter adjustment module 404 is further configured to: obtain the azimuth error, pixel offset error, target prediction distance, and target movement speed at each moment based on the positioning prediction results of the tracking target at each moment and the position prediction relationship between the mobile platform and the tracking target at each moment, as well as the actual positioning results of the tracking target at each moment and the actual position relationship between the mobile platform and the tracking target at each moment; obtain the pitch and heading angle control information of the camera pan / tilt unit based on the azimuth error and the pixel offset error at each moment; obtain the focal length adjustment information of the camera pan / tilt unit based on the target prediction distance at each moment; obtain the shooting frame rate adjustment information of the camera pan / tilt unit based on the target movement speed at each moment; and obtain the video tracking control command at each moment based on the pitch and heading angle control information, focal length adjustment information, and shooting frame rate adjustment information of the camera pan / tilt unit, so as to drive the video acquisition unit to adjust the shooting parameters of the camera pan / tilt unit according to the video tracking control command at each moment.

[0255] In one embodiment, the device further includes a re-locking module, configured to: obtain a tracking target matching degree based on the positioning prediction results and the actual positioning results of the tracking target at each time point; when the tracking target matching degree is less than a matching degree threshold, estimate the motion trajectory of the tracking target in a set future time period based on the positioning prediction results of the tracking target at historical time points and a Kalman prediction algorithm; drive the camera pan-tilt unit of the video acquisition unit to scan according to the motion trajectory of the tracking target in the set future time period, and restart the target detection algorithm to lock the tracking target.

[0256] Each module in the aforementioned positioning and tracking linkage device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0257] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data for embodiments of the positioning and tracking linkage method. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a positioning and tracking linkage method.

[0258] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0259] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0260] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0261] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0262] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0263] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0264] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0265] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A positioning, tracking, and linkage method, characterized in that, The method includes: The absolute position coordinates of the mobile platform and the tracking target at each moment are obtained through the Beidou positioning unit, and the attitude and motion parameter data of the mobile platform at each moment are obtained through the inertial navigation measurement unit. The tracking target area images at each moment are acquired by the video acquisition unit, and the feature information of the tracking target at each moment is extracted from the tracking target area images at each moment; The absolute position coordinates of the mobile platform and the tracking target at each moment, the attitude and motion parameter data of the mobile platform at each moment, and the feature information of the tracking target at each moment are fused to obtain the positioning prediction results of the tracking target at each moment and the position prediction relationship of the mobile platform relative to the tracking target at each moment. Based on the positioning prediction results of the tracking target at each moment and the position prediction relationship between the mobile platform and the tracking target at each moment, video tracking control commands are obtained at each moment to drive the video acquisition unit to adjust the shooting parameters of the camera pan-tilt unit according to the video tracking control commands at each moment; the camera pan-tilt unit is a component in the video acquisition unit.

2. The method according to claim 1, characterized in that, After obtaining the absolute position coordinates of the mobile platform and the tracking target at various times through the BeiDou positioning unit, and obtaining the attitude and motion parameter data of the mobile platform at various times through the inertial navigation measurement unit, the method further includes: Based on the absolute position coordinates of the mobile platform and the tracking target at each moment, the attitude and motion parameter data of the mobile platform at each moment, and the least squares method, under the optimization goal of unifying the timestamps of the BeiDou positioning unit and the inertial navigation measurement unit to the same benchmark, the clock drift coefficients and deviations of the BeiDou positioning unit and the inertial navigation measurement unit are fitted to calibrate the timestamps of the BeiDou positioning unit and the inertial navigation measurement unit. Using the timestamp of the calibrated BeiDou positioning unit as a reference, linear interpolation is performed on the attitude and motion parameter data of the mobile platform at each moment to align the absolute position coordinates of the mobile platform and the tracking target at each moment with the attitude and motion parameter data of the mobile platform at each moment from the time dimension.

3. The method according to claim 1, characterized in that, The step of extracting feature information of the tracking target at each time step from the tracking target region image at each time step includes: Extract the contour features, color features, and motion trajectory features of the tracking target at each moment from the tracking target area image at each moment; Based on the absolute position coordinates of the mobile platform and the tracking target at each moment, the target distance between the mobile platform and the tracking target at each moment is obtained; Based on the target distance at each time point, adjust the weights of the contour features and color features of the tracked target at each time point; Based on the respective weights of the contour features and color features of the tracked target at each time step, the contour features and color features of the tracked target at each time step are weighted and fused to obtain the fused features of the tracked target at each time step. Based on the fusion features and motion trajectory features of the target at each moment, the feature information of the target at each moment is obtained.

4. The method according to claim 1, characterized in that, The process of fusing the absolute position coordinates of the mobile platform and the tracking target at each moment, the attitude and motion parameter data of the mobile platform at each moment, and the feature information of the tracking target at each moment to obtain the positioning prediction result of the tracking target at each moment and the position prediction relationship of the mobile platform relative to the tracking target at each moment includes: Determine the state vector and observation vector; the state vector includes the motion characteristic term of the mobile platform and the trend coupling term of the tracked target; Based on the state vector and the posterior state prediction value of the previous time step, the prior state prediction value of the current time step is obtained. The prior covariance matrix at the current time is obtained from the posterior covariance matrix at the previous time step; the posterior covariance matrix at the previous time step is obtained from the Kalman gain matrix and the prior covariance matrix at the previous time step. Based on the prior covariance matrix at the current time, the Kalman gain matrix at the current time is obtained; Based on the observation vector, the absolute position coordinates of the mobile platform and the tracking target at the current moment, the attitude and motion parameter data of the mobile platform at the current moment, and the feature information of the tracking target at the current moment, the observation value at the current moment is obtained. Based on the Kalman gain matrix at the current time and the observation value at the current time, the posterior state prediction value at the current time is obtained; Based on the posterior state prediction value at the current moment, the positioning prediction result of the tracking target at the current moment and the position prediction relationship of the mobile platform relative to the tracking target at the current moment are obtained.

5. The method according to claim 4, characterized in that, The method further includes: Based on the positioning prediction results of the tracking target at each time and the position prediction relationship of the mobile platform relative to the tracking target at each time, as well as the actual positioning results of the tracking target at each time and the actual position relationship of the mobile platform relative to the tracking target at each time, the positioning error at each time is obtained. The average positioning error for each sliding window is obtained by using the sliding window algorithm and the positioning error at each time point. Based on the average positioning error corresponding to the current sliding window, adjust the fusion weights of the absolute position coordinates of the mobile platform and the tracking target at the current moment, the attitude and motion parameter data of the mobile platform at the current moment, and the feature information of the tracking target at the current moment. Based on the adjusted absolute position coordinates of the mobile platform and the tracking target at the current moment, the fusion weights of the current attitude and motion parameter data of the mobile platform and the current feature information of the tracking target, the Kalman gain matrix of the previous moment and the prior covariance matrix of the previous moment, the posterior covariance matrix of the current moment is obtained.

6. The method according to claim 1, characterized in that, The step of obtaining video tracking control commands at each moment based on the positioning prediction results of the tracking target at each moment and the position prediction relationship between the mobile platform and the tracking target at each moment, to drive the video acquisition unit to adjust the shooting parameters of the camera pan-tilt unit according to the video tracking control commands at each moment, includes: Based on the positioning prediction results of the tracking target at each time and the position prediction relationship of the mobile platform relative to the tracking target at each time, as well as the actual positioning results of the tracking target at each time and the actual position relationship of the mobile platform relative to the tracking target at each time, the azimuth error, pixel offset error, target prediction distance, and tracking target movement speed at each time are obtained. Based on the azimuth error and pixel offset error at each time moment, the pitch and yaw control information of the camera pan-tilt unit is obtained. Based on the predicted target distance at each time moment, the focus adjustment information of the camera pan-tilt unit is obtained; Based on the target's movement speed at each moment, the camera's pan-tilt-zoom (PTZ) frame rate adjustment information is obtained. Based on the camera pan-tilt angle and yaw angle control information, focal length adjustment information, and shooting frame rate adjustment information, video tracking control commands are obtained at each moment to drive the video acquisition unit to adjust the shooting parameters of the camera pan-tilt head according to the video tracking control commands at each moment.

7. The method according to claim 1, characterized in that, The method further includes: The tracking target matching degree is obtained based on the predicted positioning results of the tracking target at each time and the actual positioning results of the tracking target at each time. When the matching degree of the tracking target is less than the matching degree threshold, the trajectory of the tracking target in the set future time period is estimated based on the positioning prediction results of the tracking target at historical moments and the Kalman prediction algorithm. The camera pan-tilt unit of the video acquisition unit is driven to scan the motion trajectory of the target in the future time period as set, and the target detection algorithm is restarted to lock the target.

8. A positioning and tracking linkage device, characterized in that, The device includes: The positioning and inertial navigation data acquisition module is used to acquire the absolute position coordinates of the mobile platform and the tracking target at each moment through the Beidou positioning unit, and to acquire the attitude and motion parameter data of the mobile platform at each moment through the inertial navigation measurement unit. The feature information acquisition module is used to acquire images of the tracking target area at each moment through the video acquisition unit, and extract feature information of the tracking target at each moment from the images of the tracking target area at each moment; The data fusion module is used to fuse the absolute position coordinates of the mobile platform and the tracking target at each moment, the attitude and motion parameter data of the mobile platform at each moment, and the feature information of the tracking target at each moment, to obtain the positioning prediction results of the tracking target at each moment and the position prediction relationship of the mobile platform relative to the tracking target at each moment. The shooting parameter adjustment module is used to obtain video tracking control commands at each moment based on the positioning prediction results of the tracking target at each moment and the position prediction relationship between the mobile platform and the tracking target at each moment, so as to drive the video acquisition unit to adjust the shooting parameters of the camera pan-tilt head according to the video tracking control commands at each moment; the camera pan-tilt head is a component in the video acquisition unit.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images