Target tracking method and system based on path planning and follow-up holder cooperation

Abstract

The invention provides a target tracking method and system based on path planning and follow-up holder cooperation, and relates to the technical field of holder control, and the method comprises the steps: obtaining the dynamic trajectory data of a target in real time, constructing a three-dimensional motion trajectory diagram for path tracking, and obtaining a target tracking path; setting an expected pose parameter of the follow-up holder, synchronizing the target tracking path to the follow-up holder for dynamic updating, and obtaining a parameter coupling degree; and performing path planning based on the parameter coupling degree to generate a cooperative control instruction set, driving the follow-up holder to perform bidirectional control, and performing tracking control on the target through a bidirectional control instruction. According to the target tracking method and device, the technical problem that the target is lost or the cradle head cannot stably align to the target due to the fact that the target movement speed is high and the environment is complex is solved, and the target tracking precision in a complex scene is improved by calculating the parameter coupling degree and optimizing cooperation of path planning and cradle head control.

Description

Technical Field

[0001] This application relates to the field of gimbal control technology, and in particular to a target tracking method and system based on path planning and servo gimbal collaboration. Background Technology

[0002] A pan-tilt head (PTZ) is a type of gimbal that automatically adjusts its orientation and angle based on external changes or control signals, used to support and control cameras. Currently, target tracking typically employs vision-based tracking methods, combined with data fusion from sensors such as LiDAR and ultrasonic sensors to enhance environmental awareness. However, these methods still have many limitations when facing complex factors such as dynamic obstacles, changing environments, and path curvature. When a target moves at high speed, its position changes very rapidly, causing existing target tracking systems to fail to update the target's position in time, resulting in target loss. Target recognition and tracking are difficult in complex environments (such as dynamic obstacles, changing lighting, occlusion, etc.), especially when obstacles rapidly obscure the target or the target quickly changes direction. Existing tracking algorithms struggle to adjust quickly, leading to real-time alignment failures and resulting in tracking interruptions or decreased accuracy.

[0003] In summary, existing technologies suffer from technical problems such as target loss or gimbal inability to stably align with the target due to the high speed of the target movement and the complexity of the environment, which affects the accuracy of target tracking. Summary of the Invention

[0004] The purpose of this application is to provide a target tracking method and system based on path planning and servo gimbal coordination, in order to solve the technical problems in the prior art where the target is lost or the gimbal cannot be stably aligned with the target due to the high speed of the target movement and the complex environment, thus affecting the accuracy of target tracking.

[0005] In view of the above problems, this application provides a target tracking method and system based on path planning and servo gimbal coordination.

[0006] Firstly, this application provides a target tracking method based on path planning and servo gimbal coordination. This method is implemented through a target tracking system based on path planning and servo gimbal coordination. The method includes: acquiring dynamic trajectory data of the target in real time, constructing a three-dimensional motion trajectory map for path tracking, and obtaining a target tracking path; setting desired pose parameters for the servo gimbal, synchronizing the target tracking path to the servo gimbal according to the desired pose parameters for dynamic updating, and obtaining parameter coupling degree; generating a collaborative control instruction set based on the parameter coupling degree, driving the servo gimbal for bidirectional control according to the collaborative control instruction set, and performing target tracking control through bidirectional control instructions.

[0007] Optionally, a multimodal sensor array is used to collect data on the target in real time to obtain a multimodal dataset, which includes lidar point cloud data, target visual depth data, and angular velocity data. The lidar point cloud data, target visual depth data, and angular velocity data are spatiotemporally aligned to determine a three-dimensional trajectory point sequence. The target motion state is predicted according to the three-dimensional trajectory point sequence to obtain a target predicted trajectory sequence. Trajectory conflict detection is performed based on the target predicted trajectory sequence to determine multiple predicted trajectory data. Motion backward inference is performed based on the multiple predicted trajectory data, and the inference results are labeled to construct the three-dimensional motion trajectory map.

[0008] Optionally, a dynamic obstacle prediction factor is introduced into the three-dimensional motion trajectory map to construct a four-dimensional search space; gimbal field of view constraints are extracted based on the servo gimbal, and a motion feasibility assessment is performed on the four-dimensional search space according to the gimbal field of view constraints to generate a motion feasibility score; multiple executable paths are identified on the three-dimensional motion trajectory map based on the motion feasibility score; the multiple executable paths are traversed and simulated for screening to determine the target tracking path.

[0009] Optionally, a mobile platform coordinate system for the target is constructed based on the three-dimensional motion trajectory diagram, and a gimbal base coordinate system for the servo gimbal is constructed based on the four-dimensional search space; the mobile platform coordinate system and the gimbal base coordinate system are spatially transformed to construct a transformation relationship matrix; multiple path points are extracted by traversing the target tracking path, and the multiple path points are transformed according to the transformation relationship matrix to set the initial pose parameters of the servo gimbal; the mechanical limit constraints of the servo gimbal are extracted to construct a feasible pose parameter space for the servo gimbal, and the initial pose parameters are synchronized to the feasible pose parameter space to determine the desired pose parameters.

[0010] Optionally, attitude analysis of the target is performed based on the mobile platform coordinate system to construct a homogeneous transformation matrix; installation offset analysis is performed based on the gimbal base coordinate system to construct the gimbal kinematic chain; and coordinate system rotation transformation compensation is performed based on the path point velocity vector introduced by the target tracking path to construct the transformation relationship matrix.

[0011] Optionally, the target tracking path is synchronized to the servo gimbal for parsing according to the desired pose parameters to determine the three-dimensional waypoint sequence of the target tracking path. The three-dimensional waypoint sequence includes path curvature features and timestamp information. Based on the path curvature features and the timestamp information, a decoupling evaluation is performed, and a parameter coupling index is set. Parameter coupling is calculated according to the parameter coupling index to obtain the parameter coupling degree.

[0012] Optionally, dynamic coupling analysis is performed based on the parameter coupling degree to determine the dynamic coupling degree index, and coupling state feature vectors are extracted based on the dynamic coupling degree index; weighted fusion is performed based on the coupling state feature vectors to generate a collaborative control instruction set; bidirectional data interaction feedback is performed on the servo gimbal according to the collaborative control instruction set to generate composite control parameters, and dynamic gain is performed based on the composite control parameters to construct the bidirectional control instructions for target tracking control.

[0013] Optionally, a weighted fusion is performed based on the coupled state feature vector to generate path correction and position adjustment amounts; dynamic priority allocation is performed based on the path correction and position adjustment amounts to determine a first execution weight and a second execution weight; control analysis is performed according to the first execution weight and the path correction amount to generate a mobile platform path correction instruction; control analysis is performed according to the second execution weight and the position adjustment amount to generate a gimbal pose adjustment instruction; the mobile platform path correction instruction and the gimbal pose adjustment instruction are integrated to generate a collaborative control instruction set.

[0014] Optionally, the collaborative control instruction set is parsed in real time to separate and generate the mobile platform path correction instruction and the gimbal pose adjustment instruction; the gimbal is adjusted and recorded based on the gimbal pose adjustment instruction to obtain gimbal status data; a dual-channel control architecture is constructed, which includes a forward channel and a reverse channel; the mobile platform path correction instruction is transmitted through the forward channel, and the gimbal status data is fed back through the reverse channel to generate composite control parameters; forward coupling analysis is performed based on the composite control parameters to generate a feedforward gain matrix; reverse coupling analysis is performed based on the composite control parameters to generate a feedback compensation vector; and control loop gain is calculated based on the feedforward gain matrix and the feedback compensation vector to construct the bidirectional control instruction.

[0015] Secondly, this application also provides a target tracking system based on path planning and servo gimbal coordination, used to execute the target tracking method based on path planning and servo gimbal coordination as described in the first aspect. The target tracking system based on path planning and servo gimbal coordination includes: a path tracking module for acquiring dynamic trajectory data of the target in real time, constructing a three-dimensional motion trajectory map for path tracking, and obtaining a target tracking path; a dynamic update module for setting the desired pose parameters of the servo gimbal, synchronizing the target tracking path to the servo gimbal according to the desired pose parameters for dynamic updating, and obtaining a parameter coupling degree; and a bidirectional control module for generating a collaborative control instruction set based on the parameter coupling degree, driving the servo gimbal to perform bidirectional control according to the collaborative control instruction set, and performing target tracking control through bidirectional control instructions.

[0016] One or more technical solutions provided in this application have at least the following beneficial effects: By acquiring the target's dynamic trajectory data in real time, a 3D motion trajectory map is constructed for path tracking, obtaining the target tracking path. The desired pose parameters of the servo gimbal are set, and the target tracking path is synchronized to the servo gimbal for dynamic updates according to these parameters, obtaining the parameter coupling degree. Based on the parameter coupling degree, path planning is performed to generate a collaborative control instruction set. The servo gimbal is then driven for bidirectional control according to this command set, enabling target tracking control. In other words, by acquiring the target's dynamic trajectory data in real time, constructing a 3D motion trajectory map, and rapidly responding to the target's high-speed movement, the target's motion mode in complex environments is determined. The desired pose parameters of the gimbal are set and dynamically updated to maintain stable alignment with the target. Path planning based on the parameter coupling degree improves the accuracy and robustness of target tracking in complex scenarios. Closed-loop control reduces tracking errors and balances system energy consumption.

[0017] The above description is merely an overview of the technical solution of this application. To better understand the technical means of this application and to facilitate its implementation according to the description, and to make the above and other objects, features, and advantages of this application more apparent, specific embodiments of this application are described below. It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent through the following description. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the target tracking method based on path planning and servo gimbal coordination in this application.

[0020] Figure 2 This is a schematic diagram of the target tracking system based on path planning and servo gimbal coordination in this application.

[0021] Figure labeling: Path tracking module 11, dynamic update module 12, bidirectional control module 13. Detailed Implementation

[0022] This application provides a target tracking method and system based on path planning and servo gimbal coordination. It addresses the technical problem in existing technologies where high target speeds and complex environments lead to target loss or gimbal instability in alignment, thus affecting target tracking accuracy. By acquiring dynamic trajectory data of the target in real time, constructing a 3D motion trajectory map, and rapidly responding to high-speed target movement, the method determines the target's motion pattern in complex environments, sets and dynamically updates the desired pose parameters of the gimbal to maintain stable alignment, and performs path planning based on parameter coupling. This improves the accuracy and robustness of target tracking in complex scenarios. Closed-loop control reduces tracking errors and balances system energy consumption.

[0023] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. It should be understood that this application is not limited to the exemplary embodiments described herein. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. It should also be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all of them.

[0024] Example 1, please refer to the appendix. Figure 1 This application provides a target tracking method based on path planning and gimbal coordination. The target tracking method based on path planning and gimbal coordination is executed by a target tracking system based on path planning and gimbal coordination. The target tracking method based on path planning and gimbal coordination specifically includes the following steps: S100: Acquires dynamic trajectory data of the target in real time, constructs a three-dimensional motion trajectory map for path tracking, and obtains the target tracking path.

[0025] Furthermore, this application S100 includes: A multimodal sensor array is used to collect data on the target in real time to obtain a multimodal dataset, which includes lidar point cloud data, target visual depth data, and angular velocity data. The lidar point cloud data, target visual depth data, and angular velocity data are spatiotemporally aligned to determine a three-dimensional trajectory point sequence. The target motion state is predicted according to the three-dimensional trajectory point sequence to obtain a target predicted trajectory sequence. Trajectory conflict detection is performed based on the target predicted trajectory sequence to determine multiple predicted trajectory data. Motion backward inference is performed based on the multiple predicted trajectory data, and the inference results are labeled to construct the three-dimensional motion trajectory map.

[0026] Specifically, a multimodal sensor array refers to an array that integrates multiple different types of sensors to acquire information across multiple dimensions. Different sensors possess different data acquisition capabilities, providing complementary information. A multimodal sensor array typically includes at least a lidar system, a binocular vision sensor, and an inertial measurement unit (IMU). It collects lidar point cloud data, target visual depth data from the binocular vision sensor, and angular velocity data from the IMU. The lidar scans the surrounding environment with a laser beam to obtain lidar point cloud data and 3D point cloud data of the object; the target visual depth data is obtained by calculating the parallax between two cameras to acquire the target's depth information; the IMU is a sensor used to measure dynamic information such as object acceleration and angular velocity, providing the target's angular velocity data.

[0027] Data collected by each sensor exists in different temporal and spatial dimensions, necessitating spatiotemporal alignment. Spatiotemporal alignment refers to the synchronous processing and fusion of data from different sensors across time and space, ensuring that these data correspond to the same spatial location at the same point in time. This allows for precise analysis by combining data from various sensors. Based on the timestamps of each sensor, it is ensured that samples collected from different data sources are synchronized at the same moment. For spatial alignment, coordinate transformation is typically performed based on the relative position and orientation of the sensors. For example, the coordinate system of the LiDAR is converted to a coordinate system consistent with that of the vision sensor, or the coordinate system is adjusted based on the angular velocity information provided by the inertial measurement unit to ensure that all data can be combined within the same three-dimensional spatial coordinate system.

[0028] A 3D trajectory point sequence is a path sequence composed of multiple 3D points, describing the target's motion trajectory in 3D space over time. Each 3D point represents the target's position at a given moment (represented by its X, Y, Z coordinates). Through spatiotemporal alignment, data from multiple sensors are integrated into a continuous trajectory sequence, displaying the target's movement path.

[0029] Based on the three-dimensional trajectory point sequence, the target's motion state is predicted, i.e., the possible future trajectories of the target are predicted, forming a target predicted trajectory sequence. Based on the obtained three-dimensional trajectory point sequence, the target's motion state at each time point is calculated, and the target's velocity, acceleration, and direction of motion are estimated by analyzing position changes. Using the known target motion state (such as position, velocity, acceleration, etc.) and combining it with a conventional kinematic model, the target's position at multiple future time points is predicted, and the prediction results are used to generate a new trajectory point sequence, i.e., the target predicted trajectory sequence. For example, suppose the target's current position is (5.3, 2.1, 7.4), its velocity is (0.2, 0.1, 0.1) m / s, and its current acceleration is (0.1, 0.05, 0.05) m / s². Using the prediction model, assuming the target moves at a constant velocity, the possible position the target can reach at the next time step (t+1) is calculated to be (5.5, 2.2, 7.5), and the velocity at the next time step is (0.22, 0.11, 0.11) m / s. Based on this prediction, the trajectory is calculated for further time steps, eventually yielding a predicted sequence of trajectory points, such as: position at time T1 (5.3, 2.1, 7.4), position at time T2 (5.5, 2.2, 7.5), position at time T3 (5.7, 2.3, 7.6), and position at time T4 (5.9, 2.4, 7.8), thus outputting a predicted trajectory sequence for the target. If the target accelerates, an acceleration trajectory is generated, and other acceleration trajectories are generated; if the target suddenly turns, other turning trajectories are generated, and so on.

[0030] Based on the predicted trajectory sequence of the targets, trajectory conflict detection is performed. Each predicted trajectory is compared with other trajectories to check for spatial overlap or intersection, identifying trajectories that may collide, intersect, or overlap, and making adjustments to avoid the impact of conflicts. The spatial positions of different trajectories at the same or different time points are checked for overlap; overlap indicates a conflict between the two trajectories, requiring avoidance. Multiple trajectories are analyzed to determine if they intersect or cross within the same time period, and whether there are potential collisions between targets. The speed differences between targets on different trajectories may also affect the time and location of their encounter when the speeds differ significantly; this must also be considered. Conflict detection involves not only comparing the overlap of a single trajectory with other trajectories but also performing a comprehensive analysis of multiple trajectory data to identify all trajectories that may potentially collide. For example, if there are multiple targets or multiple predicted trajectories, all trajectory data that may overlap or intersect are identified and marked.

[0031] Once a trajectory conflict is detected, the conflicting trajectory is marked, and the predicted trajectory is adjusted to change the target's direction of movement or speed; or the target's path planning is modified to avoid intersections or overlaps; in extreme cases, some targets are temporarily stopped or their speed is adjusted to avoid conflict with other targets. Multiple predicted trajectory data are obtained after trajectory conflict detection. Motion back-analysis is performed based on these multiple predicted trajectory data, that is, the target's past motion state is deduced from the known predicted trajectory data, understanding the target's historical trajectory and thus better understanding the target's motion patterns. Through back-analysis, the target's past trajectory can be inferred from its future state; back-analysis helps to understand the target's motion patterns and predict future trajectory changes.

[0032] After performing reverse engineering, different trajectories are identified based on the predicted historical or future trajectories of the target. These trajectories distinguish different types of trajectories. Based on the prediction results and the identification, the target's motion trajectory is plotted as a 3D graph. The 3D motion trajectory graph typically consists of the target's position points on the X, Y, and Z axes, and its movement path is updated in real time as time progresses. The 3D motion trajectory graph not only displays the target's motion trajectory but also clearly shows the relative positions between targets, their speeds, and potential trajectory conflict areas. Through this visualization, operators or automated systems can intuitively understand the target's movement.

[0033] By predicting multiple trajectories and performing collision detection, potential collision risks can be identified in advance, and corresponding control decisions can be made. By combining motion back-analysis and 3D trajectory map construction, the motion behavior of the target can be accurately analyzed and the path planning can be optimized to ensure stable tracking of the target.

[0034] Furthermore, this application also includes the following steps: A dynamic obstacle prediction factor is introduced into the three-dimensional motion trajectory map to construct a four-dimensional search space; gimbal field of view constraints are extracted based on the gimbal, and a motion feasibility assessment is performed on the four-dimensional search space according to the gimbal field of view constraints to generate a motion feasibility score; multiple executable paths are identified based on the motion feasibility score of the three-dimensional motion trajectory map; the multiple executable paths are traversed and simulated for screening to determine the target tracking path.

[0035] Specifically, the presence of dynamic obstacles during target movement affects path planning and target tracking. Based on information such as the position, velocity, and acceleration of dynamic obstacles (e.g., other vehicles, pedestrians) in the target's surrounding environment, the future dynamic behavior and impact of these obstacles are calculated and predicted, generating dynamic obstacle prediction factors. These dynamic obstacle prediction factors are then introduced into the constructed 3D motion trajectory map, creating a four-dimensional search space. This four-dimensional search space adds a time dimension to the 3D space, forming a four-dimensional space. By incorporating time into spatial planning, the four-dimensional search space accurately reflects the changes in the target and its surrounding environment in the dynamic environment, improving the accuracy of path planning.

[0036] Gimbals have a certain field of view and motion capabilities. Therefore, it is necessary to assess whether the target's trajectory is within the gimbal's trackable field of view, based on the gimbal's actual adjustable angle and range. Gimbal field of view constraints refer to the spatial area limited by the gimbal's field of view and motion capabilities (such as rotation angle and speed). Taking into account the gimbal's rotation speed, rotation angle, and target acquisition capabilities, the assessment evaluates whether the target's path meets these conditions.

[0037] Based on the gimbal's field of view constraints, the feasibility of movement is assessed for each point in the four-dimensional search space, generating a movement feasibility score. Based on the movement feasibility score, the three-dimensional motion trajectory is marked, identifying multiple executable paths. A score is calculated for each path based on the movement feasibility score; a higher score indicates stronger path feasibility. Multiple paths with higher scores are selected as executable paths for further screening.

[0038] All executable paths are traversed, and further verification is achieved through simulation. Using simulation software, a simulation environment is set up based on the target's dynamic trajectory data, gimbal parameters (such as the gimbal's rotation angle range, maximum rotation speed, and energy consumption characteristics), and environmental information (such as dynamic / static obstacles). This ensures the virtual environment realistically reflects dynamic obstacles, gimbal response, and the target tracking process in complex scenarios. During simulation, key evaluation criteria need to be defined, including path length and gimbal energy consumption. Multiple executable paths are simulated, and each path is evaluated according to the key evaluation criteria. Considering factors such as time, energy consumption, and accuracy, the optimal balance point is sought, and a score is obtained. Based on the simulation results, if a path cannot achieve a balance between energy consumption and time, some parameters of that path are optimized (such as changing the gimbal's rotation speed or adjusting the target prediction algorithm) until the optimal solution is found. After evaluating multiple paths, the optimal solution that satisfies both the shortest path and balanced gimbal energy consumption is selected as the target tracking path.

[0039] By introducing dynamic obstacle prediction factors, the behavior of obstacles can be accurately predicted, avoiding potential conflicts. By introducing gimbal field of view constraints, the target can be prevented from being untracked due to being outside the field of view, thus improving the stability and accuracy of target tracking. Through the screening and simulation testing of executable paths, the optimal path is selected to ensure that the target can be stably tracked in complex environments while avoiding obstacles and other interference.

[0040] S200: Set the desired pose parameters of the servo gimbal, and synchronize the target tracking path to the servo gimbal for dynamic updating according to the desired pose parameters to obtain the parameter coupling degree.

[0041] Furthermore, this application S200 includes: Based on the three-dimensional motion trajectory map, a mobile platform coordinate system for the target is constructed, and a gimbal base coordinate system for the servo gimbal is constructed based on the four-dimensional search space. The mobile platform coordinate system and the gimbal base coordinate system are spatially transformed to construct a transformation relationship matrix. Multiple path points are extracted by traversing the target tracking path, and the multiple path points are transformed according to the transformation relationship matrix to set the initial pose parameters of the servo gimbal. The mechanical limit constraints of the servo gimbal are extracted to construct a feasible pose parameter space for the servo gimbal, and the initial pose parameters are synchronized to the feasible pose parameter space to determine the desired pose parameters.

[0042] Furthermore, this application also includes the following steps: The target's attitude is analyzed based on the mobile platform coordinate system, and a homogeneous transformation matrix is ​​constructed. Installation offset analysis is performed based on the gimbal base coordinate system, and a gimbal kinematic chain is constructed. Based on the target tracking path, the path point velocity vector is introduced to perform coordinate system rotation transformation compensation, and the transformation relationship matrix is ​​constructed.

[0043] Specifically, based on the 3D motion trajectory diagram, a coordinate system for the moving platform where the target is located is constructed. This coordinate system is typically set at a fixed point on the moving platform, with the coordinate axes aligned with the platform's motion direction. The moving platform coordinate system is built upon the target's motion path; its origin is usually located at a fixed point on the moving platform, and its direction is consistent with the platform's motion direction. It is used to represent the target's spatial position and motion state. Based on the four-dimensional search space, considering the gimbal's motion range, viewing angle, and changes over time, a gimbal base coordinate system is constructed. This gimbal base coordinate system is closely related to the gimbal's field of view and mechanical structure. The origin is typically set at the center of the gimbal, and the coordinate axes point towards the gimbal's line of sight.

[0044] A spatial transformation is performed between the target's mobile platform coordinate system and the gimbal's base coordinate system, constructing a transformation matrix to convert coordinates in the target's coordinate system to coordinates in the gimbal's coordinate system. The gimbal can then adjust its line of sight for alignment based on the target's position in the platform coordinate system. Specifically, attitude analysis is performed on the target based on the mobile platform coordinate system. By analyzing the target's motion state in the mobile platform coordinate system, the target's attitude (i.e., orientation) is represented using quaternions. The target's rotation and position changes can be expressed using a homogeneous transformation matrix, which includes both the target's rotation and position changes. Quaternions avoid the singularities that may occur with traditional Euler angles and rotation matrices during continuous rotations. For example, if the target rotates from one coordinate system to another, a quaternion is used to represent the rotation angle, and the target's translational changes are added to the homogeneous transformation matrix as translation vectors. Target attitude analysis refers to analyzing the target's orientation (or facing direction) relative to a reference coordinate system using mathematical methods. Attitude is typically represented by rotation angles or quaternions, helping to determine the target's orientation in space. Quaternions are a mathematical representation used to describe rotations in space. Compared to Euler angles and rotation matrices, quaternions avoid gimbal lock and have better computational efficiency. Homogeneous transformation matrices are used to transform a point from one coordinate system to another. They can represent rotation and translation simultaneously, allowing a single matrix to handle both translation and rotation.

[0045] Installation offset analysis is performed based on the gimbal's base coordinate system. The gimbal's base coordinate system may have a certain installation offset relative to the mobile platform coordinate system; therefore, it is necessary to analyze this offset and construct the gimbal's kinematic chain. The gimbal kinematic chain describes the spatial relationship from the base coordinate system to the gimbal's end effector (such as a camera or sensor), understanding how the gimbal moves from the center of the base coordinate system to its actuator end effector. For example, assuming the gimbal starts from a fixed coordinate system and is mounted on a mobile platform, the kinematic chain describes how the angle of each joint affects the motion of the end effector based on the movement of each joint of the gimbal. The kinematic chain typically consists of multiple matrix products, each matrix corresponding to a moving part of the gimbal. Using the chain rule, the spatial position of the gimbal's end effector can be calculated from the gimbal's base coordinate system.

[0046] In the target tracking path, considering the field-of-view offset error caused by the platform's own motion, rotational transformation compensation is performed on the target path points. By introducing the velocity vectors of the path points and updating the transformation matrix in real time, errors caused by platform motion are corrected, ensuring that the gimbal's field of view always remains at the correct target position. The path point velocity vector refers to the direction and magnitude of the velocity of each path point as the target moves along the path during target tracking, used to describe the instantaneous velocity of the target at a specific time. Coordinate system rotation transformation compensation is used to eliminate the field-of-view offset error caused by platform motion (such as platform rotation or displacement). Errors are corrected by updating the rotation or transformation matrix in real time, ensuring accurate field-of-view tracking.

[0047] Based on the pathpoint velocity vectors and the target's motion state, a transformation matrix is ​​calculated and updated in real time to transform pathpoints in the mobile platform coordinate system to the gimbal base coordinate system. This matrix allows the gimbal to effectively adjust its attitude, eliminating field-of-view offset errors caused by platform motion. The transformation matrix parameters are updated in real time, ensuring that the pathpoint transformation's timeliness error is always less than 1 millisecond.

[0048] The target tracking path is traversed, and multiple path points are extracted to represent the target's position at different time points, forming the target's movement trajectory. When extracting path points, the path can be discretized according to the time step of the target's motion to ensure that there is a path point at each time step. Assuming the target moves along a curve in three-dimensional space, the path point can be the target's position at each time scale; for example, the target's position at t=0 is (1,2,3), the target's position at t=1 is (1.5,2.5,3.5), and so on.

[0049] By transforming the relationship matrix, the extracted path points are transformed from the target coordinate system to the gimbal coordinate system. The transformation process includes rotation and translation operations to ensure that the gimbal can accurately adjust its position and attitude according to the target's location. Based on the transformation results of the path points, the initial pose parameters of the gimbal are set, including the gimbal's initial position (displacement vector) and orientation (rotation matrix or quaternion). These serve as reference values ​​for the servo gimbal and determine its initial attitude. The initial pose parameters of the servo gimbal refer to the gimbal's initial position and orientation, usually represented by a rotation matrix or quaternion. The displacement part is represented by a translation vector, which is the desired pose parameter set of the gimbal's line-of-sight center.

[0050] Analyzing the physical structure of the gimbal, the mechanical constraints on its movement are extracted, including hardware limitations such as rotation angle, pitch angle, and lateral offset, which determine the gimbal's range of motion. Based on these mechanical constraints, a feasible pose parameter space for the gimbal is constructed, defining the set of all legal poses that the gimbal can execute. For example, the feasible pose parameter space is a set of rotation angles between [-90°, +90°] and pitch angles between [-45°, +45°].

[0051] The initial pose parameters are synchronized to the gimbal's feasible pose parameter space to ensure that the gimbal's initial position and orientation are within its feasible range. If the initial parameters are not within the feasible range, they need to be corrected to conform to the gimbal's motion constraints. Based on the extracted path points and the feasible pose parameter space, the desired pose parameters of the gimbal are determined to guide the servo gimbal in real-time adjustments according to the target path. The desired pose parameters refer to the final position and attitude that the gimbal should achieve based on factors such as the target tracking path and transformation matrix; they represent the ideal position and orientation that the gimbal should achieve at a certain moment.

[0052] By combining homogeneous transformation matrices and quaternion methods, the motion state of the target can be accurately represented, and errors caused by platform motion can be eliminated, ensuring that the gimbal can always accurately track the target, maintaining tracking stability even when the platform undergoes significant movement. By constructing the gimbal's kinematic chain, the movement of each joint ensures precise control of the gimbal's end effector, allowing the gimbal to more flexibly adjust its position and attitude to handle complex target tracking tasks. By transforming path points to the gimbal coordinate system and setting initial pose parameters, accurate target tracking by the gimbal is ensured. Mechanical constraints on the gimbal ensure that its movement does not exceed the hardware's capabilities, avoiding damage or tracking failure due to hardware overload. Based on the target's real-time position and path points, the gimbal's pose parameters are dynamically adjusted, ensuring that the gimbal can flexibly adapt to changes in the target and achieve more precise target tracking.

[0053] Furthermore, this application also includes the following steps: The target tracking path is synchronized to the servo gimbal for analysis according to the desired pose parameters to determine the three-dimensional waypoint sequence of the target tracking path. The three-dimensional waypoint sequence includes path curvature features and timestamp information. Based on the path curvature features and the timestamp information, a decoupling evaluation is performed, and a parameter coupling index is set. Parameter coupling is calculated according to the parameter coupling index to obtain the parameter coupling degree.

[0054] Specifically, based on the desired pose parameters, the target tracking path is synchronized to the servo gimbal for analysis. Each key point (i.e., waypoint) on the target path corresponds to a specific motion command from the gimbal. The servo gimbal then calculates its adjustment parameters based on the target's 3D trajectory and desired position. The 3D waypoint sequence is a set of all key locations traversed in 3D space along the target tracking path. Each waypoint contains not only position coordinates but also timestamp information and motion characteristics (such as velocity and acceleration).

[0055] Based on multiple waypoints along the path, the curvature characteristics of the path are calculated, and each waypoint is labeled with timestamp information. Curvature is defined by the rate of change between waypoints, while timestamps assign a time attribute to each waypoint, describing the target's motion state at different points on the path. Path curvature characteristics refer to the degree of bending of the target path. Curvature is a physical quantity describing the rate of change of the path curve, usually represented by the change in angle between waypoints. Higher curvature means a more curved path, while lower curvature means a more straight path. Timestamp information is a time stamp associated with each waypoint, indicating the time position of that waypoint during the target's movement.

[0056] Based on the curvature characteristics and timestamp information of the path, a decoupling evaluation is performed to assess the synchronization degree between the target tracking path and the gimbal. If the target's motion changes significantly (e.g., drastic changes in curvature or rapid changes in speed), it will lead to inaccurate or lagging gimbal tracking. By setting a parameter coupling degree index, the gimbal's response capability can be evaluated to see if it can keep up with the target's changes in a timely manner. The parameter coupling degree index is used to quantify the relationship between path characteristics (such as path curvature) and the response of the servo gimbal control system. It reflects the coordination between the target path and the gimbal control system. The higher the coupling degree, the better the synchronization and target tracking can be performed.

[0057] Based on the set parameter coupling index, parameter coupling calculations are performed. Combining the target path curvature, timestamp information, and the gimbal's response characteristics, the matching degree between the gimbal and the target path is evaluated, indicating the optimization direction when controlling the gimbal for that path. If the target path has a large curvature and the intervals between timestamps are short, the gimbal may need to make large angle adjustments in a short time. The parameter coupling calculation adjusts the gimbal control strategy based on this information to ensure that the gimbal can track the target in a timely and accurate manner. Parameter coupling calculations are performed based on path characteristics (such as curvature, timestamps, etc.) and the responsiveness of the servo gimbal control (such as rotation angle, speed, etc.) to evaluate the degree of cooperation between the gimbal and the target path, thereby determining the priority and optimization strategy of gimbal control.

[0058] By analyzing the three-dimensional waypoint sequence of the target tracking path and combining the path curvature features and timestamp information, the tracking accuracy of the gimbal can be improved. Discoupling evaluation and parameter coupling degree calculation can effectively evaluate the adaptability of the gimbal to the target path. Especially for complex paths or dynamic environments, the gimbal's control strategy can be optimized by adjusting the coupling degree index, thereby improving the stability of target tracking.

[0059] S300: Based on the parameter coupling degree, a path planning is performed to generate a collaborative control instruction set. The follow-up gimbal is driven to perform bidirectional control according to the collaborative control instruction set, and the target is tracked and controlled through the bidirectional control instructions.

[0060] Furthermore, this application S300 includes: Dynamic coupling analysis is performed based on the parameter coupling degree to determine the dynamic coupling degree index. The coupling state feature vector is extracted based on the dynamic coupling degree index. The coupling state feature vector is weighted and fused to generate a collaborative control instruction set. The servo gimbal is subjected to bidirectional data interaction feedback according to the collaborative control instruction set to generate composite control parameters. Dynamic gain is applied based on the composite control parameters to construct the bidirectional control instructions for target tracking control.

[0061] Specifically, based on the parameter coupling degree, dynamic coupling analysis is performed to analyze how the interdependence and influence between gimbal pose parameters change during the tracking process due to the dynamic changes in the target's motion state. For example, in a gimbal control system, the gimbal angle and the position of the moving platform may have a certain coupling relationship. By analyzing their dynamic coupling degree, it is determined whether the control of these two parameters is closely related and how to adjust the control strategy to cope with changes. Based on the dynamic coupling analysis results, a dynamic coupling degree index is determined to evaluate the gimbal coupling characteristics, reflecting the changes and interactions of control parameters in the dynamic process. Based on the dynamic coupling degree index, a coupling state feature vector is extracted from the coupling analysis results, which contains multiple feature values ​​to reflect the coupling characteristics of the gimbal at the current moment, including the target's motion characteristics (such as velocity, acceleration, etc.) and the gimbal's response characteristics (such as angle change, motion speed, etc.), to quantify the coordination between the target and the gimbal.

[0062] Based on the coupling state feature vector, weighted fusion is performed. This involves allocating weights to the path planning of the mobile platform and the position adjustment of the servo gimbal based on the coupling relationship characteristics of various control parameters. The coupling weights of path planning and gimbal control are adjusted in real time to generate coordinated commands, including multiple control objectives, and to coordinate the motion and attitude adjustments of various subsystems (such as the mobile platform and gimbal). Control commands and feedback information are transmitted in real time through bidirectional data interaction feedback. Based on the feedback results, the control strategy is adjusted to generate composite control parameters. Bidirectional data interaction feedback refers to the bidirectional flow of control signals and feedback signals in the control system. Commands are transmitted in the forward channel, while feedback information is transmitted in the reverse channel, ensuring that the control strategy can be adjusted based on the feedback information. Feedback information refers to the data returned by the gimbal after the command is executed.

[0063] Composite control parameters are comprehensive control parameters obtained by fusing multiple control parameters and feedback signals. They include the outputs of multiple independent control systems and are combined to optimize overall behavior. For example, a mobile platform (such as an autonomous vehicle) adjusts its motion path based on gimbal status data, while the gimbal adjusts its angle based on feedback from the platform; the two coordinate through bidirectional data interaction. Based on composite control parameters, dynamic gain is used to optimize the control system's response. Feedforward gain and feedback compensation vector are dynamically adjusted to achieve optimal target tracking control. If environmental disturbances occur while tracking the target, the dynamic gain mechanism can enhance or reduce the strength of the control signal to ensure that control accuracy is not affected. Through dynamic coupling analysis and weighted fusion, the relationship between various control parameters is precisely coordinated, improving the coordination between the mobile platform and the gimbal. This ensures stable operation in dynamic environments under the coordination of path planning and the servo gimbal. Through weighted fusion and dynamic gain, the control strategy is adjusted according to environmental changes, ensuring stable operation even when encountering disturbances or unforeseen circumstances.

[0064] Furthermore, this application also includes the following steps: Based on the weighted fusion of the coupled state feature vectors, path correction and position adjustment quantities are generated; dynamic priority allocation is performed according to the path correction and position adjustment quantities to determine the first execution weight and the second execution weight; control analysis is performed according to the first execution weight and the path correction quantity to generate a mobile platform path correction instruction; control analysis is performed according to the second execution weight and the position adjustment quantity to generate a gimbal pose adjustment instruction; the mobile platform path correction instruction and the gimbal pose adjustment instruction are integrated to generate a collaborative control instruction set.

[0065] Specifically, target tracking is performed autonomously by a mobile platform. Based on coupled state feature vectors, the motion state of the mobile platform (such as an autonomous vehicle) and the response state of the gimbal are weighted and fused. Multiple input information (such as the mobile platform's velocity, acceleration, gimbal's angular velocity, and position deviation) are combined to generate path correction and position adjustment quantities. The path correction quantity represents the deviation between the target trajectory and the desired trajectory, while the position adjustment quantity represents the deviation between the gimbal's current attitude and the desired attitude.

[0066] Dynamic priority allocation is performed based on the magnitude of path correction and position adjustment. Generally, a larger path correction indicates a significant deviation in the mobile platform's path, requiring priority for path correction; conversely, a larger position adjustment indicates a significant deviation in the gimbal's attitude, necessitating priority for pose adjustment. Prioritization ensures that the most critical adjustment tasks are executed first. Execution weights represent the priority and importance of control tasks. Depending on the control objectives, tasks may have different execution weights to ensure that critical tasks are completed first.

[0067] Based on the first execution weight and the path correction amount, a path correction analysis is performed on the mobile platform, generating control commands to adjust the platform's speed, acceleration, and direction, enabling it to move along the correct path. For example, suppose the mobile platform deviates significantly from the predetermined path at a certain moment. Based on this correction amount, the platform's path is adjusted, and the commands guide it to move along the new path.

[0068] Based on the second execution weight and position adjustment amount, the gimbal's pose adjustment analysis is performed, generating gimbal control commands to adjust the gimbal's rotation angle, attitude, or field of view direction so that it can be aligned with the target. For example, if the gimbal's field of view deviates from the target, the position adjustment amount is large. The command will instruct the gimbal to rotate a certain angle to ensure that the gimbal can be re-aligned with the target and maintain accurate tracking.

[0069] By integrating mobile platform path correction commands with gimbal pose adjustment commands, a collaborative control command set is generated. This set contains multiple control commands that work together on various components (such as the gimbal and mobile platform) to ensure accurate tracking of the mobile platform (e.g., an autonomous vehicle). For example, if the target's movement path changes, the path correction command and the gimbal pose adjustment command work together to ensure the target adjusts its path and direction, and the gimbal adjusts its field of view in real time, both working together to achieve accurate target tracking. The collaborative control command set contains multiple control commands used to simultaneously adjust the behavior of multiple components (such as the gimbal and the target path), enabling all components to work in coordination and optimizing overall performance.

[0070] By weighted fusion of path correction and position adjustment, the deviation between the target and the gimbal can be accurately assessed and calibrated in a timely manner, maintaining accurate target tracking in dynamic environments and reducing path deviation and gimbal field of view errors. The dynamic priority allocation mechanism ensures that the most critical tasks are executed first during multi-tasking, quickly adjusting the mobile platform path and gimbal pose to ensure efficient and accurate tracking. The collaborative control instruction set integrates the control tasks of the mobile platform and the gimbal, ensuring that they work together, avoiding conflicts or incoordination between different parts, and adjusting the gimbal and target path in real time to avoid asynchronous or redundant control instructions.

[0071] Furthermore, this application also includes the following steps: The collaborative control instruction set is analyzed in real time to separate and generate the mobile platform path correction instruction and the gimbal pose adjustment instruction. Based on the gimbal pose adjustment instruction, the follow-up gimbal is adjusted and recorded to obtain gimbal status data. A dual-channel control architecture is constructed, including a forward channel and a reverse channel. The mobile platform path correction instruction is transmitted through the forward channel, and the gimbal status data is fed back through the reverse channel to generate composite control parameters. Forward coupling analysis is performed based on the composite control parameters to generate a feedforward gain matrix. Reverse coupling analysis is performed based on the composite control parameters to generate a feedback compensation vector. The control loop gain is calculated based on the feedforward gain matrix and the feedback compensation vector to construct the bidirectional control instruction.

[0072] Specifically, the collaborative control instruction set is analyzed in real time, separating the mobile platform path correction instructions from the gimbal pose adjustment instructions. The path correction instructions focus on adjusting the target's motion trajectory, while the gimbal pose adjustment instructions focus on adjusting the gimbal's position and attitude. Based on the generated gimbal pose adjustment instructions, the gimbal is actually adjusted, and its state data (such as position, angle, and speed) is recorded. For example, if the gimbal needs to adjust its angle to align with the target, the instruction will control the gimbal to rotate by a certain angle, and the gimbal's state data (such as current angle and rotation speed) will be recorded in real time. The gimbal pose adjustment instructions are used to adjust the gimbal's position and attitude, ensuring that the gimbal can correctly align with the target by controlling its angle and position.

[0073] A dual-channel control architecture is constructed, including a forward channel and a reverse channel, used for control and feedback functions respectively. The forward channel transmits control commands from the controller to the actuators, while the reverse channel transmits feedback information from the actuators to the controller. The forward channel is used to transmit path correction commands for the mobile platform, and the reverse channel is used to provide feedback on the gimbal's status data.

[0074] The mobile platform transmits path correction commands via the forward channel and feeds back gimbal status data via the reverse channel, generating composite control parameters. These parameters combine the path correction commands and the gimbal's real-time status data for more precise control. The composite control parameters are control quantities generated by combining control commands from the forward channel and status data from the reverse channel.

[0075] Forward coupling analysis is performed based on the composite control parameters to generate a feedforward gain matrix. The gain of the control command is analyzed based on the content of the composite control parameters, and a feedforward gain matrix is ​​generated. The feedforward gain matrix is ​​used for pre-control based on inputs (such as path correction commands). For example, if a path correction command instructs the target to adjust its direction of travel, the feedforward gain matrix will calculate how much force the platform needs to apply and how to change the travel angle.

[0076] Based on composite control parameters, reverse coupling analysis is performed. By analyzing the feedback data from the reverse channel, a feedback compensation vector is generated. This vector is used to adjust control commands according to the feedback data to eliminate errors or optimize control. For example, if there is a deviation between the current viewing angle of the gimbal and the desired viewing angle, the feedback compensation vector will analyze the deviation to correct the gimbal's movement, ensuring that the gimbal is adjusted to the correct position.

[0077] Based on the feedforward gain matrix and feedback compensation vector, the control loop gain is calculated, and the system's response speed and stability are optimized by adjusting the gain value. The control loop gain is used to adjust the gain values ​​of the feedforward and feedback signals, generating bidirectional control commands. Combining forward commands and feedback compensation, it can precisely adjust the control parameters of the mobile platform and the gimbal, achieving efficient collaborative control. During task execution, the path of the mobile platform (such as an autonomous vehicle) tracking the target and the field of view of the gimbal are adjusted according to the bidirectional control commands to ensure accurate target tracking.

[0078] By combining a dual-channel control architecture with feedforward and feedback mechanisms, the control strategy can be adjusted in real time according to changes in the target, enhancing response accuracy and stability. The bidirectional data flow between the forward and reverse channels ensures that the mobile platform and the gimbal can work in coordination, optimizing target path tracking and field of view adjustment. Based on forward coupling analysis and reverse coupling analysis, the control gain is dynamically adjusted to generate flexible bidirectional control commands, improving the accuracy of target tracking.

[0079] In summary, the target tracking method based on path planning and servo gimbal coordination provided in this application has the following beneficial effects: By acquiring the target's dynamic trajectory data in real time, a 3D motion trajectory map is constructed for path tracking, obtaining the target tracking path. The desired pose parameters of the servo gimbal are set, and the target tracking path is synchronized to the servo gimbal for dynamic updates according to these parameters, obtaining the parameter coupling degree. Based on the parameter coupling degree, path planning is performed to generate a collaborative control instruction set. The servo gimbal is then driven for bidirectional control according to this command set, enabling target tracking control. In other words, by acquiring the target's dynamic trajectory data in real time, constructing a 3D motion trajectory map, and rapidly responding to the target's high-speed movement, the target's motion mode in complex environments is determined. The desired pose parameters of the gimbal are set and dynamically updated to maintain stable alignment with the target. Path planning based on the parameter coupling degree improves the accuracy and robustness of target tracking in complex scenarios. Closed-loop control reduces tracking errors and balances system energy consumption.

[0080] Example 2: Based on the same inventive concept as the target tracking method based on path planning and gimbal coordination in Example 1, this application also provides a target tracking system based on path planning and gimbal coordination. Please refer to the appendix. Figure 2 The target tracking system based on path planning and servo gimbal coordination includes: The path tracking module 11 is used to acquire the dynamic trajectory data of the target in real time, construct a three-dimensional motion trajectory map for path tracking, and obtain the target tracking path; the dynamic update module 12 is used to set the desired pose parameters of the servo gimbal, and synchronize the target tracking path to the servo gimbal according to the desired pose parameters for dynamic update, thereby obtaining the parameter coupling degree; the bidirectional control module 13 is used to generate a collaborative control instruction set based on the parameter coupling degree, drive the servo gimbal for bidirectional control according to the collaborative control instruction set, and perform target tracking control through bidirectional control instructions.

[0081] Furthermore, the path tracking module 11 in the target tracking system based on path planning and servo gimbal coordination is also used for: A multimodal sensor array is used to collect data on the target in real time to obtain a multimodal dataset, which includes lidar point cloud data, target visual depth data, and angular velocity data. The lidar point cloud data, target visual depth data, and angular velocity data are spatiotemporally aligned to determine a three-dimensional trajectory point sequence. The target motion state is predicted according to the three-dimensional trajectory point sequence to obtain a target predicted trajectory sequence. Trajectory conflict detection is performed based on the target predicted trajectory sequence to determine multiple predicted trajectory data. Motion backward inference is performed based on the multiple predicted trajectory data, and the inference results are labeled to construct the three-dimensional motion trajectory map.

[0082] Furthermore, the path tracking module 11 in the target tracking system based on path planning and servo gimbal coordination is also used for: A dynamic obstacle prediction factor is introduced into the three-dimensional motion trajectory map to construct a four-dimensional search space; gimbal field of view constraints are extracted based on the gimbal, and a motion feasibility assessment is performed on the four-dimensional search space according to the gimbal field of view constraints to generate a motion feasibility score; multiple executable paths are identified based on the motion feasibility score of the three-dimensional motion trajectory map; the multiple executable paths are traversed and simulated for screening to determine the target tracking path.

[0083] Furthermore, the dynamic update module 12 in the target tracking system based on path planning and servo gimbal coordination is also used for: Based on the three-dimensional motion trajectory map, a mobile platform coordinate system for the target is constructed, and a gimbal base coordinate system for the servo gimbal is constructed based on the four-dimensional search space. The mobile platform coordinate system and the gimbal base coordinate system are spatially transformed to construct a transformation relationship matrix. Multiple path points are extracted by traversing the target tracking path, and the multiple path points are transformed according to the transformation relationship matrix to set the initial pose parameters of the servo gimbal. The mechanical limit constraints of the servo gimbal are extracted to construct a feasible pose parameter space for the servo gimbal, and the initial pose parameters are synchronized to the feasible pose parameter space to determine the desired pose parameters.

[0084] Furthermore, the dynamic update module 12 in the target tracking system based on path planning and servo gimbal coordination is also used for: The target's attitude is analyzed based on the mobile platform coordinate system, and a homogeneous transformation matrix is ​​constructed. Installation offset analysis is performed based on the gimbal base coordinate system, and a gimbal kinematic chain is constructed. Based on the target tracking path, the path point velocity vector is introduced to perform coordinate system rotation transformation compensation, and the transformation relationship matrix is ​​constructed.

[0085] Furthermore, the dynamic update module 12 in the target tracking system based on path planning and servo gimbal coordination is also used for: The target tracking path is synchronized to the servo gimbal for analysis according to the desired pose parameters to determine the three-dimensional waypoint sequence of the target tracking path. The three-dimensional waypoint sequence includes path curvature features and timestamp information. Based on the path curvature features and the timestamp information, a decoupling evaluation is performed, and a parameter coupling index is set. Parameter coupling is calculated according to the parameter coupling index to obtain the parameter coupling degree.

[0086] Furthermore, the bidirectional control module 13 in the target tracking system based on path planning and servo gimbal coordination is also used for: Dynamic coupling analysis is performed based on the parameter coupling degree to determine the dynamic coupling degree index. The coupling state feature vector is extracted based on the dynamic coupling degree index. The coupling state feature vector is weighted and fused to generate a collaborative control instruction set. The servo gimbal is subjected to bidirectional data interaction feedback according to the collaborative control instruction set to generate composite control parameters. Dynamic gain is applied based on the composite control parameters to construct the bidirectional control instructions for target tracking control.

[0087] Furthermore, the bidirectional control module 13 in the target tracking system based on path planning and servo gimbal coordination is also used for: Based on the weighted fusion of the coupled state feature vectors, path correction and position adjustment quantities are generated; dynamic priority allocation is performed according to the path correction and position adjustment quantities to determine the first execution weight and the second execution weight; control analysis is performed according to the first execution weight and the path correction quantity to generate a mobile platform path correction instruction; control analysis is performed according to the second execution weight and the position adjustment quantity to generate a gimbal pose adjustment instruction; the mobile platform path correction instruction and the gimbal pose adjustment instruction are integrated to generate a collaborative control instruction set.

[0088] Furthermore, the bidirectional control module 13 in the target tracking system based on path planning and servo gimbal coordination is also used for: The collaborative control instruction set is analyzed in real time to separate and generate the mobile platform path correction instruction and the gimbal pose adjustment instruction. Based on the gimbal pose adjustment instruction, the follow-up gimbal is adjusted and recorded to obtain gimbal status data. A dual-channel control architecture is constructed, including a forward channel and a reverse channel. The mobile platform path correction instruction is transmitted through the forward channel, and the gimbal status data is fed back through the reverse channel to generate composite control parameters. Forward coupling analysis is performed based on the composite control parameters to generate a feedforward gain matrix. Reverse coupling analysis is performed based on the composite control parameters to generate a feedback compensation vector. The control loop gain is calculated based on the feedforward gain matrix and the feedback compensation vector to construct the bidirectional control instruction.

[0089] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Figure 1 The target tracking method and specific examples based on path planning and gimbal coordination in Example 1 are also applicable to the target tracking system based on path planning and gimbal coordination in this example. Through the foregoing detailed description of the target tracking method based on path planning and gimbal coordination, those skilled in the art can clearly understand the target tracking system based on path planning and gimbal coordination in this example. Therefore, for the sake of brevity, it will not be described in detail here.

[0090] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0091] Obviously, those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A target tracking method based on path planning and servo gimbal coordination, characterized in that, include: Real-time acquisition of dynamic trajectory data of the target, construction of a three-dimensional motion trajectory map for path tracking, and acquisition of the target tracking path; Set the desired pose parameters of the servo gimbal, and synchronize the target tracking path to the servo gimbal according to the desired pose parameters for dynamic updating, thereby obtaining the parameter coupling degree; Based on the parameter coupling degree, path planning is performed to generate a collaborative control instruction set. The follow-up gimbal is driven to perform bidirectional control according to the collaborative control instruction set, and the target is tracked and controlled through bidirectional control instructions.

2. The target tracking method based on path planning and servo gimbal coordination as described in claim 1, characterized in that, Real-time acquisition of the target's dynamic trajectory data, and construction of a 3D motion trajectory map, including: The target is collected in real time by a multimodal sensor array to obtain a multimodal dataset, which includes lidar point cloud data, target visual depth data, and angular velocity data. The lidar point cloud data, the target visual depth data, and the angular velocity data are spatiotemporally aligned to determine a three-dimensional trajectory point sequence; The target motion state is predicted according to the three-dimensional trajectory point sequence to obtain the target predicted trajectory sequence. Trajectory conflict detection is performed based on the target predicted trajectory sequence to determine multiple predicted trajectory data. Based on the multiple predicted trajectory data, motion inverse deduction is performed, and the deduction results are labeled to construct the three-dimensional motion trajectory map.

3. The target tracking method based on path planning and servo gimbal coordination as described in claim 1, characterized in that, Construct a 3D motion trajectory map for path tracking to obtain the target tracking path, including: A dynamic obstacle prediction factor is introduced into the three-dimensional motion trajectory map to construct a four-dimensional search space; Based on the gimbal's field of view constraints extracted from the servo gimbal, a movement feasibility assessment is performed on the four-dimensional search space according to the gimbal's field of view constraints, and a movement feasibility score is generated. Based on the mobility feasibility score, multiple executable paths are determined for the three-dimensional motion trajectory map identifier; The target tracking path is determined by traversing the multiple executable paths and performing simulation filtering.

4. The target tracking method based on path planning and servo gimbal coordination as described in claim 3, characterized in that, Set the desired pose parameters of the servo gimbal, including: Based on the three-dimensional motion trajectory diagram, a mobile platform coordinate system for the target is constructed, and a gimbal base coordinate system for the servo gimbal is constructed based on the four-dimensional search space. The coordinate system of the mobile platform is spatially transformed from the coordinate system of the gimbal base to construct a transformation relationship matrix; The target tracking path is traversed to extract multiple path points, and the multiple path points are transformed according to the transformation relationship matrix to set the initial pose parameters of the servo gimbal. Extract the mechanical limit constraints of the servo gimbal to construct a feasible pose parameter space for the servo gimbal, synchronize the initial pose parameters to the feasible pose parameter space, and determine the desired pose parameters.

5. The target tracking method based on path planning and servo gimbal coordination as described in claim 4, characterized in that, The mobile platform coordinate system and the gimbal base coordinate system are spatially transformed to construct a transformation relationship matrix, including: Based on the coordinate system of the mobile platform, the attitude of the target is analyzed, and a homogeneous transformation matrix is ​​constructed. Based on the gimbal base coordinate system, an installation offset analysis is performed to construct the gimbal kinematic chain; Based on the target tracking path, the velocity vector of the path point is introduced to perform coordinate system rotation transformation compensation, and the transformation relationship matrix is ​​constructed.

6. The target tracking method based on path planning and servo gimbal coordination as described in claim 1, characterized in that, The target tracking path is synchronized to the servo gimbal for dynamic updating according to the desired pose parameters to obtain the parameter coupling degree, including: The target tracking path is synchronized to the servo gimbal for analysis according to the desired pose parameters to determine the three-dimensional waypoint sequence of the target tracking path. The three-dimensional waypoint sequence includes path curvature features and timestamp information. Based on the path curvature characteristics and the timestamp information, a decoupling evaluation is performed, and a parameter coupling index is set. The parameter coupling degree is obtained by performing parameter coupling calculations according to the parameter coupling degree index.

7. The target tracking method based on path planning and servo gimbal coordination as described in claim 1, characterized in that, Based on the parameter coupling degree, path planning is performed to generate a collaborative control instruction set. The servo gimbal is then driven for bidirectional control according to this instruction set. Target tracking control is achieved through bidirectional control commands, including: Dynamic coupling analysis is performed based on the aforementioned parameter coupling degree to determine the dynamic coupling degree index, and coupling state feature vectors are extracted based on the dynamic coupling degree index. A collaborative control instruction set is generated by weighted fusion based on the coupling state feature vectors. The servo gimbal is subjected to bidirectional data interaction feedback according to the collaborative control instruction set, composite control parameters are generated, dynamic gain is applied based on the composite control parameters, and the bidirectional control instructions are constructed to track and control the target.

8. The target tracking method based on path planning and servo gimbal coordination as described in claim 7, characterized in that, Based on the weighted fusion of the coupled state feature vectors, a collaborative control instruction set is generated, including: Based on the weighted fusion of the coupling state feature vector, path correction amount and position adjustment amount are generated; Dynamic priority allocation is performed based on the path correction amount and the position adjustment amount to determine the first execution weight and the second execution weight; Based on the first execution weight and the path correction amount, control analysis is performed to generate a mobile platform path correction instruction. Control analysis is performed based on the second execution weight and the position adjustment amount to generate gimbal pose adjustment instructions; The mobile platform path correction command and the gimbal pose adjustment command are integrated to generate a collaborative control command set.

9. The target tracking method based on path planning and servo gimbal coordination as described in claim 8, characterized in that, According to the collaborative control instruction set, the servo gimbal undergoes bidirectional data interaction feedback to generate composite control parameters. Based on these composite control parameters, dynamic gain is applied, and the bidirectional control instructions are constructed to track and control the target, including: The collaborative control instruction set is analyzed in real time to separate and generate the mobile platform path correction instruction and the gimbal pose adjustment instruction; The gimbal is adjusted and recorded based on the gimbal pose adjustment command to obtain gimbal status data. A dual-channel control architecture is constructed, which includes a forward channel and a reverse channel; The mobile platform path correction command is transmitted through the forward channel, and the gimbal status data is fed back through the reverse channel to generate composite control parameters. Based on the composite control parameters, a forward coupling analysis is performed to generate a feedforward gain matrix; Based on the composite control parameters, a reverse coupling analysis is performed to generate a feedback compensation vector; The control loop gain is calculated based on the feedforward gain matrix and the feedback compensation vector, and the bidirectional control command is constructed.

10. A target tracking system based on path planning and servo gimbal coordination, characterized in that, The steps for implementing the target tracking method based on path planning and servo gimbal coordination according to any one of claims 1 to 9, wherein the target tracking system based on path planning and servo gimbal coordination comprises: The path tracking module is used to acquire the target's dynamic trajectory data in real time, construct a three-dimensional motion trajectory map for path tracking, and obtain the target tracking path; The dynamic update module is used to set the desired pose parameters of the servo gimbal, and to synchronize the target tracking path to the servo gimbal according to the desired pose parameters for dynamic update, thereby obtaining the parameter coupling degree. The bidirectional control module is used to generate a collaborative control instruction set based on the parameter coupling degree, drive the follow-up gimbal to perform bidirectional control according to the collaborative control instruction set, and track and control the target through bidirectional control instructions.

Images