Pan-tilt automatic tracking method and system based on predictive control

Abstract

The invention relates to the technical field of computer vision and automatic control, and discloses an automatic cradle head tracking method and system based on predictive control, and the method comprises the following steps: constructing a scene semantic navigation map which comprises a plurality of semantic anchor points and passable paths connected with the anchor points; acquiring a real-time motion state of a tracked target, and determining a map position on the scene semantic navigation map; performing probability deduction on a plurality of candidate intentions in combination with the target motion state and the scene semantic navigation map to obtain intention probability distribution; and based on the intention probability distribution, generating and executing a control instruction for driving a holder to track the target. According to the method, intention deduction is carried out by constructing the scene semantic map and fusing dynamic prior, predictive cooperative control and map adaptive evolution are realized, tracking delay is reduced, and the tracking success rate of a complex scene is improved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and automatic control technology, and in particular to a method and system for automatic gimbal tracking based on predictive control. Background Technology

[0002] Pan-tilt cameras are widely used in video surveillance of large-scale scenes, such as squares, stations, and parks, due to their flexibility in adjusting the shooting direction and optical zoom. To achieve continuous and uninterrupted monitoring of specific moving targets (such as pedestrians and vehicles), automatic tracking technology has emerged.

[0003] Currently, the automatic tracking technology of PTZ cameras mainly relies on real-time target detection algorithms. The typical technical solution involves continuously detecting the target's position in the video stream, using the target's positional deviation in the image as an error signal for the control system, and then driving the PTZ rotation through feedback control methods such as a proportional-integral-derivative (PID) controller to keep the target centered in the field of view. Some improved solutions build upon this by employing state estimation methods such as Kalman filters to perform short-time linear extrapolation of the target's trajectory, providing a degree of predictive capability.

[0004] While existing technologies can achieve automatic tracking of moving targets to some extent, several shortcomings remain: their control logic is inherently reactive, and tracking performance is limited by system latency. This is because the generation of control commands heavily depends on changes in the target's position. The inherent delays in image acquisition, data transmission, analysis, and gimbal mechanical response cause control commands to consistently lag behind the target's actual motion state. This can easily lead to tracking lag, frequent adjustments, or even loss of tracking when the target is moving at high speed or undergoing sudden changes in direction. Furthermore, their predictive capabilities are significantly limited when dealing with complex scenarios with multiple path choices (such as intersections). This is because simple kinematic models only perpetuate the target's current motion trend, lacking semantic understanding of the scene. They cannot utilize environmental topological constraints (such as paths and obstacles) to eliminate unreasonable predictions, nor do they incorporate historical behavioral statistics (such as frequently used paths) to guide predictions. Finally, the tracking strategies of existing systems are typically static, and their tracking performance does not improve over time. The root cause lies in the lack of an effective closed-loop feedback and learning mechanism. The system cannot quantify and solidify successful tracking experiences or lessons learned from failures into its tracking model, resulting in its inability to adapt to behavioral patterns in specific scenarios. Summary of the Invention

[0005] The purpose of this invention is to provide a gimbal automatic tracking method and system based on predictive control, which solves the problems of tracking lag, inaccurate prediction and easy loss of tracking in complex scenarios caused by the reactive control logic, lack of scene understanding and adaptive learning mechanism of existing gimbal automatic tracking technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] The first aspect of this invention provides a gimbal automatic tracking method based on predictive control. This method constructs an adaptive and evolving scene semantic navigation map, combines the target's real-time motion state with historical behavior data accumulated in the map, performs probabilistic deduction of the target's future intentions, and coordinates the motion and field of view of the gimbal based on the deduction results. Finally, it continuously optimizes the map model through closed-loop feedback to achieve efficient, smooth, and predictive automatic tracking of the target.

[0008] Preferably, the method includes the following steps:

[0009] Step S1: Construct a scene semantic navigation graph. This step provides the system with an initial environmental cognitive structure.

[0010] First, image processing technology is used to analyze the monitored scene, identifying and extracting static elements with interactive potential, which are defined as semantic anchors, such as building entrances / exits, road intersections, stairwells, or elevator entrances. Then, a set of preset dynamic attributes is associated with each semantic anchor for recording and quantifying its historical interaction information, specifically including passage frequency (initially 0) and the most recent passage timestamp (initially empty). Finally, a directed graph, the scene semantic navigation graph, is constructed using all semantic anchors as nodes and the physically accessible paths between anchors as edges. This graph stores the static layout and dynamic behavior patterns of the scene in a structured manner.

[0011] Step S2: Obtain the target's motion state and perform map localization. This step associates the real-time observed target with the previously constructed scene cognition structure:

[0012] The system detects the tracked target in real time from the video stream and calculates its kinematic state in the world coordinate system through coordinate transformation. This state includes at least the current time. Location and speed At the same time, the target's location Mapping onto the scene semantic navigation graph, determining the current path edge and the relative relationships between the upstream and downstream semantic anchor nodes connected to that path, thus completing the graph localization of the target.

[0013] Step S3: Perform target intent probability deduction. This step is the core of achieving predictive control.

[0014] Based on the target's location on the graph, the system retrieves all reachable neighboring semantic anchors and treats the action of going to each neighboring anchor as a candidate intent. For each candidate intent... The system uses a Bayesian inference model to calculate its posterior probability. .

[0015] Specifically, the probability value of the prior term is related to the candidate intent. The dynamic properties of associated semantic anchors (such as frequency of access) are positively correlated. A semantic anchor that has been frequently interacted with in history has a higher prior probability of being an intended target.

[0016] Through this step, the system obtains an intent probability distribution that represents the likelihood of all candidate intents occurring. .

[0017] Step S4: Generate and execute gimbal control commands based on the intent probability distribution. This step transforms the derived intent probability distribution into specific gimbal hardware actions.

[0018] This step can be further broken down into two parallel sub-steps:

[0019] The system first calculates the intent probability distribution. Information entropy This is then converted into an intention confidence level used to quantify the current forecast uncertainty. This calculation is performed using the following formula:

[0020] ;

[0021] In the formula, The probability distribution of the stated intent; The information entropy of the probability distribution of the stated intent; The number of the multiple candidate intentions; The confidence level of the stated intent; This represents the maximum entropy value.

[0022] The system then calculates the intent confidence level. Compare with two preset confidence thresholds, high and low. If If the value is above the high confidence threshold, it indicates that the system is very certain about the target's intention, and the instruction to increase optical zoom is executed; if... If the value is below the low confidence threshold, it indicates that the system is in an uncertain state (such as the target being located at an intersection with multiple high-probability exits). In this case, the system will execute a command to reduce the optical zoom to expand the field of view and ensure that the target is not lost.

[0023] The system is based on the probability distribution of intent. Select the N candidate intentions with the highest probability values. Instantiate an independent motion predictor (such as a constrained Kalman filter) for each selected intention and generate N parallel future trajectory predictions.

[0024] Ultimately, the control commands for panning and tilting the gimbal are generated by weighted fusion of these N predicted trajectories. The weight of each predicted trajectory is determined by normalizing the probability value of its corresponding candidate intent. This fusion method allows the gimbal's movement to smoothly gravitate towards the most probable path while also considering other possibilities, avoiding abrupt changes at decision points.

[0025] Step S5: Perform graph adaptive evolution. This step constitutes the closed-loop learning mechanism of the system.

[0026] During subsequent tracking, once the system verifies, through continuous observation, that the tracked target has ultimately completed its true intent (i.e., the target has actually reached a semantic anchor point), the system will feed this verification result back to the scene semantic navigation graph. Specifically, the system will increment the frequency count of the semantic anchor point associated with the true intent and update its most recent access timestamp to the current system time.

[0027] Through this step, the dynamic prior information stored in the graph is continuously updated and evolved, enabling the system to adapt to behavioral patterns in specific scenarios, thereby making more accurate judgments in future intent deductions.

[0028] A second aspect of the present invention provides a gimbal automatic tracking system based on predictive control, the system being configured to perform any of the methods described above.

[0029] Preferably, the system includes:

[0030] The scene modeling module is used to parse scene images, extract semantic anchor points, and construct an initial scene semantic navigation graph.

[0031] The real-time perception module is used to detect targets from the video stream, calculate their motion state, and locate them on the map;

[0032] The intent reasoning module is used to calculate the probability distribution of the target's intent based on the target's state and graph information;

[0033] The PTZ control module is used to generate and execute motion and field-of-view control commands for the PTZ based on the probability distribution of intent.

[0034] The graph management and evolution module is used to store, maintain, and update the scene semantic navigation graph based on verified target behaviors.

[0035] In summary, the present invention has at least one of the following beneficial technical effects:

[0036] 1. This invention constructs a scene semantic navigation graph and uses a probabilistic inference model based on this graph to predict the future intentions of the target, enabling the gimbal to generate predictive control commands in advance. Compared to existing methods that rely on reactive tracking based on the target's current position, this invention can plan a smooth trajectory before the target reaches decision points such as intersections, effectively overcoming the tracking delay and abrupt camera movement problems of traditional methods, and improving the continuity and smoothness of tracking.

[0037] 2. This invention improves tracking robustness by calculating the confidence level of the intent probability distribution and performing adaptive field-of-view control based on this confidence level. When the system has high uncertainty in predicting the target's intent, such as when the target is at multiple path exits with similar probabilities, the system will actively reduce the optical zoom to expand the field of view. This mechanism ensures that even if the target chooses a path with a lower probability, it will still remain within the camera's field of view, thereby significantly reducing the risk of target loss due to incorrect prediction.

[0038] 3. This invention, by setting up closed-loop feedback and adaptive evolution steps for the semantic navigation graph, enables the system to learn from long-term observations. The system uses verified target intents to update dynamic attributes in the scene semantic navigation graph, such as passage frequency. This allows the graph to dynamically reflect behavioral patterns in specific scenarios, thereby providing more accurate dynamic prior probabilities in subsequent intent inference, achieving adaptive optimization and long-term improvement of system performance. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the system structure of the present invention;

[0040] Figure 2 This is a schematic diagram of the method flow of the present invention;

[0041] Figure 3 This is a schematic diagram of the target map localization process of the present invention;

[0042] Figure 4 This is a schematic diagram illustrating the probability deduction process of the target intent of this invention;

[0043] Figure 5 This is a schematic diagram of the adaptive field-of-view control process of the present invention;

[0044] Figure 6 This is a schematic diagram of the multi-hypothesis fusion motion control process of the present invention;

[0045] Figure 7This is a schematic diagram of the adaptive evolution process of the spectrum in this invention. Detailed Implementation

[0046] The following is in conjunction with the appendix Figure 1 -Appendix Figure 7 The present invention will be further described in detail below.

[0047] like Figure 1 As shown, Figure 1 This is a schematic diagram of the system structure of an automatic gimbal tracking system according to an embodiment of the present invention. The automatic gimbal tracking system based on predictive control provided by the present invention may include: a scene modeling module 110, a real-time perception module 120, an intent reasoning module 130, a gimbal control module 140, and a map management and evolution module 150.

[0048] The scene modeling module 110 is used to parse scene images, extract semantic anchors, and construct an initial scene semantic navigation graph.

[0049] The real-time perception module 120 is used to detect targets from the video stream, calculate their motion state, and locate them on the map.

[0050] The intent reasoning module 130 is used to calculate the probability distribution of the target's intent based on the target's state and graph information.

[0051] The PTZ control module 140 is used to generate and execute motion and field-of-view control commands for the PTZ based on the intent probability distribution.

[0052] The graph management and evolution module 150 is used to store, maintain, and update the scene semantic navigation graph based on verified target behaviors.

[0053] See attached document Figure 2 , Figure 2 This is a schematic flowchart of a gimbal automatic tracking method according to an embodiment of the present invention. The gimbal automatic tracking method based on predictive control provided by the present invention may include the following steps:

[0054] S210: Construct a scene semantic navigation graph.

[0055] S220: Acquire the target's motion state and perform map localization.

[0056] S230: Perform a probability deduction of the target's intent.

[0057] S240: Generates and executes predictive cooperative control commands.

[0058] S250: Perform adaptive evolution of the map.

[0059] During the offline phase of system deployment, the scene modeling module 110 executes step S210. This module analyzes pre-acquired monitoring scene images, identifies and extracts static elements with interactive potential in the scene as semantic anchors, and constructs an initial scene semantic navigation graph. This graph stores the static layout of the scene and the framework used to carry subsequent dynamic behavior information in a structured manner.

[0060] During the online phase of system operation, the real-time perception module 120 executes step S220. This module continuously detects the tracked target from the video stream, calculates its real-time target motion state in the world coordinate system, and maps the target's position onto the scene semantic navigation map maintained by the map management and evolution module 150 to complete the target's map localization. The real-time perception module 120 outputs the calculated target motion state and map position information to the intent reasoning module 130.

[0061] After receiving data from the real-time perception module 120, the intent reasoning module 130 executes step S230. This module obtains the latest scene semantic navigation map from the map management and evolution module 150, and, combined with the input target motion state and map position, performs probabilistic inference on all candidate intents of the target. This inference process calculates the posterior probability of each candidate intent using the following formula to obtain an intent probability distribution: ;

[0062] In the formula, Given the target's motion state and the scene semantic navigation graph, the intention of the tracked target is the first... The posterior probability of each candidate intent; For the first One candidate intent; The target motion state; For scene semantic navigation graph; Normalization factor; This is the likelihood term calculated based on the motion consistency between the target motion state and the candidate intent; It is a dynamic prior.

[0063] The intent reasoning module 130 outputs the calculated intent probability distribution to the gimbal control module 140.

[0064] After receiving the intent probability distribution, the gimbal control module 140 executes step S240. Based on the input intent probability distribution, this module generates gimbal field-of-view control commands and motion control commands in parallel. When generating the field-of-view control commands, the module first calculates the intent confidence level using the following formula:

[0065] ;

[0066] In the formula, The probability distribution of intent; The information entropy of the intention probability distribution; The number of candidate intents; For the confidence level of the intention; This represents the maximum entropy value.

[0067] Subsequently, the gimbal control module 140 compares the calculated intent confidence level with a preset threshold to generate instructions for adjusting the optical zoom level of the gimbal. When generating motion control instructions, the module selects multiple high-probability candidate intents based on the intent probability distribution, generates an independent prediction path for each intent, and performs weighted fusion based on the probability of each intent to finally generate control instructions that drive the gimbal to pan and tilt.

[0068] During the execution of tracking commands by the gimbal control module 140, the system verifies the true intent of the tracked target through continuous observation by the real-time perception module 120. This verification result is sent to the graph management and evolution module 150, which executes step S250. Based on the verified true intent, the graph management and evolution module 150 updates the dynamic attributes of the corresponding semantic anchors in the scene semantic navigation graph. This update operation allows the graph to evolve adaptively. The updated graph will be invoked by the intent reasoning module 130 in subsequent tracking cycles, thus forming a complete closed-loop workflow, optimizing the system's tracking performance over time.

[0069] The offline scene modeling and semantic graph construction steps of the present invention aim to transform unstructured raw scene information into a computer-understandable, structured graph data model, providing an environmental cognitive foundation for subsequent online tracking and intent inference.

[0070] In one specific embodiment, this step first requires extracting semantic anchor points from the scene. The operator can provide one or more scene images covering the entire monitored area, such as a top-view plan or a stitched panoramic image. A pre-trained image semantic segmentation model is applied to the scene image; this model assigns a semantic label to each pixel in the image, such as a doorway, window, stairwell, or road intersection. After obtaining pixel-level semantic segmentation results, the system clusters connected pixel regions with the same semantic label to identify individual semantic element instances. Subsequently, the geometric centroid of each instance is calculated, and the world coordinate system position of this centroid is determined as the location of a semantic anchor point.

[0071] Each extracted semantic anchor is instantiated as a data structure containing specific information. This data structure includes at least:

[0072] An anchor point ID is used to uniquely identify the anchor point; an anchor point type is used to identify its type (e.g., doorway, intersection); a position vector is used to represent its three-dimensional position in the world coordinate system; and a set of dynamic attributes are used to record its historical interaction information. The dynamic attributes include at least a passage frequency counter with an initial value of 0; and a recent passage timestamp with an initial value of null or set to 0.

[0073] After all semantic anchors are extracted and instantiated, the system constructs a scene semantic navigation graph based on these anchors. This graph is a directed graph. , where the set of nodes It consists of all the semantic anchors extracted in the preceding steps. (Edge set) This represents the passable path between anchor points.

[0074] The process of constructing edges is as follows: For the set of nodes Any pair of anchor nodes The system determines whether there is an unobstructed, straight, passable path between the two. If such a path exists, a path is created in the graph from the point where the two paths meet. point to directed edges And another one from point to directed edges Also added to the edge set In the data structure of each edge, the corresponding physical path length is stored. This length can be calculated as the Euclidean distance between the position vectors of the two anchor nodes connected by the edge.

[0075] See attached document Figure 3 , Figure 3 This is a schematic diagram of the target map localization process according to an embodiment of the present invention. The purpose of the online real-time perception and target map localization steps of the present invention is to obtain dynamic information of the tracked target from a continuous video stream and associate it with an offline constructed scene cognitive structure (i.e., scene semantic navigation map).

[0076] In one specific embodiment, this step first requires estimating the motion state of the tracked target. The system acquires a real-time video stream from a PTZ camera and processes each frame. A target detector, such as a deep learning-based YOLO (YouOnlyLookOnce) or SSD (SingleShotMultiBoxDetector) model, is used to identify the location of the tracked target in the image and output its two-dimensional bounding box.

[0077] After obtaining the target's two-dimensional bounding box, the system uses the calibrated camera intrinsic and extrinsic parameters to convert the two-dimensional image position into three-dimensional world coordinates. Specifically, it assumes the tracked target is always moving on a predefined ground plane. The system draws a ray from the camera's optical center, passing through the bottom center point of the target's two-dimensional bounding box. The intersection of this ray and the ground plane is determined as the target's current position. Three-dimensional position in the world coordinate system .

[0078] target speed This can be calculated by observing changes in its continuous position. For example, by observing the position at the current moment... Position relative to the previous moment Take the difference, then divide by the time interval between the two moments. This yields an instantaneous velocity vector. To obtain a smoother and more stable motion state estimate, a Kalman filter is used to continuously track the target. This filter takes the observed position of each frame as input and outputs a filtered and optimized position and velocity estimate. These two vectors together constitute the target's real-time motion state. .

[0079] After acquiring the target's real-time motion state Next, the system performs target localization. The goal of this process is to determine the target's three-dimensional world coordinates. Associated with scene semantic navigation graph On a specific path edge. The system traverses all edges in the graph, and for each edge... (Connecting anchor points) and ), calculate the target location point From the anchor point and The perpendicular distance of the line segment defined by the position.

[0080] The system selects the target location point The edge with the smallest vertical distance is taken as the path edge where the target is currently located. Simultaneously, the orthogonal projection point of the target's position on this edge is determined as the target's geographic location. Through this step, the target's real-time position is constrained to a predefined, traversable geographic path, thus providing precise and structured location input for subsequent intent deduction.

[0081] See attached document Figure 4 , Figure 4This is a schematic diagram of the target intent probability deduction process according to an embodiment of the present invention. The target intent probability deduction step based on dynamic prior in the method of the present invention aims to quantitatively predict the next direction of the target by combining the real-time movement trend of the target with the historical behavior pattern of the scene.

[0082] In one specific embodiment, this step begins after the target's location in the geographic map is determined. The system first generates a discrete set of candidate intentions. If the target is currently located on a path edge connecting two anchor points, then its downstream anchor points are identified as candidate intentions. If the target is located at an anchor point, then all downstream anchor points directly connected to that anchor point via outgoing edges constitute the set of candidate intentions. Each candidate intention... All correspond to moving to a specific target anchor point. .

[0083] For each candidate intent in the candidate intent set The system calculates its posterior probability using a Bayesian inference model. This calculation is achieved through the following formula:

[0084] ;

[0085] In the formula, Given the target's motion state and the scene semantic navigation graph, the intention of the tracked target is the first... The posterior probability of each candidate intent; For the first One candidate intent; The target motion state; For scene semantic navigation graph; Normalization factor; This is the likelihood term calculated based on the motion consistency between the target motion state and the candidate intent; It is a dynamic prior.

[0086] The calculation process incorporates a likelihood term and a dynamic prior term.

[0087] Its posterior probability From the likelihood term With dynamic priors The product of these products is obtained by normalization.

[0088] Likelihood Term The calculation process for representing motion consistency is as follows: First, determine the direction from the target's current position to the candidate intent. The direction vector corresponding to the anchor point position is then used to calculate the relationship between this direction vector and the target's current motion state. The angle between the velocity vectors. The smaller the angle, the higher the consistency of motion, and the higher the value of the likelihood term.

[0089] Dynamic priors The historical behavioral patterns are represented and calculated as follows: from the graph Query candidate intent The dynamic attributes of the corresponding anchor points are the frequency of passage and the most recent passage timestamp. This module uses a preset function to map anchor points with higher passage frequencies and more recent passage times to a higher prior probability value.

[0090] Finally, the module combines the posterior probability values ​​of all candidate intents to form a complete intent probability distribution. It is then output to the PTZ control module 140.

[0091] See attached document Figure 5 and attached Figure 6 , attached Figure 5 This is a schematic diagram of an adaptive field-of-view control process according to an embodiment of the present invention, with appended... Figure 6 This is a schematic diagram of a multi-hypothesis fusion motion control process according to one embodiment of the present invention. The purpose of the predictive cooperative control command generation and execution step in the method of the present invention is to transform the intent probability distribution derived from the aforementioned steps into specific control commands that drive the coordinated actions of the gimbal hardware. In one embodiment, this step is decomposed into two parallel subtasks: adaptive field-of-view control and multi-hypothesis fusion motion control.

[0092] The adaptive field-of-view control subtask aims to dynamically adjust the optical zoom level of the gimbal based on the degree of uncertainty in the prediction. First, the system bases its decisions on the probability distribution of the received intent. Calculate the confidence score of an intent. In one specific embodiment, the calculation is performed using the formula described above ( )Finish.

[0093] After calculating the confidence level of intent Then, the system compares it with a preset high-confidence threshold. and low confidence threshold Compare. If This indicates that the system has a high degree of certainty in predicting the target's intent. At this point, the system generates a command to increase the optical zoom level to obtain clearer target details. If... This indicates that the system is in a state of high uncertainty (e.g., the target is at an intersection with multiple equally likely exits). In this case, the system generates an instruction to reduce the optical zoom level to expand the field of view, ensuring that the target does not leave the monitoring screen if it makes an unexpected choice. If the intent confidence level is between two thresholds, the current optical zoom level is maintained.

[0094] The multi-hypothesis fusion motion control subtask aims to generate smooth and predictive gimbal motion commands based on the intent probability distribution. First, the system selects the N candidate intents (e.g., N=3) with the highest probability values ​​from the intent probability distribution. For each selected candidate intent, the system instantiates an independent motion predictor, such as a constrained Kalman filter, whose state transition model is constrained to move along a path leading to the anchor point corresponding to that intent. Each predictor independently predicts the target motion trajectory forward within a time window (e.g., 2 seconds), thus forming N parallel, probabilistically possible future paths.

[0095] The system then performs a weighted fusion of these N parallel paths. At each discrete time point within a future time window, a final fusion target location is calculated by taking a weighted average of the positions of the N paths at that time point.

[0096] The weights for each path are obtained by normalizing the posterior probability values ​​of their corresponding candidate intentions. In this way, the final predicted trajectory after fusion will smoothly lean towards the intention path with the highest probability, while also being appropriately influenced by other intention paths with the next highest probability.

[0097] Finally, the fused predicted trajectory is fed into a low-level controller, such as a PID (proportional-integral-derivative) controller. This controller uses the next position point of the predicted trajectory as the target setpoint and calculates the precise angular velocity or angle increment required to drive the pan and tilt motors, thereby generating and executing the final hardware control commands to achieve smooth tracking of the target.

[0098] See attached document Figure 7 , Figure 7 This is a schematic diagram of the map adaptive evolution process according to one embodiment of the present invention. The closed-loop feedback and map adaptive evolution steps in the method of the present invention aim to establish a learning mechanism, enabling the system to continuously optimize its internal scene cognition model by continuously observing the real behavior of the target, thereby improving the prediction accuracy under long-term operation.

[0099] In one specific embodiment, this step continues throughout the system's target tracking process. First, the system needs to verify the true intent of the tracked target. The real-time perception module continuously outputs the target's position in the world coordinate system. The system continuously compares this position with the positions of all semantic anchors in the scene semantic navigation map. When the target's position matches a certain semantic anchor... When the Euclidean distance between the geometric centers of the target and the anchor point is less than a preset distance threshold, the system determines that the target has reached the anchor point and will proceed to the anchor point. The action is confirmed as a genuine intent that has been fulfilled.

[0100] After verifying the true intent, the system immediately executes the dynamic attribute update mechanism for the geographic graph. The geographic graph management and evolution module receives verification information about the completed true intent, which includes at least the anchor point ID of the target's arrival. Based on this ID, the module locates the corresponding anchor point node in the scene semantic navigation graph's data structure. .

[0101] Subsequently, the module performs an update operation on the dynamic attributes of the anchor node. Specifically, it increments the pass frequency count stored in its data structure by one and updates its most recent pass timestamp field to the current system time.

[0102] This update operation constitutes the closed-loop feedback loop of the method of the present invention. After the update, the anchor point... Because of its higher frequency of access and updated timestamps, when an intent reappears as a candidate intent at any future time during intent deduction, its corresponding dynamic prior probability will be calculated to a higher value. This allows the entire system to dynamically learn high-frequency paths or behavioral patterns in specific scenarios and adaptively reflect these patterns in its future predictions, thereby achieving continuous self-optimization of tracking performance.

[0103] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A gimbal automatic tracking method based on predictive control, characterized in that, Includes the following steps: Construct a scene semantic navigation graph, wherein the scene semantic navigation graph includes multiple semantic anchors representing static elements in the scene that have the possibility of interaction, and traversable paths connecting the semantic anchors; The real-time motion state of the tracked target is obtained, and the map position of the tracked target is determined on the constructed scene semantic navigation map; Based on the acquired target motion state, the determined map position, and the scene semantic navigation map, the probability of multiple candidate intentions of the tracked target is probabilistically extrapolated to obtain an intention probability distribution that characterizes the likelihood of each of the multiple candidate intentions occurring. Based on the obtained intent probability distribution, control commands for driving the gimbal are generated and executed to achieve automatic tracking of the tracked target.

2. The automatic gimbal tracking method based on predictive control according to claim 1, characterized in that, The steps for constructing a scene semantic navigation graph specifically include: Multiple semantic anchor points were identified by analyzing scene images; For each of the semantic anchor points, a preset dynamic attribute is associated to represent historical interaction information; The semantic anchors are used as nodes of a graph, and the passable paths are used as edges of the graph to construct the scene semantic navigation graph.

3. The automatic gimbal tracking method based on predictive control according to claim 2, characterized in that, The method further includes a step performed after the step of generating and executing control commands for driving the gimbal, for implementing adaptive evolution of the scene semantic navigation map, which specifically includes: During the automatic tracking process, observe and verify the true intent of the tracked target; Based on the verified true intent, update the dynamic properties of the semantic anchor associated with the true intent.

4. The automatic gimbal tracking method based on predictive control according to claim 1, characterized in that, The step of acquiring the real-time target motion state of the tracked target and determining the map position of the tracked target on the constructed scene semantic navigation map specifically includes: The position and velocity of the tracked target in the world coordinate system are calculated based on the target detection results in the video frame, which is used as the target motion state; The position in the world coordinate system is mapped onto the scene semantic navigation map to determine its relative relationship with the surrounding semantic anchor points, which is then used as the map position.

5. The automatic gimbal tracking method based on predictive control according to claim 1, characterized in that, The step of probabilistically extrapolating multiple candidate intentions of the tracked target specifically includes: For each candidate intent Its intention probability The calculation is performed using the following formula: ； In the formula, Given the target's motion state and the scene semantic navigation map, the intention of the tracked target is the first... The posterior probability of each candidate intent; For the first One candidate intent; The target motion state; For the semantic navigation graph of the scene; Normalization factor; The likelihood term is calculated based on the motion consistency between the target motion state and the candidate intention; It is a dynamic prior.

6. The gimbal automatic tracking method based on predictive control according to claim 5, characterized in that, The dynamic priors It is based on the candidate intent. The dynamic attributes of the associated semantic anchor are determined by the frequency of historical interactions and / or the most recent timestamp of the semantic anchor.

7. The automatic gimbal tracking method based on predictive control according to claim 1, characterized in that, Prior to the step of generating and executing control commands for driving the gimbal, an adaptive field-of-view control step is also included, which specifically includes: Based on the intent probability distribution, an intent confidence level is calculated to characterize the uncertainty of the current intent prediction; Based on the comparison between the intent confidence level and preset high confidence thresholds and low confidence thresholds, when the intent confidence level is higher than the high confidence threshold, the optical zoom level is increased; and when the intent confidence level is lower than the low confidence threshold, the optical zoom level is decreased.

8. The automatic gimbal tracking method based on predictive control according to claim 7, characterized in that, The confidence level of the intent The calculation is performed using the following formula: ； In the formula, The probability distribution of the stated intent; The information entropy of the probability distribution of the stated intent; The number of the multiple candidate intentions; The confidence level of the stated intent; This represents the maximum entropy value.

9. The automatic gimbal tracking method based on predictive control according to claim 1, characterized in that, The step of generating and executing control commands for driving the gimbal specifically includes: Based on the intent probability distribution, at least two high-probability candidate intents are selected; For each selected high-probability candidate intent, an independent prediction path is generated to form a multi-hypothesis path prediction. Based on the intent probabilities corresponding to the high-probability candidate intents, the multi-hypothesis path predictions are weighted and fused to generate the final control commands used to drive the gimbal.

10. A gimbal automatic tracking system based on predictive control, characterized in that, A method for executing a gimbal tracking system based on predictive control as described in any one of claims 1-9 includes: The scene modeling module is used to parse scene images, extract semantic anchor points, and construct an initial scene semantic navigation graph. The real-time perception module is used to detect targets from the video stream, calculate their motion state, and locate them on the map; The intent reasoning module is used to calculate the probability distribution of the target's intent based on the target's state and graph information; The PTZ control module is used to generate and execute motion and field-of-view control commands for the PTZ based on the probability distribution of intent. The graph management and evolution module is used to store, maintain, and update the scene semantic navigation graph based on verified target behaviors.

Images