A dynamic adaptive graph-based image object tracking and motion prediction method

Abstract

The application discloses a kind of dynamic self-adapting graph-based image target tracking and motion prediction method, comprising: image frame is collected and is cached;Target detection is carried out and semantic attribute is extracted;Constitute confidence-enhanced graph node;Sparse graph topology based on Delaunay triangulation algorithm is constructed;Confidence weighted node affinity is calculated;Edge affinity is calculated;Secondary integer programming matching model is constructed;Continuous relaxation and Sinkhorn projection solution are carried out;Hungarian discretization and optimal matching output are carried out;Adaptive backtracking and trajectory chain generation are carried out;Adaptive prediction order is determined and motion derivative is calculated;Taylor series extrapolation prediction is carried out and prediction node set is output.The method realizes high robustness improvement in trajectory association aspect.Introduce sparse graph structure based on Delaunay triangulation and confidence perception mechanism, and combine reverse matching strategy, effectively utilize the spatial relationship constraint of target group, significantly enhance the tracking stability under complex scenes such as occlusion, detection loss.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, specifically to a dynamic adaptive graph-based image target tracking and motion prediction method. Background Technology

[0002] Image target tracking technology is a core component of computer vision and artificial intelligence perception systems, responsible for the continuous localization and identity association of targets of interest, such as vehicles and pedestrians, within video sequences. However, in practical applications, system delays are unavoidable in the acquisition, encoding, transmission, and algorithm processing of image data, causing perception results to often lag behind real-world timestamps. Therefore, simply tracking the target in the current frame is insufficient; more crucially, it is necessary to accurately predict the target's position, velocity, and trajectory at future moments based on historical observation data to ensure the safety of downstream decision-making and planning. This technology has extremely important application value in fields with extremely high requirements for timeliness and safety, such as high-speed autonomous driving, intelligent traffic monitoring, remote real-time control, and cloud robotics.

[0003] However, existing image target tracking and motion prediction technologies struggle to achieve an effective balance between real-time performance and accuracy. On one hand, while trajectory prediction algorithms based on deep learning (such as RNN, LSTM, or Transformer) have theoretically achieved high accuracy, their complex model structures and massive parameter counts make them highly dependent on computational resources, hindering millisecond-level real-time inference on resource-constrained in-vehicle embedded platforms or edge devices. On the other hand, lightweight methods based on Kalman filtering or constant velocity / acceleration assumptions, widely adopted in engineering practice to meet real-time requirements, while computationally inexpensive, become overly rigid when faced with complex dynamic scenes. Specifically, these traditional methods suffer from two core flaws: first, they typically assume linear motion, making it difficult to capture nonlinear behaviors such as sharp turns or speed changes; second, they use fixed historical step sizes or sliding windows to fit the model, a "one-size-fits-all" mechanism that fails to flexibly utilize the long- and short-term motion patterns of different targets based on their actual duration of presence in the field of view, leading to suboptimal mining of historical information and severely limiting prediction accuracy.

[0004] In summary, current image target tracking and motion prediction technologies still face a significant challenge in balancing real-time performance and accuracy. While deep learning methods can model complex motion well, their high computational complexity makes them difficult to deploy in real-time on resource-constrained edge devices. Traditional filtering methods, though lightweight and fast, are limited by rigid motion assumptions and fixed time windows, failing to fully exploit the long- and short-term historical information of different targets and adapt to nonlinear changes. These issues limit the predictive performance of systems in complex dynamic scenarios, making it difficult to simultaneously meet the demands of high-timeliness and high-safety applications such as high-speed autonomous driving and intelligent monitoring. Therefore, there is an urgent need for an intelligent motion prediction method that balances computational efficiency and adaptive capabilities. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes a dynamic adaptive graph-based image target tracking and motion prediction method. This method aims to significantly improve prediction accuracy by flexibly mining the unique historical motion information of each target while ensuring efficient operation at the edge. The core idea of ​​this method comprises two levels: At the trajectory association level, an adaptive step-size reverse matching strategy is employed. By constructing a spatiotemporal relationship graph and dynamically searching in the negative direction of the time axis, the historical trajectory length of each independent target is maximized, thereby mining the most complete motion context information of that target. At the motion prediction level, a Taylor series-enhanced prediction model is introduced. Based on the variable-length trajectory obtained from the previous level, the order of the prediction model is automatically adjusted. For targets with long trajectories, higher-order derivatives are used to accurately fit their nonlinear maneuvers, while for targets with short trajectories, the model smoothly degenerates into a lower-order model. This "tailor-made" dynamic prediction mechanism effectively overcomes the limitations of traditional fixed-step-size models, achieving high-precision, low-latency prediction for complex dynamic scenes.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A dynamically adaptive graph-based image target tracking and motion prediction method includes: Step 1: Acquire image frames and buffer them; Step 2: Perform target detection and extract semantic attributes; Step 3: Construct graph nodes with enhanced confidence; Step 4: Construct a sparse graph topology based on the Delaunay triangulation algorithm; Step 5: Calculate the confidence-weighted node affinity; Step 6, calculate edge affinity; Step 7: Construct a quadratic integer programming matching model; Step 8: Perform continuous relaxation and Sinkhorn projection solution; Step 9: Perform Hungarian discretization and optimal matching output; Step 10: Perform adaptive reverse backtracking and trajectory chain generation; Step 11: Determine the adaptive prediction order and calculate the motion derivative; Step 12: Perform Taylor series extrapolation prediction and output the predicted node set.

[0007] Further, step 1 includes: The image sensor continuously acquires a sequence of raw image frames, and the currently received Nth frame is denoted as I. N And store it in the frame buffer queue Q I This frame buffer queue maintains the image data of the most recent W frames in a first-in-first-out manner, that is: , Where W is the queue window length; Step 2 includes: For the current frame image I obtained in step 1 N Execute the YOLO object detection algorithm to obtain the set of objects within the current field of view. : , Where m is the number of targets detected in the current frame, and for each detected target... Extracting semantic attribute triples : , in, Represents the i-th target category of the N-th frame image. Let (x, y) represent the pixel coordinates (x, y) of the i-th target center point in the N-th frame of the image in the image coordinate system. The bounding box size of the i-th target in the N-th frame image, including width w and height h, is represented by the detection result Q. N Add to the detection result queue Q D This makes it compatible with the frame buffer queue Q. I Maintain time alignment: .

[0008] Further, step 3 includes: Each detection target obtained in step 2 Mapped to nodes of a graph Besides semantic attributes In addition, each node also carries the detection confidence score output by the object detection algorithm. To form enhanced node attributes : , Based on the confidence score, a normalized weight is calculated for each node. : , The node set V of the current frame N N Recorded as: , This represents the mapping of the graph node corresponding to the i-th target in the N-th frame of the image; For the detection result queue Q D Each historical frame stored in the system undergoes the same node mapping and weight calculation operations to form an enhanced node set for each frame.

[0009] Further, step 4 includes: The same frame node set V obtained in step 3 N Target center pixel coordinates of each node Using control points, calculate the Delaunay triangulation on the 2D image plane. : , Delaunay is a triangulation algorithm that only considers adjacent node pairs determined by Delaunay triangulation. Construct directed edges between them Each edge encodes the relative spatial vector between two targets. for: , This forms the sparse relation graph of the current frame. The relationship diagrams of each frame are organized according to time sequence to form a spatiotemporal relationship diagram structure. : .

[0010] Further, step 5 includes: In the spatiotemporal relationship graph constructed in step S4, for any two nodes in two adjacent frames n and n-1... and Calculate the node affinity A for the fusion detection confidence. V : , in, This is a confidence geometric mean adjustment factor. When both nodes come from high-confidence detections, its value is close to 1. When either node is a low-confidence detection, it automatically reduces the affinity score of the matching pair. Used to calculate the matching degree between target i and target p in two adjacent frames n and n-1, equal VFor a category matching function, if and only if Returns 1 if the condition is met, otherwise returns 0. This represents the distance between the center points of the i-th target in the n-th frame and the p-th target in the (n-1)-th frame. μ1 and μ2 represent the difference between the bounding boxes of the i-th target in frame n and the p-th target in frame (n-1); μ1 and μ2 are weighting coefficients; λ1 and λ2 are normalization coefficients; and e is the natural constant.

[0011] Further, step 6 includes: In step 4, among the edges determined by Delaunay triangulation, calculate the corresponding edges in adjacent frames. and The similarity, i.e., edge affinity A E : , Where cp represents the type pair of connected nodes. Representative category pair and The matching degree, where m is the relative space vector defined in step S4. Represents the target vector and The distance is λ3, where λ3 is the normalization coefficient and e is the natural constant.

[0012] Further, step 7 includes: The target association problem between adjacent frames is modeled as a quadratic integer programming problem. The globally optimal one-to-one matching scheme is solved by jointly optimizing node affinity and edge affinity, and a binary decision variable matrix is ​​defined. , where m and m' are the target numbers in the current frame and the previous frame, respectively, with the optimization objective being to maximize the joint score of node affinity and edge affinity: , x ip =1 indicates that the current frame node will be... Matched the node from the previous frame ; Let J(X) represent the function that maximizes the matching score, with three sets of constraints: Group 1 ; The second group, if Then x ip =0; Group 3, x ip ∈ {0, 1}; In the formula, τ V This represents the affinity threshold, and μ3 is a balance coefficient used to adjust the weight of edge affinity in the total score. Let n be the set of edges between targets in the nth frame. Let x be the set of edges between targets in the (n-1)th frame. ip This indicates whether the current frame node is correctly displayed. Matched the node from the previous frame .

[0013] Further, step 8 includes: First, apply the binary constraint x ip The relaxation of ∈ {0,1} is a continuous constraint x ip ∈[0,1], the feasible region of the decision matrix X is expanded into a birandom matrix space; subsequently, the Sinkhorn iterative algorithm is used to alternately normalize the row and column of the relaxed affinity matrix: , Where r and c are the normalized vectors that make the row sum and column sum satisfy the constraints, respectively. Let be the decision matrix at the t-th iteration, and diag(r) represents converting vector r into a diagonal matrix; Step 9 includes: The continuous soft allocation matrix obtained by Sinkhorn iterative projection in step 8 The Hungarian algorithm is applied for discretization, mapping continuous solutions to integer optimal solutions that satisfy one-to-one constraints. The final output matching result This is the globally optimal target association scheme between adjacent frames.

[0014] Further, step 10 includes: Starting from the current frame N, using the matching results from steps 7 to 9, perform frame-by-frame reverse backtracking independently for each target along the negative time axis. Once the maximum node affinity at a certain moment falls below the affinity threshold τ, V If no candidate node can be found, the trajectory extension of the target is terminated. Therefore, for each target in the current frame... Generate a variable-length historical trajectory chain : , in The length of this historical trajectory chain, This represents the distance of the trajectory chain.

[0015] Further, step 11 includes: For each historical trajectory chain output in step 10, determine the order of the Taylor series expansion from the position sequence. Calculate average speed σ, the standard deviation of velocity variation vAnd curvature index κ; secondly, construct motion complexity index Γ: , Among them W k W v W σ W κ Represents the weighting coefficients, and each normalization factor k max、 v max、 σ max、 κ max To determine k through the standard scenario test set: max As a limit on the maximum length of the observed trajectory, v max σ is the upper limit of the typical target speed. max κ is the threshold for the significance of the velocity change. max This represents the limit value of the trajectory curvature; Based on the threshold range of the Γ value, the order n∈{1,2,3,4} is automatically selected: Γ<0.25:n=1, representing the case of constant velocity. 0.25≤Γ<0.55: n=2, representing a smooth speed change. 0.55≤Γ<0.75: n=3, representing a case of significant nonlinearity. Γ≥0.75: n=4, representing a highly complex case; Finally, the central difference method is used to calculate the motion derivatives up to the nth order. For the jth order derivative, j≥1, the recursive formula of adjacent difference sequences is used, keeping the time complexity at a lightweight level of O(n).

[0016] Further, step 12 includes: First, a high-order Taylor extrapolation calculation is performed: Taylor expansion is applied to extrapolate and predict all four spatial components, which are the center position x, y and the bounding box dimensions w, h. , Let j represent the predicted parameter value for the N+Kth frame in the future, and j! represent the factorial of j. This represents the j-th derivative of parameter p at the current frame N. Represents K raised to the power of j; obtains the predicted position. and predicted bounding boxes ,in Let x and y be the x and y coordinates of the i-th target in the N+K-th frame. Let w and h be the bounding box of the i-th target in the N+K-th frame; where the truncation error O(K) n+1 The accuracy of the prediction is adaptively determined by the prediction order n; secondly, prediction nodes are constructed: for each target Construct prediction nodes containing three types of complete information. ,in This represents the target category of the i-th target in the N-th frame. This represents the predicted position of the i-th target in the N+K-th frame. This represents the bounding box of the i-th target in the N+K-th frame; finally, the output is the set of predicted nodes: summarizing the predicted nodes of all targets, and finally outputting the complete set of predicted nodes. , Complete information about the predicted node of the m-th target in the N+K-th frame.

[0017] The main advantages of this invention are as follows: This method achieves significantly improved robustness in trajectory association. By introducing a sparse graph structure based on Delaunay triangulation and a confidence-aware mechanism, combined with a reverse matching strategy, it effectively utilizes the spatial relationship constraints of the target group, significantly enhancing tracking stability in complex scenarios such as occlusion and detection loss. Simultaneously, by modeling target association as a quadratic integer programming problem, jointly optimizing node affinity and edge affinity, and employing a Sinkhorn-Hungarian solver to obtain the globally optimal one-to-one matching, it effectively avoids the cascading errors of traditional greedy strategies.

[0018] In motion prediction, this method overcomes the limitations of traditional approaches. It can dynamically and adaptively adjust based on the actual observation history of each target, accurately capturing the nonlinear maneuvering behavior of long-trajectory targets while robustly handling short-trajectory targets. Through a collaborative mechanism between the frame buffer queue and the detection result queue, it achieves efficient organization and rapid indexing of historical observation data.

[0019] Furthermore, this method exhibits excellent computational efficiency and real-time performance. Compared to heavyweight deep learning models, the graph matching and Taylor expansion methods employed in this technique have low computational complexity, can run in real-time on edge devices without the need for a GPU, and the Sinkhorn iteration typically converges in 3-5 iterations. The Hungarian solution for 50 targets takes only about 0.05ms to solve.

[0020] In summary, this method achieves a good balance in terms of robustness, predictive adaptability, and real-time performance, providing reliable technical support for high-safety-requirement scenarios such as high-speed autonomous driving. Attached Figure Description

[0021] Figure 1 This is a flowchart of a dynamic adaptive graph-based image target tracking and motion prediction method according to the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Please see Figure 1 A dynamic adaptive graph-based image target tracking and motion prediction method, comprising:

[0024] Step 1, acquire image frames and buffer them, including:

[0025] The image sensor continuously acquires a sequence of raw image frames, and the currently received Nth frame is denoted as I. N And store it in the frame buffer queue Q I This frame buffer queue maintains the image data of the most recent W frames in a first-in-first-out (FIFO) manner, that is:

[0026]

[0027] Where W is the queue window length, which determines the maximum number of historical frames that the system can trace, providing historical frame data support for subsequent reverse matching.

[0028] In this embodiment, the queue window length W=30, meaning the system caches a maximum of the most recent 30 frames, corresponding to approximately 1 second of historical data in a 30fps video stream. When the queue is full, the earliest frame is automatically dequeued to release memory.

[0029] Step 2, perform object detection and extract semantic attributes, including:

[0030] For the current frame image I obtained in step 1 N Execute the YOLO (You Only Look Once) object detection algorithm to obtain the set of objects within the current field of view. :

[0031]

[0032] Where m is the number of targets detected in the current frame, and for each detected target... Extracting semantic attribute triples : in, This represents the i-th target category in the N-th frame image, such as "car", "pedestrian", or "cyclist". Let (x, y) represent the pixel coordinates (x, y) of the i-th target center point in the N-th frame of the image in the image coordinate system. The bounding box size of the i-th target in the N-th frame image, including width w and height h, is represented by the detection result Q. N Add to the detection result queue Q D This makes it compatible with the frame buffer queue Q. I Maintain time alignment:

[0033]

[0034] This queue provides a structured index of multi-frame detection data for subsequent reverse matching steps.

[0035] Step 3, construct confidence-enhanced graph nodes, including:

[0036] Each detection target obtained in step 2 Mapped to nodes of a graph Besides semantic attributes In addition, each node also carries the detection confidence score output by the object detection algorithm. To form enhanced node attributes :

[0037]

[0038] Based on the confidence score, a normalized weight is calculated for each node. :

[0039]

[0040] For example, when three targets are detected in a frame with confidence levels of 0.95, 0.82, and 0.43, their quality weights are 1.0, 0.86, and 0.45, respectively. High-confidence nodes (such as clearly visible targets) will have a greater influence in subsequent matching, while the influence of low-confidence nodes (such as partially occluded or blurred targets) will be reasonably suppressed, thereby improving the overall reliability of matching.

[0041] This quality weight will serve as a reliability adjustment factor in subsequent affinity calculations, allowing high-confidence detections to dominate the matching process, while the impact of low-confidence detections (such as partially occluded targets) will be appropriately suppressed.

[0042] The node set V of the current frame N N Recorded as:

[0043]

[0044] This represents the mapping of the graph node corresponding to the i-th target in the N-th frame of the image.

[0045] For the detection result queue Q D The same node mapping and weight calculation operations are performed on each historical frame stored in the database to form an enhanced node set for each frame. .

[0046] Step 4: Construct a sparse graph topology based on the Delaunay triangulation algorithm, including:

[0047] Unlike traditional methods that construct fully connected edges for all targets within the same frame (generating m(m-1) directed edges), this embodiment uses the Delaunay triangulation algorithm from computational geometry to adaptively determine the graph's topology. This embodiment uses the set of nodes V obtained in step 3 within the same frame... N Target center pixel coordinates of each node Using control points, calculate the Delaunay triangulation on the 2D image plane. :

[0048]

[0049] Delaunay is a triangulation algorithm that only considers adjacent node pairs determined by Delaunay triangulation. Construct directed edges between them Each edge encodes the relative spatial vector between two targets. for:

[0050] That is, the direction vector from the center of target i to the center of target j.

[0051] Delaunay triangulation has the following key advantages: (1) It ensures that each node is connected to its spatial nearest neighbor through an edge, without missing any important local spatial relationships; (2) The number of edges is reduced to O(m), compared to O(m) of a fully connected graph. 2 (3) The generated triangular mesh naturally reflects the real spatial neighborhood structure of the target group on the image plane, avoiding redundant noise edges introduced by distant target pairs in the fully connected graph.

[0052] This forms the sparse relation graph of the current frame. The relationship diagrams of each frame are organized according to time sequence to form a spatiotemporal relationship diagram structure. :

[0053] .

[0054] Step 5, calculate the confidence-weighted node affinity, including:

[0055] In the spatiotemporal relationship graph constructed in step S4, for any two nodes in two adjacent frames (e.g., frame n and frame n-1) and Calculate the node affinity A for the fusion detection confidence. V :

[0056] In this context, the subscripts n and n-1 in the above formula assume that the two adjacent frames are n and n-frames, while the subscript N in the previous formula assumes that the current frame is N. This is a confidence geometric mean adjustment factor. When both nodes come from high-confidence detections, its value is close to 1. When either node is a low-confidence detection, it automatically reduces the affinity score of the matching pair. Used to calculate the matching degree between target i and target p in two adjacent frames n and n-1, equal V For a category matching function, if and only if Returns 1 if the condition is met, otherwise returns 0. This represents the distance between the center points of the i-th target in the n-th frame and the p-th target in the (n-1)-th frame. λ1 represents the difference between the bounding boxes of the i-th target in frame n and the p-th target in frame (n-1); μ1 and μ2 are weighting coefficients used to balance the importance of position and size; λ1 and λ2 are normalization coefficients used to handle pixel scale differences at different resolutions; and e is the natural constant. In this embodiment, the weighting parameters μ1 = 0.6 and μ2 = 0.2 are set, indicating that positional consistency is more important than size consistency. The normalization coefficients λ1 and λ2 are set according to the image resolution (e.g., λ = 1 / 100) and are used to map the pixel difference to the [0,1] interval.

[0057] For example, when a sharp target with a confidence level of 0.95 is matched with a vague target with a confidence level of 0.40, the confidence adjustment factor is: The affinity score of the match will be reduced to 65% of the original value, thereby reducing the probability of such unreliable matches being selected.

[0058] Step 6, calculate edge affinity, including:

[0059] To ensure the structural consistency of the target group, in step 4, the corresponding edges in adjacent frames are calculated from the set of edges determined by Delaunay triangulation. and The similarity, i.e., edge affinity A E :

[0060] Here, cp represents the category pair of the connected nodes (such as the "vehicle-person" pair). Representative category pair and The matching degree is given by m, where m is the relative spatial vector defined in step S4, λ3 is the normalization coefficient, and e is the natural constant. Since the edge set has been constrained to local spatial neighborhood relations by Delaunay triangulation, the structural stability measured by this index focuses more on the true nearest neighbor topology of the target, excluding noise interference from distant unrelated target pairs.

[0061] Step 7, construct a quadratic integer programming matching model, including:

[0062] Unlike the local approach of independently selecting the optimal match for each target in a simple greedy strategy, this embodiment models the target association problem between adjacent frames as an Integer Quadratic Programming (IQP) problem. It solves for the globally optimal one-to-one matching scheme by jointly optimizing node affinity and edge affinity, rather than the local strategy of independently selecting each target in traditional greedy matching. A binary decision variable matrix is ​​defined. , where m and m' are the target numbers in the current frame and the previous frame, respectively, with the optimization objective being to maximize the joint score of node affinity and edge affinity:

[0063]

[0064] x ip =1 indicates that the current frame node will be... Matched the node from the previous frame ;x ip =0 indicates a mismatch. μ3 is the structural weight coefficient, and in this embodiment, μ3=0.2.

[0065] Let J(X) represent the function that maximizes the matching score, with three sets of constraints:

[0066] Group 1 (One-to-one matching constraint);

[0067] The second group, if Then x ip =0 (affinity gating constraint), τ V Indicates the affinity threshold;

[0068] Group 3, x ip ∈ {0, 1} (binary constraint).

[0069] The first set of one-to-one matching constraints ensures that each target is assigned at most one matching object (allowing new or vanished targets not to be matched); the second set of gating constraints uses an affinity threshold τ. VPre-filtering impossible matching pairs reduces the search space. In this embodiment, τ V The first constraint is 0.4; the third set of binary constraints limits the decision variables to discrete binary variables, ensuring that the matching result is a deterministic integer solution. μ3 is the balance coefficient, used to adjust the weight of edge affinity in the total score. Let n be the set of edges between targets in the nth frame. Let x be the set of edges between targets in the (n-1)th frame. ip This indicates whether the current frame node is correctly displayed. Matched the node from the previous frame .

[0070] Step 8, perform continuous relaxation and Sinkhorn projection solution, including:

[0071] Since the IQP established in step 7 is an NP-hard problem, this embodiment uses a continuous relaxation strategy to transform it into a convex approximation problem that can be solved efficiently. First, the binary constraint x... ip The relaxation of ∈ {0,1} is a continuous constraint x ip ∈[0,1], the feasible region of the decision matrix X is expanded into a double stochastic matrix space; subsequently, the Sinkhorn iterative algorithm is used to alternately normalize the row and column of the relaxed affinity matrix:

[0072] Where r and c are the normalized vectors that make the row sum and column sum satisfy the constraints, respectively. Let be the decision matrix at the t-th iteration, and diag(r) represent converting vector r into a diagonal matrix. The Sinkhorn iteration converges exponentially, typically achieving engineering accuracy (error < 10) in 3-5 iterations. -6 The time taken for a single solution is less than 0.1ms, ensuring real-time performance.

[0073] Step 9, perform Hungarian discretization and optimal matching output, including:

[0074] The continuous soft allocation matrix obtained by Sinkhorn iterative projection in step 8 The Hungarian algorithm (also known as the Kuhn-Munkres algorithm) is applied for discretization, mapping the continuous solutions to integer optimal solutions that satisfy one-to-one constraints. The time complexity of the Hungarian algorithm is O(n log n). For typical traffic scenarios with a number of targets (m≤50), the solution can be completed in sub-millisecond time, fully meeting real-time requirements, and the final output matching results are... This is the globally optimal target association scheme between adjacent frames.

[0075] For example, when there are 8 targets in a scenario, traditional greedy matching may result in a local optimum causing target A to preempt the correct match of target B, while IQP global optimization uses second-order terms. By taking into account the spatial structural constraints of A and B, it can identify global allocation schemes with higher overall scores, thereby avoiding such cascading errors.

[0076] Step 10, perform adaptive reverse backtracking and trajectory chain generation, including:

[0077] Starting from the current frame N, using the matching results from steps 7 to 9, perform frame-by-frame reverse backtracking independently for each target along the negative time axis (N-1, N-2, ...). Once the maximum node affinity at a certain moment falls below the affinity threshold τ... V If no candidate node can be found, the trajectory extension of the target is terminated. Therefore, the system performs a trajectory extension for each target in the current frame. Generate a variable-length historical trajectory chain :

[0078] in The length of the historical trajectory chain (i.e., the number of observation frames it contains) varies for different targets, reflecting the adaptive nature of the chain. This represents the distance of the trajectory chain. For example, target A may have a historical trajectory of length 10 (N→N-9), while the newly emerging target B may only have a historical trajectory of length 2 (N→N-1), demonstrating the adaptive characteristic.

[0079] Step 11, determine the adaptive prediction order and calculate the motion derivative, including:

[0080] For each historical trajectory chain output in step 10, determine the order of the Taylor series expansion from the position sequence. Calculate average speed σ, the standard deviation of velocity variation v And the curvature index κ; where the average velocity is defined as the mean of the displacement between adjacent frames, the velocity standard deviation captures the degree of change in acceleration, and the curvature index reflects the curvature of the trajectory (calculated by the change in the angle between adjacent velocity vectors). Next, the motion complexity index Γ is constructed:

[0081] Among them W k W v W σ W κ This represents the weighting coefficient; in this embodiment, the weighting coefficient is... Each normalization factor k max、 vmax、 σ max、 κ max Determined using the standard scenario test set: k max The maximum length limit for the observation trajectory (usually 50 frames), v max σ represents the target typical speed limit (50 pixels / frame). max κ is the threshold for the significance of speed changes (taken as 10 pixels / frame). max This is the limit value of the trajectory curvature (taken as 0.1 radians / frame).

[0082] Based on the threshold range of the Γ value, the order n∈{1,2,3,4} is automatically selected:

[0083] Γ<0.25:n=1 (constant velocity)

[0084] 0.25≤Γ<0.55:n=2(smooth speed change)

[0085] 0.55≤Γ<0.75:n=3(significant nonlinearity)

[0086] Γ≥0.75: n=4 (highly complex).

[0087] Finally, the central difference method is used to calculate the motion derivatives up to the nth order. For the jth order derivative, j≥1, the recursive formula of adjacent difference sequences is used, keeping the time complexity at a lightweight level of O(n).

[0088] Step 12, perform Taylor series extrapolation prediction and output the predicted node set, including:

[0089] Based on the prediction order n and the motion derivatives of each order determined in step 11, for each target... The complete motion state of N+K at future time points is predicted using Taylor series extrapolation. The prediction process involves independently expanding each component of the center position x, y and the bounding box dimensions w, h.

[0090] The center position of the target Perform Taylor expansions along the x-axis and y-axis respectively:

[0091]

[0092] Target bounding box size Taylor expansion is also performed independently:

[0093]

[0094] The above expansion can be uniformly represented as:

[0095]

[0096] Let j represent the predicted parameter value for the N+Kth frame in the future, and j! represent the factorial of j. This represents the j-th derivative of parameter p at the current frame N. Let j represent K raised to the power of K. For the current observation value, Let be the j-th order derivative of motion calculated in step 11 using the finite difference method. The truncation error is... .

[0097] Obtain the predicted location and predicted bounding boxes ,in Let x and y be the x and y coordinates of the i-th target in the N+K-th frame. Let w and h be the bounding box values ​​of the i-th target in the N+K-th frame. The truncation error O(K) n+1 The accuracy of the prediction is adaptively determined by the prediction order n; secondly, prediction nodes are constructed: for each target Construct prediction nodes containing three types of complete information. ,in This represents the target category of the i-th target in the N-th frame. This represents the predicted position of the i-th target in the N+K-th frame. The bounding box of the i-th target in the N+K-th frame is represented by the target category. Keeping the position and bounding box unchanged, the final output is the predicted node set:

[0098]

[0099] The complete information of the predicted node representing the m-th target in the N+K-th frame can be directly used by downstream modules such as decision planning, collision avoidance, and trajectory planning.

[0100] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A dynamically adaptive graph-based image target tracking and motion prediction method, characterized in that, include: Step 1: Acquire image frames and buffer them; Step 2: Perform target detection and extract semantic attributes; Step 3: Construct graph nodes with enhanced confidence; Step 4: Construct a sparse graph topology based on the Delaunay triangulation algorithm; Step 5: Calculate the confidence-weighted node affinity; Step 6, calculate edge affinity; Step 7: Construct a quadratic integer programming matching model; Step 8: Perform continuous relaxation and Sinkhorn projection solution; Step 9: Perform Hungarian discretization and optimal matching output; Step 10: Perform adaptive reverse backtracking and trajectory chain generation; Step 11: Determine the adaptive prediction order and calculate the motion derivative; Step 12: Perform Taylor series extrapolation prediction and output the predicted node set.

2. The method for dynamic adaptive graph-based image target tracking and motion prediction according to claim 1, characterized in that, Step 1 includes: The image sensor continuously acquires a sequence of raw image frames, and the currently received Nth frame is denoted as I. N And store it in the frame buffer queue Q I This frame buffer queue maintains the image data of the most recent W frames in a first-in-first-out manner, that is: ， Where W is the queue window length; Step 2 includes: For the current frame image I obtained in step 1 N Execute the YOLO object detection algorithm to obtain the set of objects within the current field of view. : ， Where m is the number of targets detected in the current frame, and for each detected target... Extracting semantic attribute triples : ， in, Represents the i-th target category of the N-th frame image. Let (x, y) represent the pixel coordinates (x, y) of the i-th target center point in the N-th frame of the image in the image coordinate system. The bounding box size of the i-th target in the N-th frame image, including width w and height h, is represented by the detection result Q. N Add to the detection result queue Q D This makes it compatible with the frame buffer queue Q. I Maintain time alignment: 。 3. The method for dynamic adaptive graph-based image target tracking and motion prediction according to claim 1, characterized in that, Step 3 includes: Each detection target obtained in step 2 Mapped to nodes of a graph Besides semantic attributes In addition, each node also carries the detection confidence score output by the object detection algorithm. To form enhanced node attributes : ， Based on the confidence score, a normalized weight is calculated for each node. : ， The node set V of the current frame N N Recorded as: ， This represents the mapping of the graph node corresponding to the i-th target in the N-th frame of the image; For the detection result queue Q D Each historical frame stored in the system undergoes the same node mapping and weight calculation operations to form an enhanced node set for each frame.

4. The method for dynamic adaptive graph-based image target tracking and motion prediction according to claim 1, characterized in that, Step 4 includes: The same frame node set V obtained in step 3 N Target center pixel coordinates of each node Using control points, calculate the Delaunay triangulation on the 2D image plane. : ， Delaunay is a triangulation algorithm that only considers adjacent node pairs determined by Delaunay triangulation. Construct directed edges between them Each edge encodes the relative spatial vector between two targets. for: ， This forms the sparse relation graph of the current frame. The relationship diagrams of each frame are organized according to time sequence to form a spatiotemporal relationship diagram structure. : 。 5. The method for dynamic adaptive graph-based image target tracking and motion prediction according to claim 1, characterized in that, Step 5 includes: In the spatiotemporal relationship graph constructed in step S4, for any two nodes in two adjacent frames n and n-1... and Calculate the node affinity A for the fusion detection confidence. V : ， in, This is a confidence geometric mean adjustment factor. When both nodes come from high-confidence detections, its value is close to 1. When either node is a low-confidence detection, it automatically reduces the affinity score of the matching pair. Used to calculate the matching degree between target i and target p in two adjacent frames n and n-1, equal V For a category matching function, if and only if Returns 1 if the condition is met, otherwise returns 0. This represents the distance between the center points of the i-th target in the n-th frame and the p-th target in the (n-1)-th frame. μ1 and μ2 represent the difference between the bounding boxes of the i-th target in frame n and the p-th target in frame (n-1); μ1 and μ2 are weighting coefficients; λ1 and λ2 are normalization coefficients; and e is the natural constant.

6. The method for dynamic adaptive graph-based image target tracking and motion prediction according to claim 1, characterized in that, Step 6 includes: In step 4, among the edges determined by Delaunay triangulation, calculate the corresponding edges in adjacent frames. and The similarity, i.e., edge affinity A E : ， Where cp represents the type pair of connected nodes. Representative category pair and The matching degree, where m is the relative space vector defined in step S4. Represents the target vector and The distance is λ3, where λ3 is the normalization coefficient and e is the natural constant.

7. The method for dynamic adaptive graph-based image target tracking and motion prediction according to claim 1, characterized in that, Step 7 includes: The target association problem between adjacent frames is modeled as a quadratic integer programming problem. The globally optimal one-to-one matching scheme is solved by jointly optimizing node affinity and edge affinity, and a binary decision variable matrix is ​​defined. , where m and m' are the target numbers in the current frame and the previous frame, respectively, with the optimization objective being to maximize the joint score of node affinity and edge affinity: ， x ip =1 indicates that the current frame node will be... Matched the node from the previous frame ; Let J(X) represent the function that maximizes the matching score, with three sets of constraints: Group 1 ; The second group, if Then x ip =0; Group 3, x ip ∈ {0, 1}; In the formula, τ V This represents the affinity threshold, and μ3 is a balance coefficient used to adjust the weight of edge affinity in the total score. Let n be the set of edges between targets in the nth frame. Let x be the set of edges between targets in the (n-1)th frame. ip This indicates whether the current frame node is correctly displayed. Matched the node from the previous frame .

8. The method for dynamic adaptive graph-based image target tracking and motion prediction according to claim 7, characterized in that, Step 8 includes: First, apply the binary constraint x ip The relaxation of ∈ {0,1} is a continuous constraint x ip ∈[0,1], the feasible region of the decision matrix X is expanded into a birandom matrix space; subsequently, the Sinkhorn iterative algorithm is used to alternately normalize the row and column of the relaxed affinity matrix: ， Where r and c are the normalized vectors that make the row sum and column sum satisfy the constraints, respectively. Let be the decision matrix at the t-th iteration, and diag(r) represents converting vector r into a diagonal matrix; Step 9 includes: The continuous soft allocation matrix obtained by Sinkhorn iterative projection in step 8 The Hungarian algorithm is applied for discretization, mapping continuous solutions to integer optimal solutions that satisfy one-to-one constraints. The final output matching result This is the globally optimal target association scheme between adjacent frames.

9. The method for dynamic adaptive graph-based image target tracking and motion prediction according to claim 8, characterized in that, Step 10 includes: Starting from the current frame N, using the matching results from steps 7 to 9, perform frame-by-frame reverse backtracking independently for each target along the negative time axis. Once the maximum node affinity at a certain moment falls below the affinity threshold τ, V If no candidate node can be found, the trajectory extension of the target is terminated. Therefore, for each target in the current frame... Generate a variable-length historical trajectory chain : ， in The length of this historical trajectory chain, This represents the distance of the trajectory chain.

10. The method for dynamic adaptive graph-based image target tracking and motion prediction according to claim 9, characterized in that, Step 11 includes: For each historical trajectory chain output in step 10, determine the order of the Taylor series expansion from the position sequence. Calculate average speed σ, the standard deviation of velocity variation v And curvature index κ; secondly, construct motion complexity index Γ: ， Among them W k W v W σ W κ Represents the weighting coefficients, and each normalization factor k max、 v max、 σ max、 κ max To determine k through the standard scenario test set: max As a limit on the maximum length of the observed trajectory, v max σ is the upper limit of the typical target speed. max κ is the threshold for the significance of the velocity change. max This represents the limit value of the trajectory curvature; Based on the threshold range of the Γ value, the order n∈{1,2,3,4} is automatically selected: Γ<0.25:n=1, representing the case of constant velocity. 0.25≤Γ<0.55: n=2, representing a smooth speed change. 0.55≤Γ<0.75: n=3, representing a case of significant nonlinearity. Γ≥0.75: n=4, representing a highly complex case; Finally, the central difference method is used to calculate the motion derivatives up to the nth order. For the jth order derivative, j≥1, the recursive formula of adjacent difference sequences is used, keeping the time complexity at a lightweight level of O(n).

11. The method for dynamic adaptive graph-based image target tracking and motion prediction according to claim 10, characterized in that, Step 12 includes: First, a high-order Taylor extrapolation calculation is performed: Taylor expansion is applied to extrapolate and predict all four spatial components, which are the center position x, y and the bounding box dimensions w, h. ， Let j represent the predicted parameter value for the (N+K)th frame in the future, and j! represent the factorial of j. This represents the j-th derivative of parameter p at the current frame N. Represents K raised to the power of j; obtains the predicted position. and predicted bounding boxes ,in Let x and y be the x and y coordinates of the i-th target in the N+K-th frame. Let w and h be the bounding box of the i-th target in the N+K-th frame; where the truncation error O(K) n+1 The accuracy of the prediction is adaptively determined by the prediction order n; secondly, prediction nodes are constructed: for each target Construct prediction nodes containing three types of complete information. ,in This represents the target category of the i-th target in the N-th frame. This represents the predicted position of the i-th target in the N+K-th frame. This represents the bounding box of the i-th target in the N+K-th frame; finally, the output is the set of predicted nodes: summarizing the predicted nodes of all targets, and finally outputting the complete set of predicted nodes. , Complete information about the predicted node of the m-th target in the N+K-th frame.

Images