A spatiotemporal evolution enhanced ground target long-time series motion prediction method

Abstract

This invention relates to the fields of computer vision and video surveillance processing technology, and discloses a method and system for predicting long-term motion of ground targets with enhanced spatiotemporal evolution. The method extracts the state features of targets from video data streams to construct an observation state sequence, which is then processed by a sequence coding network to generate a hidden layer temporal feature vector. Simultaneously, the scene image is divided into discrete grids, and a static spatial topological mask matrix is ​​established based on the boundaries of physical obstacles. The hidden layer temporal feature vector is input into a hybrid density network to output an initial multimodal trajectory probability distribution. Spatial alignment and tensor inner product operations are performed between the mask matrix and the probability distribution, and asymmetric physical pruning and weight normalization are applied to the probability density to output an enhanced multimodal trajectory probability distribution. An irregular tracking search region is established, and similarity comparison is performed using historical appearance feature templates to output the real-time tracking position. This invention effectively integrates physical spatial constraints, improving the accuracy of occlusion re-capture and long-term tracking.

Description

Technical Field

[0001] This invention relates to the field of computer vision and video surveillance processing technology, specifically to a method for predicting the long-term motion of ground targets with spatiotemporal evolution enhancement. Background Technology

[0002] In the fields of video surveillance and computer vision, continuous and stable tracking of ground targets is a fundamental and core task. In complex real-world scenarios, targets often temporarily disappear due to their own maneuvering, environmental occlusion, or changes in perspective. To quickly and accurately retrieve targets when they reappear, motion prediction techniques are typically introduced to estimate the probability distribution of the target's future trajectory, thus assisting visual tracking algorithms. However, existing motion prediction and tracking methods still suffer from the following technical shortcomings in practical applications: The movement of ground targets is strictly constrained by the physical environment of the real-world scenario. Most existing trajectory probability prediction models are calculated in an assumed open free space, failing to effectively establish a topological relationship between the prediction algorithm and the monitored physical space. This results in the trajectory probability distribution output by the model often covering areas with inaccessible physical obstacles such as buildings, water bodies, or walls, making the prediction results lack basic spatial rationality and prone to generating prediction branches that deviate significantly from common sense.

[0003] The motion of a ground target is a continuous process with dynamic evolution. Current predictive tracking methods often rely excessively on the instantaneous state of the target at the current moment or on observation slices within a very short time window, failing to fully mine the kinematic characteristics of the target over a longer historical period. Due to the lack of effective accumulation and feature mapping of long-term temporal states and dynamic evolution parameters, the model struggles to provide accurate underlying feature support when facing complex maneuvers of the target, resulting in limited decoding accuracy of subsequent multimodal intentions.

[0004] When a target is physically occluded in a video frame, conventional visual tracking algorithms typically employ a global blind search or a rectangular search box that simply expands outward according to a fixed ratio during target recapture. This passive search mechanism, lacking guidance from motion patterns, not only generates a large amount of unnecessary computational redundancy but also easily leads to feature mismatches when there is significant background interference or similar targets are present, causing long-term tracking tasks to fail. Summary of the Invention

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for predicting the long-term motion of ground targets with enhanced spatiotemporal evolution, comprising the following steps: Step S100: Receive video data stream, extract the state features of ground targets in the video data stream at continuous time steps to construct an observation state sequence; obtain scene images corresponding to the video data stream, and perform discrete grid division on the monitoring physical space where the scene images are located to obtain multiple two-dimensional discrete grids; Step S200: The observation state sequence is input into the configured sequence coding network for processing to generate hidden layer temporal feature vectors; the boundaries of physical obstacles in the scene image are determined, and corresponding matrix element values ​​are assigned to each of the two-dimensional discrete grids according to the boundaries of physical obstacles to characterize the spatial reachability state, and a static spatial topology mask matrix is ​​established. Step S300: Input the hidden layer temporal feature vector into the configured hybrid density network for decoding calculation, and output an initial multimodal trajectory probability distribution. The initial multimodal trajectory probability distribution contains multiple Gaussian components, and each Gaussian component has a corresponding probability density. Step S400: Perform spatial alignment calculation and tensor inner product operation between the static spatial topology mask matrix and the initial multimodal trajectory probability distribution; use the matrix element values ​​of the static spatial topology mask matrix to perform asymmetric physical pruning on the probability density of each Gaussian component, and perform probability weight normalization calculation based on the calculation result of the tensor inner product operation to output the enhanced multimodal trajectory probability distribution. Step S500: Based on the enhanced multimodal trajectory probability distribution, determine the probability contour line boundary parameters, set the video frame coordinate system, and establish an irregular tracking search area in the video frame coordinate system; acquire the video image frame of the video data stream at the latest time step, acquire the pre-extracted historical appearance feature template of the ground target, limit it to the irregular tracking search area, and use the configured visual tracking algorithm network to perform visual image feature similarity comparison calculation with the historical appearance feature template and the video image frame at the latest time step, and output the real-time tracking position of the ground target.

[0006] Preferably, the step of extracting the state features of ground targets in the video data stream at consecutive time steps to construct an observation state sequence specifically includes: Multiple historical video image frames are extracted from the video data stream. A preset target detection algorithm model is called to extract the two-dimensional center pixel of the ground target in each historical video image frame, and the position coordinate data of each historical video image frame corresponding to the time step are mapped to generate the position coordinate data of each historical video image frame. Extract adjacent historical video image frames from the plurality of historical video image frames, obtain the time interval between the adjacent historical video image frames, perform time derivative calculation in combination with the time interval, and obtain the velocity vector data of the corresponding time step; The position coordinate data belonging to the same time step are concatenated with the velocity vector data to generate a single-frame state feature vector corresponding to each time step. Multiple single-frame state feature vectors located within a preset historical observation time window are extracted, combined, and then preprocessed by extreme value scaling to construct the observation state sequence.

[0007] Preferably, the sequence coding network employs a long short-term memory network and includes multiple time-step computation nodes, each of which contains a gating structure and a cell state unit; the step of processing the observed state sequence input into the configured sequence coding network to generate a hidden layer temporal feature vector specifically includes: Multiple single-frame state feature vectors arranged in time step order are extracted from the observed state sequence, and each single-frame state feature vector is sequentially input into each time step calculation node of the sequence coding network. When processing each of the time step computing nodes, the hidden state data transmitted by the previous time step computing node is received, the single frame state feature vector of the current time step is fused with the hidden state data, the cell state unit is updated through the gating structure and the feature output value is output. The feature output value generated at the last time step calculation node in the time dimension is extracted and mapped to the hidden layer temporal feature vector.

[0008] Preferably, the step of determining the boundaries of physical obstacles in the scene image, assigning corresponding matrix element values ​​to each of the two-dimensional discrete grids based on the boundaries of the physical obstacles to characterize the spatial reachability state, and establishing a static spatial topology mask matrix specifically includes: The scene image is processed using a pre-set semantic segmentation network to obtain an environment category label. Based on the environment category label, a set of obstacle pixel coordinates representing impassable areas is selected. The obstacle pixel coordinate set is then subjected to edge fitting to determine the boundary of the physical obstacle. The boundaries of the physical obstacles are mapped to the interior of each of the two-dimensional discrete grids, and the coverage overlap ratio between the boundaries of the physical obstacles and each of the two-dimensional discrete grids is calculated. The matrix element values ​​corresponding to the two-dimensional discrete grids whose overlapping area ratio is greater than a preset area threshold are set to zero to represent the unreachable state, and the matrix element values ​​corresponding to the remaining two-dimensional discrete grids are set to one to represent the reachable state, thus constructing the static spatial topology mask matrix.

[0009] Preferably, the hybrid density network includes a shared feature extraction layer, a weighted regression branch, a two-dimensional mean regression branch, and a covariance regression branch; the step of inputting the hidden layer temporal feature vector into the configured hybrid density network for decoding and calculation, and outputting the initial multimodal trajectory probability distribution, specifically includes: The hidden layer temporal feature vectors are mapped by the shared feature extraction layer to output a high-dimensional joint feature matrix; The high-dimensional joint feature matrix is ​​calculated in parallel by the weight regression branch, the two-dimensional mean regression branch, and the covariance regression branch, and outputs mixed weight prediction data, location mean prediction data, and covariance prediction data respectively. The mixed weight prediction data contains multiple normalized weight values, and the location mean prediction data and the covariance prediction data respectively contain mean sub-data and covariance sub-data corresponding to each Gaussian component. A Gaussian mixture model containing multiple Gaussian components is constructed using the mixed weight prediction data, the location mean prediction data, and the covariance prediction data, and the Gaussian mixture model is confirmed as the initial multimodal trajectory probability distribution.

[0010] Preferably, the step of performing the spatial alignment calculation and tensor inner product operation between the static spatial topological mask matrix and the initial multimodal trajectory probability distribution specifically includes: Determine the grid division dimensions corresponding to the plurality of two-dimensional discrete grids, and establish a discrete coordinate sampling grid based on the grid division dimensions; The coordinates of each grid center point contained in the discrete coordinate sampling grid are determined. The probability density values ​​are calculated by performing probability density calculation on the coordinates of each grid center point using the initial multimodal trajectory probability distribution. Based on the probability density values, multiple initial probability density matrices corresponding one-to-one with each Gaussian component are generated. Traverse all the initial probability density matrices, and perform element-wise multiplication of the probability density values ​​in each initial probability density matrix with the matrix element values ​​at the same coordinate positions in the static spatial topology mask matrix to generate multiple aligned and fused tensor matrices containing tensor element values, which are used as the calculation result of the tensor inner product operation.

[0011] Preferably, the step of performing asymmetric physical pruning on the probability density of each Gaussian component using the matrix element values ​​of the static spatial topological mask matrix specifically includes: Extract the tensor element values ​​from each of the alignment and fusion tensor matrices, perform a global summation calculation on the tensor element values, and generate spatial topology validity weights corresponding to each of the Gaussian components; Extract the multiple normalized weight values ​​contained in the mixed weight prediction data; Perform scalar multiplication on each of the normalized weight values ​​and the corresponding spatial topology validity weights to output multiple pruned weight values ​​to replace the original normalized weight values.

[0012] Preferably, the step of performing probability weight normalization calculation based on the calculation result of the tensor inner product operation to output the enhanced multimodal trajectory probability distribution specifically includes: The total weight value is obtained by summing all the pruned weight values. Divide each of the pruned weight values ​​by the total weight value to output multiple normalized updated weight values, and the sum of the multiple normalized updated weight values ​​is equal to one. An enhanced Gaussian mixture model is constructed by re-weighting and combining multiple normalized update weight values ​​with the position mean prediction data and the covariance prediction data, and the enhanced Gaussian mixture model is confirmed as the enhanced multimodal trajectory probability distribution.

[0013] Preferably, the step of determining the probability contour line boundary parameters based on the enhanced multimodal trajectory probability distribution, setting the video frame coordinate system, and establishing an irregular tracking search region in the video frame coordinate system specifically includes: Remove the normalized update weight values ​​and their corresponding Gaussian components that are less than the preset confidence threshold parameter, and construct the remaining Gaussian components into an effective prediction branch set. Obtain the mean sub-data and covariance sub-data corresponding to the retained Gaussian components, and perform contour line equation solving calculation based on Mahalanobis distance in combination with the preset confidence ellipse scale parameters to determine the probability contour line boundary parameters corresponding to each retained Gaussian component. Project all the probability contour line boundary parameters onto the video frame coordinate system to generate multiple two-dimensional probability elliptical regions. Perform geometric union calculation on the two-dimensional probability elliptical regions that have spatial overlapping states to extract the outer contour boundary. Generate a global search boundary formed by the outer contour boundary, extract a set of candidate pixel coordinates located inside the global search boundary, and confirm the set of candidate pixel coordinates as the irregular shape tracking search area; The process, confined within the irregular shape tracking search area, utilizes a configured visual tracking algorithm network to perform visual image feature similarity comparison calculations using the historical appearance feature templates and the video image frames from the latest time step, outputting the real-time tracking position of the ground target. Specifically, this includes: Calculate the minimum bounding rectangle boundary of the irregular tracking search region, and extract local search image blocks within the video image frame of the latest time step based on the minimum bounding rectangle boundary; The local search image patch is processed using the convolutional feature extraction layer configured inside the visual tracking algorithm network to extract the current appearance feature vector; The similarity measurement layer configured inside the visual tracking algorithm network is used to perform normalized cross-correlation calculation on the current appearance feature vector and the historical appearance feature template, and outputs a spatial feature response matrix; The response peak value is extracted from the spatial feature response matrix. When the response peak value is determined to be greater than or equal to a preset recapture matching threshold parameter, the recapture coordinate data corresponding to the response peak value is extracted, and the recapture coordinate data is confirmed as the real-time tracking position of the ground target.

[0014] Preferably, a spatiotemporally evolved enhanced long-term motion prediction system for ground targets is provided for executing the aforementioned spatiotemporally evolved enhanced long-term motion prediction method for ground targets, the system comprising: The data acquisition and temporal coding module is used to receive video data streams, extract the state features of ground targets in the video data streams at continuous time steps to construct observation state sequences, and input the observation state sequences into a configured sequence coding network for processing to generate hidden layer temporal feature vectors. The environmental topology modeling module is used to acquire scene images corresponding to the video data stream, divide the monitoring physical space where the scene image is located into discrete grids to obtain multiple two-dimensional discrete grids; determine the boundaries of physical obstacles in the scene image, assign corresponding matrix element values ​​to each of the two-dimensional discrete grids according to the boundaries of physical obstacles to represent the spatial reachability state, and establish a static spatial topology mask matrix. The multimodal probability prediction module is used to input the hidden layer temporal feature vector into the configured hybrid density network for decoding and calculation, and output an initial multimodal trajectory probability distribution. The initial multimodal trajectory probability distribution contains multiple Gaussian components, and each Gaussian component has a corresponding probability density. The collaborative pruning and enhancement module is used to perform spatial alignment calculation and tensor inner product operation between the static spatial topology mask matrix and the initial multimodal trajectory probability distribution; to perform asymmetric physical pruning on the probability density of each Gaussian component using the matrix element values ​​of the static spatial topology mask matrix, and to perform probability weight normalization calculation based on the calculation result of the tensor inner product operation, and output the enhanced multimodal trajectory probability distribution. The irregular tracking guidance module is used to determine the probability contour line boundary parameters based on the enhanced multimodal trajectory probability distribution, set the video frame coordinate system, and establish an irregular tracking search area in the video frame coordinate system; acquire the video image frame of the video data stream at the latest time step, acquire the pre-extracted historical appearance feature template of the ground target, limit it to the irregular tracking search area, and use the configured visual tracking algorithm network to perform visual image feature similarity comparison calculation between the historical appearance feature template and the video image frame at the latest time step, and output the real-time tracking position of the ground target. This invention provides a method for predicting the long-term motion of ground targets with enhanced spatiotemporal evolution. It has the following beneficial effects: (1) This invention extracts the boundaries of physical obstacles in the scene image through the environmental topology modeling module, assigns corresponding matrix element values ​​to the two-dimensional discrete grid according to the boundaries of physical obstacles, establishes a static spatial topology mask matrix, and performs spatial alignment calculation and tensor inner product operation between the static spatial topology mask matrix and the initial multimodal trajectory probability distribution using the collaborative pruning and enhancement module. The matrix element values ​​of the static spatial topology mask matrix are used to perform asymmetric physical pruning on the probability density of each Gaussian component in the initial multimodal trajectory probability distribution, outputting an enhanced multimodal trajectory probability distribution. The spatial topology validity weight is used to suppress the normalized weight values ​​pointing to physically unreachable regions, attenuating the weight ratio of erroneous Gaussian components, effectively integrating the physical boundary constraints of the monitored physical space into the prediction model, and improving the spatial rationality of trajectory prediction.

[0015] (2) This invention utilizes a data acquisition and temporal coding module to extract the observation state sequence of the target from the video data stream. This observation state sequence contains the target's position coordinates and velocity values ​​at consecutive time steps. The data acquisition and temporal coding module inputs the observation state sequence into an internally configured sequence coding network for processing, completing the generation and mapping of hidden layer temporal feature vectors. Through the computation of time step calculation nodes, the generated hidden layer temporal feature vectors can effectively contain the kinematic features and dynamic evolution parameters accumulated by the target within the historical observation time window. This provides a reliable data input basis for the subsequent multimodal decoding calculation of the multimodal probability prediction module, ensuring that the prediction process fully extracts the historical dynamic features of the target.

[0016] (3) The irregular tracking guidance module of the present invention receives the enhanced multimodal trajectory probability distribution, extracts the probability contour boundary parameters of the enhanced multimodal trajectory probability distribution, and establishes an irregular tracking search area in the video frame coordinate system based on the probability contour boundary parameters. Then, within the pixel range boundary of the irregular tracking search area, the internally configured visual tracking algorithm network is used to perform visual image feature similarity comparison calculation by combining historical appearance feature templates with the latest time step video image frame data of the video data stream, and outputs the real-time tracking position of the target. The irregular tracking search area with irregular topology structure output by the prediction model is used to guide the visual tracking algorithm network, avoiding blind search of the whole image, and effectively realizing the occlusion recapture and long-term stable tracking of ground targets. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the system framework of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] See attached document Figure 2 This invention provides a long-term motion prediction system for ground targets with spatiotemporal evolution enhancement, including a data acquisition and temporal coding module, an environmental topology modeling module, a multimodal probability prediction module, a collaborative pruning and enhancement module, and an irregular tracking guidance module.

[0020] The data acquisition and temporal coding module receives externally input video data streams, extracts the observation state sequence of the target from the video data stream, and the observation state sequence contains the target's position coordinates and velocity values ​​at continuous time steps. The data acquisition and temporal coding module processes the observation state sequence using an internally deployed sequence coding network, and outputs the hidden layer temporal feature vector after feature mapping calculation.

[0021] The environmental topology modeling module acquires the scene image corresponding to the video data stream, divides the monitoring physical space where the scene image is located into a two-dimensional discrete mesh, and establishes a static spatial topology mask matrix based on the boundaries of the physical obstacles in the scene image. The matrix element values ​​of the static spatial topology mask matrix are used to distinguish the reachable and unreachable states of the two-dimensional discrete mesh.

[0022] The multimodal probability prediction module is connected to the data acquisition and temporal coding module. The multimodal probability prediction module receives the hidden layer temporal feature vector, inputs the hidden layer temporal feature vector into the hybrid density network for decoding and calculation, and calculates the initial multimodal trajectory probability distribution containing multiple Gaussian components.

[0023] The collaborative pruning and enhancement module is communicatively connected to the environmental topology modeling module and the multimodal probability prediction module, respectively. The collaborative pruning and enhancement module obtains the static spatial topology mask matrix and the initial multimodal trajectory probability distribution, performs spatial alignment and tensor inner product operation on the static spatial topology mask matrix and the initial multimodal trajectory probability distribution, and performs probability weight normalization calculation on the numerical results of the tensor inner product operation to output the enhanced multimodal trajectory probability distribution.

[0024] The irregular tracking guidance module is communicatively connected to the collaborative pruning and enhancement module. The irregular tracking guidance module receives the enhanced multimodal trajectory probability distribution, extracts the probability contour boundary parameters of the enhanced multimodal trajectory probability distribution, establishes an irregular tracking search region in the video frame coordinate system based on the probability contour boundary parameters, extracts the historical appearance feature template of the target, limits it to the pixel range boundary of the irregular tracking search region, and performs visual image feature similarity comparison calculation using the historical appearance feature template and the latest time step video image frame data of the video data stream, outputting the real-time tracking position of the target.

[0025] See attached document Figure 1 Furthermore, this invention provides a method for predicting the long-term motion of ground targets with enhanced spatiotemporal evolution. This method is executed based on the aforementioned system for predicting the long-term motion of ground targets with enhanced spatiotemporal evolution, and the specific steps are as follows: In step S100, the data acquisition and timing coding module receives the video data stream and extracts state features from the video data stream to construct and preprocess the observation state sequence of the target. The observation state sequence includes the position coordinates and velocity values ​​of the target in continuous time steps. At the same time, the environmental topology modeling module acquires the scene image corresponding to the video data stream and performs two-dimensional discrete grid division on the monitoring physical space where the scene image is located.

[0026] In step S200, the data acquisition and temporal coding module inputs the observed state sequence into the internally configured sequence coding network for processing, and completes the generation and mapping of the hidden layer temporal feature vector. At the same time, the environment topology modeling module extracts the boundaries of physical obstacles in the scene image, and assigns corresponding matrix element values ​​to the two-dimensional discrete grid according to the boundaries of physical obstacles, thereby establishing a static spatial topology mask matrix. The static spatial topology mask matrix is ​​aligned with the two-dimensional discrete grid in terms of spatial scale, and the matrix element values ​​of the static spatial topology mask matrix indicate the spatial reachability of the two-dimensional discrete grid.

[0027] In step S300, the multimodal probability prediction module receives the hidden layer temporal feature vector and inputs it into the internally configured hybrid density network for calculation, thereby outputting an initial multimodal trajectory probability distribution containing multiple Gaussian components.

[0028] In step S400, the collaborative pruning and enhancement module obtains the static spatial topology mask matrix and the initial multimodal trajectory probability distribution, performs spatial alignment calculation of the static spatial topology mask matrix and the initial multimodal trajectory probability distribution, and then performs tensor inner product operation after spatial alignment. The collaborative pruning and enhancement module uses the matrix element values ​​of the static spatial topology mask matrix to perform asymmetric physical pruning on the probability density of each Gaussian component in the initial multimodal trajectory probability distribution, and performs probability weight normalization calculation based on the numerical results of the tensor inner product operation to output the enhanced multimodal trajectory probability distribution.

[0029] In step S500, the irregular tracking guidance module receives the enhanced multimodal trajectory probability distribution, extracts the probability contour boundary parameters of the enhanced multimodal trajectory probability distribution, and establishes an irregular tracking search area in the video frame coordinate system based on the probability contour boundary parameters. Subsequently, the irregular tracking guidance module extracts the historical appearance feature template of the target, limits it to the pixel range boundary of the irregular tracking search area, and uses the internally configured visual tracking algorithm network to perform visual image feature similarity comparison calculation by combining the historical appearance feature template with the latest time step video image frame data of the video data stream. Based on the result of the visual image feature similarity comparison calculation, the module outputs the real-time tracking position of the target.

[0030] Furthermore, the specific workflow for constructing and preprocessing the observed state sequence in step S100 is as follows: The data acquisition and timing coding module calls the preset target detection algorithm model to perform target detection calculations frame by frame on the received video data stream, obtains the two-dimensional center pixel of the target detection box, and maps the two-dimensional center pixel to the set video screen coordinate system to confirm it as position coordinate data. The position coordinate data includes the horizontal coordinate component and the vertical coordinate component in the two-dimensional plane.

[0031] The data acquisition and timing coding module calculates the difference in position coordinate data between adjacent image frames in the video data stream, obtains the frame rate parameter of the video data stream, calculates the time interval between adjacent image frames, and performs time derivative calculation in combination with the time interval between adjacent image frames to obtain velocity vector data containing horizontal and vertical velocity components.

[0032] The data acquisition and timing coding module concatenates the position coordinate data and velocity vector data at the same time step to generate a single-frame state feature vector for the corresponding time step, making the single-frame state feature vector present as a four-dimensional column vector data structure.

[0033] The data acquisition and time-series coding module sets a fixed historical observation time window, extracts multiple single-frame state feature vectors within the historical observation time window in chronological order of time steps, and combines the extracted single-frame state feature vectors in the time dimension to construct an observation state sequence.

[0034] The data acquisition and time-series coding module uses a numerical normalization algorithm to perform extreme value scaling preprocessing calculations on the position coordinate data and velocity vector data within the observation state sequence. This unifies all the values ​​contained in the observation state sequence within a preset numerical range, thereby eliminating the dimensional differences between the position coordinate data and the velocity vector data, and completing the construction and preprocessing of the observation state sequence.

[0035] Furthermore, the specific workflow for generating and mapping the hidden layer temporal feature vectors in step S200 is as follows: The data acquisition and time-series coding module has a sequence coding network deployed inside. The sequence coding network adopts the topology of a long short-term memory network. The data acquisition and time-series coding module extracts multiple single-frame state feature vectors arranged in chronological order within the observation state sequence, and inputs the multiple single-frame state feature vectors into the time-step calculation nodes of the sequence coding network in sequence according to the time step order.

[0036] Each time step computation node of the sequence coding network contains a forget gate structure, an input gate structure, an output gate structure, and a cell state unit. When processing the current time step computation node, the sequence coding network receives the single-frame state feature vector at the current time, the cell state data and hidden state data passed from the previous time step computation node, and performs a concatenation operation on the single-frame state feature vector at the current time and the hidden state data.

[0037] The sequence coding network operates a forget gate structure, using the internally configured forget gate weight matrix and forget gate bias term to perform matrix multiplication and activation function calculation on the concatenated data to determine the retention weight values ​​of historical observation data. The sequence coding network operates an input gate structure, using the internally configured input gate weight matrix and input gate bias term to perform matrix multiplication and activation function mapping calculation on the concatenated data to determine the new state value to be written to the current cell state unit.

[0038] The sequence coding network adds the retained weight values ​​to the newly added state values ​​to complete the numerical update calculation of the current cell state unit. Subsequently, the sequence coding network runs the output gate structure, using the internally configured output gate weight matrix and output gate bias term to calculate the concatenated data. Combined with the updated current cell state unit, it calculates the feature output value of the computation node at the current time step. Specifically, the mathematical logic of the above calculation process within the sequence coding network is as follows: in, Represents the current time step. Represents the previous time step; A single-frame state feature vector representing the input at the current time step; and These respectively represent the hidden state data and cell state data passed from the previous time step; The representation involves concatenating the hidden state data with the single-frame state feature vector. , , , The weight matrices represent the internal configurations of the forget gate structure, input gate structure, cell state structure, and output gate structure, respectively. , , , Each term represents the bias term corresponding to the gate structure; Characterizes the matrix multiplication operation; Characterizes the Sigmoid activation function; Characterizing the hyperbolic tangent activation function; The retention weight values ​​represent the calculated output of the forget gate structure; Characterizes the feature extraction state of the input gate structure computation output; The newly added state values ​​are represented by the calculated values; Represents the corresponding multiplication operation of matrix elements; The current cell state unit after the update is completed; Characterize the characteristic states of the output gate structure for computational output; The sequence coding network represents the feature output value of the computation node at the current time step. It is passed as an input parameter to the next time step computation node.

[0039] After all single-frame state feature vectors in the observation state sequence have been input and calculated, the data acquisition and temporal coding module extracts the feature output value generated by the calculation node of the last time step in the time dimension of the sequence coding network, and maps and confirms the feature output value as the hidden layer temporal feature vector. The hidden layer temporal feature vector contains the kinematic features and dynamic evolution parameters accumulated by the target in the historical observation time window.

[0040] The data acquisition and temporal coding module encapsulates the hidden layer temporal feature vectors into a one-dimensional tensor and transmits them to the multimodal probability prediction module, providing a data input basis for subsequent multimodal decoding calculations.

[0041] Furthermore, the environmental topology modeling module performs a two-dimensional discrete mesh division of the monitored physical space where the scene image is located. Specifically: The environmental topology modeling module extracts the pixel width and pixel height values ​​of the scene image and obtains the system's preset spatial quantization step size parameter. The environmental topology modeling module divides the pixel width and pixel height values ​​by the spatial quantization step size parameter to calculate the number of grid columns in the horizontal direction and the number of grid rows in the vertical direction. Based on the number of grid columns in the horizontal direction and the number of grid rows in the vertical direction, the physical space is directly divided into multiple two-dimensional discrete grids arranged in a two-dimensional matrix in the two-dimensional plane corresponding to the scene image.

[0042] The environmental topology modeling module establishes a grid index coordinate system and assigns a unique row index and column index to each two-dimensional discrete grid according to the arrangement order of the two-dimensional matrix. The environmental topology modeling module constructs a coordinate mapping function, which includes floor function logic. The environmental topology modeling module inputs the continuous pixel coordinate data in the scene image into the coordinate mapping function to perform division and floor function based on the spatial quantization step size parameter. Through the calculation output of the coordinate mapping function, the pixel coordinate data inside the scene image is converted into specific row index and column index. Based on the converted row index and column index, the spatial belonging relationship of the pixel coordinate data in multiple two-dimensional discrete grids is determined.

[0043] Furthermore, the environmental topology modeling module extracts entity obstacle boundaries and establishes a static spatial topology mask matrix based on the spatial attribution relationship of pixel coordinate data in multiple two-dimensional discrete grids, as follows: The environmental topology modeling module extracts image pixel data from scene images and processes the image pixel data using an internally configured semantic segmentation network. It outputs pixel-level environmental category labels through feature classification calculation of the semantic segmentation network. Based on preset physical access attribute rules, the environmental topology modeling module selects a set of pixel coordinates representing impassable areas from the environmental category labels. The preset physical access attribute rules define the mapping relationship between the semantic labels of building, water body, and wall categories and impassable attributes. The environmental topology modeling module performs edge fitting on the set of pixel coordinates to generate physical obstacle boundaries.

[0044] The environmental topology modeling module, in conjunction with the spatial attribution relationship established by the aforementioned coordinate mapping function, maps the spatial coordinates of the physical obstacle boundary to the interior of multiple corresponding two-dimensional discrete grids. Based on the mapping results, it calculates the coverage overlap ratio between the physical obstacle boundary and each two-dimensional discrete grid. The environmental topology modeling module determines that the physical properties of two-dimensional discrete grids with a coverage overlap ratio greater than a preset area threshold are in an unreachable state, and determines that the physical properties of two-dimensional discrete grids with no physical obstacle boundary coverage or a coverage overlap ratio less than or equal to a preset area threshold are in a reachable state.

[0045] The environmental topology modeling module constructs an initial numerical matrix based on the row and column indexes of multiple two-dimensional discrete grids. It sets the matrix element values ​​of the two-dimensional discrete grids in the unreachable state to zero, and sets the matrix element values ​​of the two-dimensional discrete grids in the reachable state to one.

[0046] After completing the numerical assignment of matrix elements for all two-dimensional discrete meshes, the environmental topology modeling module confirms the assigned initial numerical matrix as a static spatial topology mask matrix. The numerical values ​​of the matrix elements inside the static spatial topology mask matrix indicate the spatial reachability of the monitored physical space. The environmental topology modeling module transmits the static spatial topology mask matrix to the collaborative pruning and enhancement module to provide a data benchmark for subsequent tensor space alignment calculations.

[0047] Furthermore, the multimodal probability prediction module receives the hidden layer temporal feature vector and inputs it into the internally configured hybrid density network for calculation, thereby outputting an initial multimodal trajectory probability distribution containing multiple Gaussian components. The specific process is as follows: The multimodal probability prediction module is equipped with a hybrid density network, which adopts a multi-branch decoder topology. The hybrid density network includes a shared feature extraction layer and independent parallel weight regression branch, two-dimensional mean regression branch and covariance regression branch.

[0048] The multimodal probability prediction module inputs the hidden layer temporal feature vector into the shared feature extraction layer of the hybrid density network. The shared feature extraction layer contains a multi-layer fully connected perceptron structure. The shared feature extraction layer uses the multi-layer fully connected perceptron structure to perform matrix multiplication and nonlinear mapping calculations on the hidden layer temporal feature vector, and outputs a high-dimensional joint feature matrix.

[0049] The hybrid density network inputs the high-dimensional joint feature matrix into the weighted regression branch, the two-dimensional mean regression branch, and the covariance regression branch for parallel computation to generate a parameter set with a preset number of Gaussian components. The weighted regression branch is equipped with a normalized exponential function operation layer, which is used to map the high-dimensional joint feature matrix and output hybrid weighted prediction data that satisfies the condition that the numerical summation equals one.

[0050] The two-dimensional mean regression branch contains a linear regression operation layer, which performs numerical regression calculations on the spatial coordinates of the high-dimensional joint feature matrix, outputting location mean prediction data. The covariance regression branch is internally configured with an exponential activation operation layer, which is used to perform positive definite constraint calculation on the high-dimensional joint feature matrix and output covariance prediction data representing the spatial discrete distribution range. The covariance prediction data includes the horizontal variance value, the vertical variance value, and the correlation coefficient value. The multimodal probability prediction module extracts the mixed weight prediction data, the position mean prediction data, and the covariance prediction data to provide the underlying numerical parameters for constructing the initial multimodal trajectory probability distribution containing multiple Gaussian components.

[0051] Furthermore, the multimodal probability prediction module extracts mixed-weight prediction data, location mean prediction data, and covariance prediction data to provide underlying numerical parameters for constructing an initial multimodal trajectory probability distribution containing multiple Gaussian components. The workflow is as follows: The multimodal probability prediction module acquires mixed weight prediction data, location mean prediction data, and covariance prediction data, and performs a combination mapping calculation on the location mean prediction data and covariance prediction data to construct multiple two-dimensional Gaussian probability density functions corresponding to the preset number of Gaussian components.

[0052] The location mean prediction data includes the mean horizontal and vertical coordinates in a two-dimensional plane, used to indicate the coordinates of the geometric center point of the target's trajectory. The covariance prediction data includes the variance values ​​in the horizontal direction, the variance values ​​in the vertical direction, and the correlation coefficient values. The multimodal probability prediction module uses the covariance prediction data to construct a two-dimensional covariance matrix, using the variance values ​​in the horizontal and vertical directions as the main diagonal elements of the two-dimensional covariance matrix, and using the covariance calculated by combining the correlation coefficient values ​​as the off-diagonal elements of the two-dimensional covariance matrix. The two-dimensional covariance matrix is ​​used to indicate the spatial dispersion of the target's tendency towards the location mean prediction data distribution.

[0053] The mixed-weight prediction data contains multiple normalized weight values, which have a one-to-one mapping relationship with the two-dimensional Gaussian probability density function, and indicate the prior probability of the target selecting the trajectory branch corresponding to a specific two-dimensional Gaussian probability density function.

[0054] The multimodal probability prediction module performs a weighted summation calculation on multiple two-dimensional Gaussian probability density functions and their corresponding normalized weight values. The resulting Gaussian mixture model is then used as the initial multimodal trajectory probability distribution. The mathematical expression of this Gaussian mixture model is as follows: in, Characterizing the Gaussian mixture model in coordinate variables The numerical value of the joint probability density distribution at; A column vector representing position coordinate variables in two-dimensional space; Characterizes the number of preset Gaussian components; The index representing the Gaussian components; Characterizing the first The normalized weight values ​​corresponding to each Gaussian component; Pi is a constant representing the mathematical constant of a circle. Characterizing the first A column vector of predicted data based on the position mean of each Gaussian component; Characterizing the first A two-dimensional covariance matrix with Gaussian components; The determinant value represents the two-dimensional covariance matrix; The inverse matrix representing the two-dimensional covariance matrix; Characterizes an exponential function with the natural constant as its base; The transpose operation of a vector or matrix is ​​used to identify the Gaussian mixture model as the initial multimodal trajectory probability distribution, where the initial multimodal trajectory probability distribution represents the set of probability distributions of the target's coordinate position in the future time step.

[0055] Furthermore, the collaborative pruning and enhancement module obtains the static spatial topology mask matrix and the initial multimodal trajectory probability distribution, performs spatial alignment calculations of the static spatial topology mask matrix and the initial multimodal trajectory probability distribution, and then performs the tensor inner product operation after spatial alignment. The specific workflow is as follows: The collaborative pruning and enhancement module extracts multiple two-dimensional Gaussian probability density functions contained in the initial multimodal trajectory probability distribution and obtains the matrix dimension parameters of the static spatial topology mask matrix, where the matrix dimension parameters include the number of grid columns in the horizontal direction and the number of grid rows in the vertical direction.

[0056] The collaborative pruning and enhancement module establishes a discrete coordinate sampling grid in a two-dimensional plane based on the matrix dimension parameter and the spatial quantization step size parameter. It then uses the position mean prediction data contained in the two-dimensional Gaussian probability density function and the two-dimensional covariance matrix to perform probability density calculation on the coordinates of the grid center point of the discrete coordinate sampling grid.

[0057] The collaborative pruning and enhancement module arranges the numerical results of probability density calculation in two dimensions according to the matrix dimension parameters, generating multiple initial probability density matrices. This results in a one-to-one mapping relationship between the initial probability density matrices and the two-dimensional Gaussian probability density function. Furthermore, the initial probability density matrices and the static spatial topology mask matrix have the same matrix dimension and spatial coordinate reference, thus completing the spatial alignment calculation between the static spatial topology mask matrix and the initial multimodal trajectory probability distribution.

[0058] The collaborative pruning and enhancement module traverses all initial probability density matrices and performs element-wise multiplication on the matrix elements within each initial probability density matrix and the matrix elements in the static spatial topology mask matrix that are at the same row and column indices. Based on the results of these element-wise multiplications, multiple aligned and fused tensor matrices are generated. The corresponding tensor inner product expression is as follows: in, The row index number representing the two-dimensional discrete grid; The column index number representing the two-dimensional discrete grid; The index of the Gaussian component and its corresponding matrix; Characterizing the first The row index of the aligned and fused tensor matrix With column index number The value of the matrix element at that location; Characterizing the first The initial probability density matrices are indexed in the same row. With column index number The numerical value of the probability density at a given point; The index numbers of the same row in the static spatial topological mask matrix With column index number The spatially reachable values ​​of the state matrix elements; Ordinary multiplication operations that represent scalar values.

[0059] The number of matrices in the aligned and fused tensor matrix is ​​equal to the number of matrices in the initial probability density matrix, and the matrix element values ​​inside the aligned and fused tensor matrix indicate the product of the probability density value at a specific spatial location and the spatial reachability state, thus completing the tensor inner product operation.

[0060] Furthermore, the collaborative pruning and enhancement module utilizes the matrix element values ​​of the static spatial topology mask matrix to perform asymmetric physical pruning on the probability density of each Gaussian component in the initial multimodal trajectory probability distribution. The specific workflow is as follows: The collaborative pruning and enhancement module extracts multiple aligned and fused tensor matrices generated by the aforementioned tensor inner product operation, and performs a global summation calculation on all the matrix element values ​​of each aligned and fused tensor matrix to generate the corresponding scalar value.

[0061] The collaborative pruning and enhancement module identifies the set of multiple scalar values ​​generated as the spatial topology validity weight. The spatial topology validity weight has a one-to-one mapping relationship with multiple aligned fusion tensor matrices, and the number of elements of the spatial topology validity weight is equal to the number of the aforementioned preset Gaussian components. The magnitude of the spatial topology validity weight indicates the integral area of ​​the corresponding Gaussian component within the reachable state region in the monitored physical space.

[0062] The collaborative pruning and enhancement module obtains the mixed weight prediction data contained in the aforementioned initial multimodal trajectory probability distribution, and extracts multiple normalized weight values ​​contained in the mixed weight prediction data; subsequently, the collaborative pruning and enhancement module performs scalar multiplication calculation on each normalized weight value and the corresponding spatial topology validity weight, and outputs multiple pruned weight values ​​based on the calculation result of the scalar multiplication calculation.

[0063] Multiple pruned weight values ​​are used to replace the original normalized weight values ​​in the mixed weight prediction data. Through the above calculations, the collaborative pruning and enhancement module uses spatial topology effectiveness weights to numerically suppress the normalized weight values ​​pointing to physically unreachable regions, attenuating the weight ratio of erroneous Gaussian components, and completing the asymmetric physical pruning of each Gaussian component in the initial multimodal trajectory probability distribution.

[0064] Furthermore, the collaborative pruning and enhancement module performs probability weight normalization calculations based on the numerical results of tensor inner product operations, and outputs the enhanced multimodal trajectory probability distribution. The specific workflow is as follows: The collaborative pruning and enhancement module obtains multiple pruned weight values ​​generated by the aforementioned calculation, and performs numerical summation calculation on all pruned weight values ​​to obtain the total weight value representing the sum of global probability weights. Then, the collaborative pruning and enhancement module divides each pruned weight value by the total weight value and outputs multiple normalized updated weight values. The sum of the multiple normalized updated weight values ​​is equal to one, and the multiple normalized updated weight values ​​are used to redistribute the prior probabilities of each Gaussian component under spatial topological constraints.

[0065] The collaborative pruning and enhancement module extracts multiple two-dimensional Gaussian probability density functions contained in the aforementioned initial multimodal trajectory probability distribution, keeps the position mean prediction data and two-dimensional covariance matrix within the multiple two-dimensional Gaussian probability density functions unchanged, and performs weighted combination calculation on multiple normalized updated weight values ​​and the two-dimensional Gaussian probability density functions that present a mapping relationship to construct an enhanced Gaussian mixture model.

[0066] The collaborative pruning and enhancement module identifies the enhanced Gaussian mixture model as the enhanced multimodal trajectory probability distribution, which integrates the target motion dynamics characteristics and the physical boundary constraints of the monitoring physical space. At the same time, the collaborative pruning and enhancement module transmits the enhanced multimodal trajectory probability distribution to the irregular tracking guidance module, providing a probability density numerical benchmark for the establishment of the subsequent irregular tracking search area.

[0067] Furthermore, the specific workflow of the irregular tracking guidance module in receiving the enhanced multimodal trajectory probability distribution and extracting the probability contour boundary parameters of the enhanced multimodal trajectory probability distribution is as follows: The irregular tracking guidance module parses the enhanced multimodal trajectory probability distribution, extracts multiple normalized update weight values ​​and a two-dimensional Gaussian probability density function that presents a mapping relationship within the enhanced multimodal trajectory probability distribution, and obtains the system's preset confidence threshold parameter. It then performs a numerical comparison calculation between all the normalized update weight values ​​and the confidence threshold parameter.

[0068] When the irregular shape tracking guidance module determines that the normalized update weight value is greater than or equal to the confidence threshold parameter, it retains the corresponding normalized update weight value and the two-dimensional Gaussian probability density function that shows the mapping relationship. At the same time, when the irregular shape tracking guidance module determines that the normalized update weight value is less than the confidence threshold parameter, it removes the corresponding normalized update weight value and the two-dimensional Gaussian probability density function that shows the mapping relationship, thus completing the removal and screening of low-probability intention branches. In addition, to prevent the screening results from being empty, when all normalized update weight values ​​are less than the confidence threshold parameter, the irregular shape tracking guidance module executes the fallback retention logic, forcibly retaining the normalized update weight value with the largest value and its mapped two-dimensional Gaussian probability density function.

[0069] After screening, the irregular tracking guidance module constructs an effective prediction branch set from the retained two-dimensional Gaussian probability density functions. The effective prediction branch set contains at least one two-dimensional Gaussian probability density function and is used to indicate the location region where the target has the highest probability of appearing in the video frame coordinate system.

[0070] The irregular tracking guidance module extracts the position mean prediction data and two-dimensional covariance matrix contained in the two-dimensional Gaussian probability density function inside the effective prediction branch set, providing a mathematical basis for subsequent extraction of probability contour line boundary parameters.

[0071] Furthermore, the specific workflow of the irregular tracking guidance module in extracting the probability contour boundary parameters of the enhanced multimodal trajectory probability distribution and establishing the irregular tracking search area in the video frame coordinate system based on the probability contour boundary parameters is as follows: The irregular tracking guidance module traverses the two-dimensional Gaussian probability density function within the effective prediction branch set and obtains the system's preset confidence ellipse scale parameter. Then, the irregular tracking guidance module uses the position mean prediction data and two-dimensional covariance matrix contained in the two-dimensional Gaussian probability density function, combined with the confidence ellipse scale parameter, to perform contour line equation solving calculation based on Mahalanobis distance.

[0072] In the calculation of contour equations based on Mahalanobis distance, the irregular tracking guidance module sets the Mahalanobis distance between the pixel coordinates of the video image and the predicted mean position data to be equal to the confidence ellipse scale parameter, and uses the inverse matrix of the two-dimensional covariance matrix to construct a quadratic equation.

[0073] The irregular tracking guidance module calculates and outputs the probability contour boundary parameters corresponding to each two-dimensional Gaussian probability density function by solving the contour equation based on Mahalanobis distance. The probability contour boundary parameters include the geometric analytical equation of the closed elliptic curve in the two-dimensional plane. This geometric analytical equation defines the probability density decay boundary of the two-dimensional Gaussian probability density function at a specified confidence level.

[0074] The irregular tracking guidance module projects and maps the geometric analytical equations contained in all the probability contour boundary parameters onto the video frame coordinate system, generates multiple two-dimensional probability elliptical regions in the video frame coordinate system, and detects the spatial overlap between multiple two-dimensional probability elliptical regions in the video frame coordinate system.

[0075] The irregular tracking guidance module performs a geometric union calculation on two-dimensional probabilistic elliptical regions with overlapping spatial positions to remove the overlapping internal boundary lines between the two-dimensional probabilistic elliptical regions; at the same time, the irregular tracking guidance module extracts the external contour boundary retained after the geometric union calculation and fits the external contour boundary to generate a continuous envelope curve.

[0076] The irregular tracking guidance module unifies the continuous envelope curve and the original edge curve of the two-dimensional probabilistic elliptical region where there is no spatial overlap as the global search boundary, and identifies the set of all video frame pixel coordinates contained within the global search boundary as the irregular tracking search region. The irregular tracking search region presents an irregular topological structure corresponding to the distribution pattern of the target's multimodal maneuvering intent.

[0077] Furthermore, the specific workflow of the irregular shape tracking guidance module for guiding the visual tracking algorithm network to perform occlusion recapture and stable tracking based on the irregular shape tracking search region is as follows: The irregular tracking guidance module acquires the video image frame data of the latest time step, and calculates the minimum bounding rectangle boundary of the pixel coordinate set contained in the aforementioned irregular tracking search area, and extracts the local search image patch corresponding to the spatial position within the video image frame data based on the minimum bounding rectangle boundary; then, the irregular tracking guidance module uses the convolutional feature extraction layer contained in the internally configured visual tracking algorithm network to process the local search image patch and extract the current appearance feature vector of the local search image patch.

[0078] Before the target is physically occluded, the irregular tracking guidance module uses the current appearance feature vector under normal tracking conditions to update and save feature data in real time, so as to construct and store the historical appearance feature template before the target is physically occluded. Subsequently, the irregular tracking guidance module inputs the current appearance feature vector and the historical appearance feature template into the similarity measurement layer configured inside the visual tracking algorithm network, and uses the similarity measurement layer to perform normalized cross-correlation calculation on the current appearance feature vector and the historical appearance feature template, thereby outputting the spatial feature response matrix of each pixel position within the local search image patch.

[0079] The irregular shape tracking guidance module traverses the spatial feature response matrix and extracts the peak response values ​​within the matrix. Simultaneously, the module obtains the recapture matching threshold parameter configured within the system and performs a numerical comparison calculation between the peak response values ​​and the recapture matching threshold parameter. When the irregular tracking guidance module determines that the response peak value is greater than or equal to the recapture matching threshold parameter, it extracts the recapture coordinate data corresponding to the response peak value in the spatial feature response matrix and confirms the recapture coordinate data as the real-time tracking position of the target in the video image frame data; when the irregular tracking guidance module determines that the response peak value is less than the recapture matching threshold parameter, it determines that the target has not appeared in the local search image block and waits to process the video image frame data of the next time step.

[0080] The irregular tracking guidance module transmits the confirmed real-time tracking position to the system output and obtains the system's preset template update learning rate parameter. Combining the template update learning rate parameter, it performs a weighted update calculation on the historical appearance feature template using the current appearance feature vector to complete the occlusion re-acquisition and long-term stable tracking of the ground target.

[0081] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for predicting the long-term motion of ground targets with spatiotemporal evolution enhancement, characterized in that, Includes the following steps: Step S100: Receive video data stream, extract the state features of ground targets in the video data stream at continuous time steps to construct an observation state sequence; obtain scene images corresponding to the video data stream, and perform discrete grid division on the monitoring physical space where the scene images are located to obtain multiple two-dimensional discrete grids; Step S200: The observed state sequence is input into the configured sequence coding network for processing to generate hidden layer temporal feature vectors; Determine the boundaries of physical obstacles in the scene image, assign corresponding matrix element values ​​to each of the two-dimensional discrete grids according to the boundaries of physical obstacles to characterize the spatial reachability, and establish a static spatial topology mask matrix; Step S300: Input the hidden layer temporal feature vector into the configured hybrid density network for decoding calculation, and output an initial multimodal trajectory probability distribution. The initial multimodal trajectory probability distribution contains multiple Gaussian components, and each Gaussian component has a corresponding probability density. Step S400: Perform spatial alignment calculation and tensor inner product operation between the static spatial topology mask matrix and the initial multimodal trajectory probability distribution; use the matrix element values ​​of the static spatial topology mask matrix to perform asymmetric physical pruning on the probability density of each Gaussian component, and perform probability weight normalization calculation based on the calculation result of the tensor inner product operation to output the enhanced multimodal trajectory probability distribution. Step S500: Based on the enhanced multimodal trajectory probability distribution, determine the probability contour line boundary parameters, set the video frame coordinate system, and establish an irregular tracking search area in the video frame coordinate system; acquire the video image frame of the video data stream at the latest time step, acquire the pre-extracted historical appearance feature template of the ground target, limit it to the irregular tracking search area, and use the configured visual tracking algorithm network to perform visual image feature similarity comparison calculation with the historical appearance feature template and the video image frame at the latest time step, and output the real-time tracking position of the ground target.

2. The method for predicting the long-term motion of ground targets with spatiotemporal evolution enhancement according to claim 1, characterized in that: The step of extracting the state features of ground targets in the video data stream at consecutive time steps to construct an observation state sequence specifically includes: Multiple historical video image frames are extracted from the video data stream. A preset target detection algorithm model is called to extract the two-dimensional center pixel of the ground target in each historical video image frame, and the position coordinate data of each historical video image frame corresponding to the time step are mapped to generate the position coordinate data of each historical video image frame. Extract adjacent historical video image frames from the plurality of historical video image frames, obtain the time interval between the adjacent historical video image frames, perform time derivative calculation in combination with the time interval, and obtain the velocity vector data of the corresponding time step; The position coordinate data belonging to the same time step are concatenated with the velocity vector data to generate a single-frame state feature vector corresponding to each time step. Multiple single-frame state feature vectors located within a preset historical observation time window are extracted, combined, and then preprocessed by extreme value scaling to construct the observation state sequence.

3. The method for predicting the long-term motion of ground targets with spatiotemporal evolution enhancement according to claim 2, characterized in that: The sequence coding network employs a long short-term memory network and contains multiple time-step computation nodes, each of which includes a gating structure and a cell state unit. The process of inputting the observed state sequence into the configured sequence coding network for processing to generate hidden layer temporal feature vectors specifically includes: Multiple single-frame state feature vectors arranged in time step order are extracted from the observed state sequence, and each single-frame state feature vector is sequentially input into each time step calculation node of the sequence coding network. When processing each of the time step computing nodes, the hidden state data transmitted by the previous time step computing node is received, the single frame state feature vector of the current time step is fused with the hidden state data, the cell state unit is updated through the gating structure and the feature output value is output. The feature output value generated at the last time step calculation node in the time dimension is extracted and mapped to the hidden layer temporal feature vector.

4. The method for predicting the long-term motion of ground targets with spatiotemporal evolution enhancement according to claim 1, characterized in that: The process of determining the boundaries of physical obstacles in the scene image, assigning corresponding matrix element values ​​to each of the two-dimensional discrete grids based on the boundaries of the physical obstacles to characterize the spatial reachability, and establishing a static spatial topology mask matrix specifically includes: The scene image is processed using a pre-set semantic segmentation network to obtain an environment category label. Based on the environment category label, a set of obstacle pixel coordinates representing impassable areas is selected. The obstacle pixel coordinate set is then subjected to edge fitting to determine the boundary of the physical obstacle. The boundaries of the physical obstacles are mapped to the interior of each of the two-dimensional discrete grids, and the coverage overlap ratio between the boundaries of the physical obstacles and each of the two-dimensional discrete grids is calculated. The matrix element values ​​corresponding to the two-dimensional discrete grids whose overlapping area ratio is greater than a preset area threshold are set to zero to represent the unreachable state, and the matrix element values ​​corresponding to the remaining two-dimensional discrete grids are set to one to represent the reachable state, thus constructing the static spatial topology mask matrix.

5. The method for predicting the long-term motion of ground targets with spatiotemporal evolution enhancement according to claim 1, characterized in that: The hybrid density network includes a shared feature extraction layer, a weighted regression branch, a two-dimensional mean regression branch, and a covariance regression branch; the process of inputting the hidden layer temporal feature vector into the configured hybrid density network for decoding and calculation, and outputting an initial multimodal trajectory probability distribution, specifically includes: The hidden layer temporal feature vectors are mapped by the shared feature extraction layer to output a high-dimensional joint feature matrix; The high-dimensional joint feature matrix is ​​calculated in parallel by the weight regression branch, the two-dimensional mean regression branch, and the covariance regression branch, and outputs mixed weight prediction data, location mean prediction data, and covariance prediction data respectively. The mixed weight prediction data contains multiple normalized weight values, and the location mean prediction data and the covariance prediction data respectively contain mean sub-data and covariance sub-data corresponding to each Gaussian component. A Gaussian mixture model containing multiple Gaussian components is constructed using the mixed weight prediction data, the location mean prediction data, and the covariance prediction data, and the Gaussian mixture model is confirmed as the initial multimodal trajectory probability distribution.

6. The method for predicting the long-term motion of ground targets with spatiotemporal evolution enhancement according to claim 5, characterized in that: The process of performing spatial alignment calculation and tensor inner product operation between the static spatial topology mask matrix and the initial multimodal trajectory probability distribution specifically includes: Determine the grid division dimensions corresponding to the plurality of two-dimensional discrete grids, and establish a discrete coordinate sampling grid based on the grid division dimensions; The coordinates of each grid center point contained in the discrete coordinate sampling grid are determined. The probability density values ​​are calculated by performing probability density calculation on the coordinates of each grid center point using the initial multimodal trajectory probability distribution. Based on the probability density values, multiple initial probability density matrices corresponding one-to-one with each Gaussian component are generated. Traverse all the initial probability density matrices, and perform element-wise multiplication of the probability density values ​​in each initial probability density matrix with the matrix element values ​​at the same coordinate positions in the static spatial topology mask matrix to generate multiple aligned and fused tensor matrices containing tensor element values, which are used as the calculation result of the tensor inner product operation.

7. The method for predicting the long-term motion of ground targets with spatiotemporal evolution enhancement according to claim 6, characterized in that: The asymmetric physical pruning of the probability density of each Gaussian component using the matrix element values ​​of the static spatial topological mask matrix specifically includes: Extract the tensor element values ​​from each of the alignment and fusion tensor matrices, perform a global summation calculation on the tensor element values, and generate spatial topology validity weights corresponding to each of the Gaussian components; Extract the multiple normalized weight values ​​contained in the mixed weight prediction data; Perform scalar multiplication on each of the normalized weight values ​​and the corresponding spatial topology validity weights to output multiple pruned weight values ​​to replace the original normalized weight values.

8. The method for predicting the long-term motion of ground targets with spatiotemporal evolution enhancement according to claim 7, characterized in that: The probability weight normalization calculation based on the calculation result of the tensor inner product operation, and the output of the enhanced multimodal trajectory probability distribution, specifically includes: The total weight value is obtained by summing all the pruned weight values. Divide each of the pruned weight values ​​by the total weight value to output multiple normalized updated weight values, and the sum of the multiple normalized updated weight values ​​is equal to one. An enhanced Gaussian mixture model is constructed by re-weighting and combining multiple normalized update weight values ​​with the position mean prediction data and the covariance prediction data, and the enhanced Gaussian mixture model is confirmed as the enhanced multimodal trajectory probability distribution.

9. The method for predicting the long-term motion of ground targets with spatiotemporal evolution enhancement according to claim 8, characterized in that: The process of determining probability contour line boundary parameters based on the enhanced multimodal trajectory probability distribution, setting a video frame coordinate system, and establishing an irregular tracking search region within the video frame coordinate system specifically includes: Remove the normalized update weight values ​​and their corresponding Gaussian components that are less than the preset confidence threshold parameter, and construct the remaining Gaussian components into an effective prediction branch set. Obtain the mean sub-data and covariance sub-data corresponding to the retained Gaussian components, and perform contour line equation solving calculation based on Mahalanobis distance in combination with the preset confidence ellipse scale parameters to determine the probability contour line boundary parameters corresponding to each retained Gaussian component. Project all the probability contour line boundary parameters onto the video frame coordinate system to generate multiple two-dimensional probability elliptical regions. Perform geometric union calculation on the two-dimensional probability elliptical regions that have spatial overlapping states to extract the outer contour boundary. Generate a global search boundary formed by the outer contour boundary, extract a set of candidate pixel coordinates located inside the global search boundary, and confirm the set of candidate pixel coordinates as the irregular shape tracking search area; The process, confined within the irregular shape tracking search area, utilizes a configured visual tracking algorithm network to perform visual image feature similarity comparison calculations using the historical appearance feature templates and the video image frames from the latest time step, outputting the real-time tracking position of the ground target. Specifically, this includes: Calculate the minimum bounding rectangle boundary of the irregular tracking search region, and extract local search image blocks within the video image frame of the latest time step based on the minimum bounding rectangle boundary; The local search image patch is processed using the convolutional feature extraction layer configured inside the visual tracking algorithm network to extract the current appearance feature vector; The similarity measurement layer configured inside the visual tracking algorithm network is used to perform normalized cross-correlation calculation on the current appearance feature vector and the historical appearance feature template, and outputs a spatial feature response matrix; The response peak value is extracted from the spatial feature response matrix. When the response peak value is determined to be greater than or equal to a preset recapture matching threshold parameter, the recapture coordinate data corresponding to the response peak value is extracted, and the recapture coordinate data is confirmed as the real-time tracking position of the ground target.

10. A long-term temporal motion prediction system for ground targets with enhanced spatiotemporal evolution, characterized in that, For executing the spatiotemporal evolution-enhanced long-term motion prediction method for ground targets as described in any one of claims 1 to 9, the system comprises: The data acquisition and temporal coding module is used to receive video data streams, extract the state features of ground targets in the video data streams at continuous time steps to construct observation state sequences, and input the observation state sequences into a configured sequence coding network for processing to generate hidden layer temporal feature vectors. The environmental topology modeling module is used to acquire scene images corresponding to the video data stream, divide the monitoring physical space where the scene image is located into discrete grids to obtain multiple two-dimensional discrete grids; determine the boundaries of physical obstacles in the scene image, assign corresponding matrix element values ​​to each of the two-dimensional discrete grids according to the boundaries of physical obstacles to represent the spatial reachability state, and establish a static spatial topology mask matrix. The multimodal probability prediction module is used to input the hidden layer temporal feature vector into the configured hybrid density network for decoding and calculation, and output an initial multimodal trajectory probability distribution. The initial multimodal trajectory probability distribution contains multiple Gaussian components, and each Gaussian component has a corresponding probability density. The collaborative pruning and enhancement module is used to perform spatial alignment calculation and tensor inner product operation between the static spatial topology mask matrix and the initial multimodal trajectory probability distribution; to perform asymmetric physical pruning on the probability density of each Gaussian component using the matrix element values ​​of the static spatial topology mask matrix, and to perform probability weight normalization calculation based on the calculation result of the tensor inner product operation, and output the enhanced multimodal trajectory probability distribution. The irregular tracking guidance module is used to determine the probability contour line boundary parameters based on the enhanced multimodal trajectory probability distribution, set the video frame coordinate system, and establish an irregular tracking search area in the video frame coordinate system; acquire the video image frame of the video data stream at the latest time step, acquire the pre-extracted historical appearance feature template of the ground target, limit it to the irregular tracking search area, and use the configured visual tracking algorithm network to perform visual image feature similarity comparison calculation with the historical appearance feature template and the video image frame at the latest time step, and output the real-time tracking position of the ground target.

Images