Low-orbit space visual interference target tracking method and system based on trajectory prediction

Abstract

The present invention belongs to the field of target tracking technology. To solve the problem of reduced tracking accuracy and continuity, a low-orbit space visual interference target tracking method and system based on trajectory prediction are provided. The low-orbit space visual interference target tracking method based on trajectory prediction includes tracking the target, obtaining the target position, and extracting target features before interference, and online training of a trajectory prediction model and a graph neural network model; calculating the peak response within the selected target frame area, and determining whether the target is occluded by comparing the relative distance between the highest peak and the second highest peak; when the target is occluded, calling the trajectory prediction model and the graph neural network model and respectively predicting the long-term and short-term trajectories of the target; tracking the target based on the target prediction coordinates transmitted to the space tracking target model by the trajectory prediction model; and after interference, using the space tracking target model to detect the target's self-tracking. This method can improve tracking accuracy and continuity.

Description

Technical Field

[0001] The present invention belongs to the field of target tracking technology, and in particular relates to a low-orbit space visual interference target tracking method and system based on trajectory prediction. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Visual interference refers to the inability to fully extract a target's features or the temporary disappearance of the target from view due to the interference of a dynamic background or the influence of other objects. Currently, most tracking technologies rely on filtering algorithms within traditional tracking algorithms, such as Kalman filtering and its variants, for transient trajectory prediction. In the aerospace field, targets (such as satellites or spacecraft) can be severely obscured for extended periods while passing through star clusters, resulting in target loss and inability to track. Furthermore, the influence of star clusters can cause target features to be lost, making it impossible to track based on historically extracted target features, reducing tracking accuracy and continuity. Summary of the Invention

[0004] In order to solve the technical problem that during the target tracking process, the target is lost due to interference from dynamic background or other objects, thereby reducing the accuracy and continuity of tracking, the present invention provides a low-orbit space visual interference target tracking method and system based on trajectory prediction, which can improve the accuracy and continuity of tracking.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A first aspect of the present invention provides a low-orbit space visual interference target tracking method based on trajectory prediction.

[0007] In one or more embodiments, a method for tracking a low-orbit space visual interference target based on trajectory prediction is provided, comprising:

[0008] Before interference, the target is tracked and its position and features are obtained based on the target image frame and the pre-trained spatial tracking target model, and the trajectory prediction model and graph neural network model are trained online.

[0009] Calculate the peak response within the selected target frame area and determine whether the target is blocked by comparing the relative distance between the highest peak and the second highest peak to determine whether to call the trajectory prediction model and graph neural network model;

[0010] When the target is occluded, the graph neural network model is called to perform local correction on the target position, and the locally corrected target position is transmitted to the trajectory prediction model for target trajectory prediction. The target is then tracked based on the target prediction coordinates transmitted to the spatial tracking target model by the trajectory prediction model.

[0011] After the interference, the spatial tracking target model is used to detect the target and track it by itself. At the same time, the target tracking position transmitted by the spatial tracking target model to the trajectory prediction model and the graph neural network model is used to continuously update the trajectory prediction model.

[0012] As an implementation method, the process of calling the graph neural network model to perform local correction on the target position is as follows:

[0013] The graph neural network model is used to regard the local feature map of the target's historical trajectory as a graph node, construct a time series graph and correct the target position.

[0014] As an implementation method, the features of the neighboring nodes of the time series graph are used to smooth outliers; the position of the current abnormal node is adjusted to the weighted average of its neighboring nodes, while ensuring that neighbors closer to the current frame have a greater impact on the correction result.

[0015] As an implementation method, the target is tracked based on the target prediction coordinates transmitted to the spatial tracking target model by the trajectory prediction model, and the attention around the predicted target coordinates is enhanced based on the local attention mechanism.

[0016] As an implementation method, in the process of continuously updating the trajectory prediction model, a graph neural network model is used to compare and analyze the error between the target predicted trajectory output by the trajectory prediction model and the target actual trajectory output by the spatial tracking target model, and the weight of the trajectory prediction model is dynamically adjusted according to the error.

[0017] As an implementation method, the spatial tracking target model is a SiamRPN model.

[0018] As an implementation method, the trajectory prediction model adopts an LSTM model.

[0019] A second aspect of the present invention provides a low-orbit space visual interference target tracking system based on trajectory prediction.

[0020] In one or more embodiments, a low-orbit space visual interference target tracking system based on trajectory prediction includes:

[0021] An online training module, which is used to track the target and obtain the target position and extract target features based on the target image frame and the pre-trained spatial tracking target model before interference, and to train the trajectory prediction model and graph neural network model online;

[0022] The occlusion judgment module is used to calculate the peak response within the selected target frame area and determine whether the target is occluded by comparing the relative distance between the highest peak and the second highest peak, so as to determine whether to call the trajectory prediction model and the graph neural network model;

[0023] The occluded target tracking module is used to call the graph neural network model to perform local correction on the target position when the target is occluded, and transmit the locally corrected target position to the trajectory prediction model for target trajectory prediction. The target is then tracked based on the target prediction coordinates transmitted by the trajectory prediction model to the spatial tracking target model;

[0024] The unobstructed target tracking module is used to detect the target and track it by itself after interference using the spatial tracking target model. At the same time, it uses the target tracking position transmitted by the spatial tracking target model to the trajectory prediction model and the graph neural network model to continuously update the trajectory prediction model.

[0025] A third aspect of the present invention provides a computer-readable storage medium.

[0026] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the low-orbit space visual interference target tracking method based on trajectory prediction as described above.

[0027] A fourth aspect of the present invention provides an electronic device.

[0028] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the method for tracking a low-orbit space visual interference target based on trajectory prediction as described above are implemented.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] The present invention uses the peak value of the peak response within the selected target frame area to determine whether the target is occluded, and then determines whether to call the trajectory prediction model and the graph neural network model. When the target is occluded, the graph neural network model is called to perform local correction on the target position, and the locally corrected target position is transmitted to the trajectory prediction model. Moreover, after the interference, the spatial tracking target model is used to detect the target self-tracking, which improves the accuracy and continuity of target tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0032] Figure 11 is a flow chart of a method for tracking a low-orbit space visual interference target based on trajectory prediction according to an embodiment of the present invention;

[0033] Figure 2 is a schematic diagram of a target being blocked when passing through a star cluster according to an embodiment of the present invention;

[0034] Figure 3 This is a diagram of a low-orbit space visual interference target tracking process based on trajectory prediction according to an embodiment of the present invention;

[0035] Figure 4 1 is a peak comparison diagram of undisturbed and disturbed conditions according to an embodiment of the present invention;

[0036] Figure 5 This is a process for determining whether a target is interfered with according to an embodiment of the present invention;

[0037] Figure 6 This is the SiamRPN model architecture of an embodiment of the present invention;

[0038] Figure 7 1 is a schematic structural diagram of a low-orbit space visual interference target tracking system based on trajectory prediction according to an embodiment of the present invention;

[0039] Figure 8 This is a diagram showing the effect of visual interference target tracking based on trajectory prediction in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0041] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0042] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0043] Explanation of terms:

[0044] SiamRPN is a lightweight model based on the Siamese network that can achieve the real-time requirements of spatial target tracking while ensuring accuracy. SiamRPN can quickly generate multiple candidate regions and select the most likely target location through a scoring mechanism, so it can still demonstrate strong robustness in dealing with complex spatial backgrounds.

[0045] LSTM (Long Short-Term Memory) excels at processing and predicting time series data. Through its unique gating mechanism (input gate, forget gate, and output gate), it can effectively learn and memorize long-term dependencies, facilitating the use of historical motion states to predict future positions. GNN (Graph Neural Network) compares and analyzes LSTM predicted trajectories and SiamRPN actual trajectories, generates feedback information, and dynamically adjusts LSTM weights or introduces correction terms to improve LSTM prediction accuracy.

[0046] Figure 1 This is a flow chart of a method for tracking a low-orbit space visual interference target based on trajectory prediction in an embodiment of the present invention. Figure 1 and Figure 3 The low-orbit space visual interference target tracking method based on trajectory prediction in this embodiment may include steps S101 to S104:

[0047] S101, before interference, tracks the target based on the target image frame and the pre-trained spatial tracking target model, obtains the target position and extracts the target features, and trains the trajectory prediction model and graph neural network model online.

[0048] Figure 2 (a) is a state diagram of a target (such as a satellite or other spacecraft) when it is near a star cluster; Figure 2 In (b), the target (such as a satellite or other spacecraft) is severely obscured when passing through a star cluster, resulting in the target being lost and unable to be tracked; at the same time, the target is affected by the star cluster, resulting in the loss of target features, and it cannot be tracked based on the extracted historical target features.

[0049] In this embodiment, the spatial tracking target model is a SiamRPN model, and the trajectory prediction model is an LSTM model.

[0050] Before interference, SiamRPN is used to track the target, obtain the target position and extract the target features.

[0051] The structure of SiamRPN is as follows Figure 6 As shown in Figure 2, the process of tracking the target through SiamRPN is as follows:

[0052] Step (1.1): First, select the target in the first frame of the video and record it as the template image z; then the area in each frame that may contain the target location is recorded as the search image x.

[0053] Step (1.2): SiamRPN uses a Siamese network with two shared weight sub-networks (convolutional neural network CNN) to extract features (edge, brightness, shape) from the template image and the search image respectively, which are recorded as and .

[0054] Step (1.3): The Siamese network uses the convolutional layer to fuse the template image and the search image to obtain the correlation between the two.

[0055] ;

[0056] Indicates the similarity between the template and each pixel in the search area, that is, the degree of matching; The template image Deep features of channels; Search image No. The depth features of the channels.

[0057] Step (1.4): The SiamRPN Region Proposal Network (RPN) generates a series of candidate regions (anchors), performs object matching and confidence scoring on each region, and determines the most likely location of the target. Simultaneously, the regression layer predicts the precise location of the target and outputs the target's location and size parameters.

[0058] Step (1.5): Update the position of the target in the current frame using the precise position obtained in step (1.4) to track the target.

[0059] The process of obtaining the target position through SiamRPN is:

[0060] Step (2.1): Template image features and search image features extracted by Siamese network, where, ; ;

[0061] is the number of feature map channels, H and W are the height and width of the template image feature map respectively; and are the height and width of the search image feature map respectively; represents the set of real numbers.

[0062] Step (2.2): Calculation and Similarity :

[0063] ;

[0064] 、 Represent the features of the i-th channel in the template image and the j-th channel in the search image respectively; Indicates the similarity between the template and each pixel in the search area, that is, the degree of matching.

[0065] Step (2.3): The RPN network outputs a score map and bounding box regression prediction;

[0066] The score map represents the confidence of each candidate box, represented by P(x,y), where x and y are the horizontal and vertical coordinates of the candidate area respectively.

[0067] Bounding box regression: Each candidate box has a set of regression parameters, 、 represent horizontal and vertical displacements respectively; 、 Represent the width change and height change of the target respectively.

[0068] Step (2.4): Target location determination;

[0069] Confidence selection: Select the highest confidence candidate box using the maximum score function:

[0070] ;

[0071] in, represents the maximization score function;

[0072] Boundary regression adjustment: Adjust the candidate box coordinates according to the regression offset:

[0073] ;

[0074] ;

[0075] and Respectively represent the initial position and initial size of the selected candidate box; and Represent the adjusted position and adjusted size of the selected candidate box respectively.

[0076] Target features extracted by SiamRPN include but are not limited to:

[0077] Magnitude characteristics: Magnitude usually refers to the intensity of radiation emitted by a celestial body, which is represented on the image as irregular dots of varying brightness.

[0078] Edge features: describe the boundary information in the image, which is the dividing line between the target and the background. Edge features can help capture the outline shape of the target.

[0079] Shape features: describe the geometric shape of the target, used to describe the boundary, size, outline, etc. of the target.

[0080] Temporal features: describe the changes of the target in the time series and can capture the dynamic change information of the target between multiple frames.

[0081] In this embodiment, the core of GNN is to use graph structure modeling. Represented as a node in a graph (denoted as ), each node can contain a variety of information: location attributes , time attribute t, speed attribute , acceleration properties etc. Represent node features as vectors ,but:

[0082] ;

[0083] Position attributes: the target's position coordinates in the current frame; time attributes: the timestamp of the frame. This information is used to construct the graph structure so that the position information in the time series can reflect the target's motion trajectory.

[0084] The edges of the graph structure reflect the smoothness of the position change between two frames. The nodes of each pair of adjacent frames are connected by edges to form a time series graph.

[0085] Edge weight It is usually expressed as the Euclidean distance between targets between frames:

[0086] ;

[0087] It is a tuning parameter used to control the weight decay rate. The larger the weight, the higher the similarity.

[0088] The process of constructing a time series graph is:

[0089] Step (3.1): Add graph nodes: Each time a frame of video is processed, the target position is recorded and a new node is added to the graph.

[0090] Step (3.2): Update graph edges: connect the newly added nodes to the nodes in the previous frame through new edges.

[0091] Global update rules:

[0092] Step (3.3): The updated state of the graph is represented by G = (V, E), where:

[0093] V={ , }: A collection of nodes representing the position information of each frame in the time series.

[0094] E={ }: Edge set, representing the association information between adjacent frames in the time series.

[0095] The SiamRPN transmits the acquired location coordinates to the LSTM and GNN. The LSTM uses a dataset of 30 frames. The coordinate data of the first 27 frames is used as the training set for online trajectory prediction model training, and the last three frames are used as the validation set to verify the model's effectiveness. The GNN acquires the coordinate data, treats the location coordinates of each frame as a graph node, constructs a time series graph, and corrects local anomalies.

[0096] Among them, the local structure of the time series graph is used to determine whether a certain frame position is abnormal. The main methods of anomaly detection include:

[0097] (1) Speed ​​change detection: If the speed between two frames changes If the value exceeds the preset threshold, the node is considered abnormal.

[0098] (2) Graph signal smoothing: Use graph convolution to smooth the coordinate information of the node and determine whether the local position change is abnormal. The graph signal smoothing formula is:

[0099] ;

[0100] is the position after smoothing;

[0101] For nodes The set of neighbors of

[0102] is the edge weight between nodes;

[0103] are the coordinates of the neighbor nodes.

[0104] The process of correcting local anomalies using time series graphs is as follows:

[0105] Use the features of neighboring nodes to smooth outliers:

[0106] ;

[0107] in, Represents the corrected node location.

[0108] The position of the current abnormal node is adjusted to the weighted average of its neighboring nodes, while ensuring that neighbors closer to the current frame have a greater impact on the correction result.

[0109] This embodiment uses the strong tracking capability of SiamRPN and the strong prediction capability of LSTM to achieve tracking during the period when the target is disturbed.

[0110] S102, calculate the peak response within the selected target frame area, and determine whether the target is blocked by comparing the relative distance between the highest peak and the second highest peak to determine whether to call the trajectory prediction model and the graph neural network model.

[0111] When the target is not disturbed, the background image of the space is black, so the peak value of the target in the RPN area is as follows: Figure 4 As shown in (a), there is a large gap between the highest peak and the second highest peak; when the target is disturbed by the cluster light, the peak in the RPN is as follows Figure 4 As shown in (b), the difference between the highest peak and the second highest peak is small, and the target is judged to be disturbed. The judgment process is as follows Figure 5 shown.

[0112] S103, when the target is occluded, the graph neural network model is called to perform local correction on the target position, and the locally corrected target position is transmitted to the trajectory prediction model for target trajectory prediction, and then the target is tracked according to the target prediction coordinates transmitted to the spatial tracking target model by the trajectory prediction model.

[0113] The process of calling the graph neural network model to perform local correction on the target position is as follows:

[0114] The graph neural network model is used to regard the local feature map of the target's historical trajectory as a graph node, construct a time series graph and correct the target position.

[0115] LSTM models the target's historical trajectory through temporal sequence modeling, using memory cells to capture long-term dependencies and predict future target locations. Its core is the gating mechanism (input gate, forget gate, output gate) and the memory cell update logic.

[0116] The input of LSTM is the historical trajectory of the target;

[0117] The historical trajectory of the target is recorded as: represents the position coordinates of the target in two-dimensional space, and k is the number of time steps of the input trajectory.

[0118] The output of LSTM is the predicted trajectory of the target;

[0119] The predicted trajectory of the target in the future is:

[0120] .

[0121] The forget gate of LSTM determines which past information needs to be discarded. For example, if the target's motion pattern suddenly changes, the forget gate will choose to discard outdated trajectory information. Its expression is:

[0122] ;

[0123] Represents the output of the forget gate, with a value of [0,1], indicating the forgetting ratio;

[0124] Represents the weight matrix of the forget gate;

[0125] represents the hidden state of the previous time step;

[0126] Represents the input of the current time step;

[0127] Represents the bias term of the forget gate;

[0128] Represents the Sigmoid activation function, which maps values ​​to [0,1].

[0129] The input gate of the LSTM determines what new information needs to be added to the memory cell to update the target's motion state:

[0130] ;

[0131] ;

[0132] is the output of the input gate, with a value range of [0,1], indicating the proportion of new information;

[0133] is the memory unit state;

[0134] are the weight matrices of the input gate and memory unit respectively;

[0135] are the bias terms of the input gate and memory unit respectively;

[0136] is the hyperbolic tangent function, which maps values ​​to [−1,1].

[0137] The output gate of the LSTM determines which information is output as the hidden state. The hidden state stores the trajectory feature information of the current time step and is an important basis for predicting future trajectories:

[0138] ;

[0139] The final hidden state is:

[0140] ;

[0141] represents the hidden state of the current time step; Indicates intermediate parameters; represents the bias parameter; is element-wise multiplication; represents the intermediate weight matrix.

[0142] The LSTM memory unit accumulates long-term information of historical trajectories to capture the global laws of target motion. The memory unit combines the forget gate and the input gate to update the memory state:

[0143] .

[0144] in, is the candidate memory unit state, indicating the new information that can be added;

[0145] The LSTM target trajectory prediction process is as follows:

[0146] Step a: data preprocessing;

[0147] The historical trajectory coordinates are standardized using the StandardScaler function to reduce the impact of the numerical range on model training. The trajectory sequence is segmented into a fixed-length sliding window, and each trajectory is used as an input sample. The StandardScaler function is:

[0148] ;

[0149] is the original trajectory coordinate;

[0150] is the mean of the original trajectory coordinates;

[0151] is the standard deviation of the original trajectory coordinates;

[0152] It is the data after standardization.

[0153] Step b: time step input;

[0154] At each time step, a historical coordinate is input into the LSTM unit, combined with the hidden state and memory units , update the hidden state and memory units .

[0155] Step c: multi-layer LSTM processing;

[0156] If a multi-layer LSTM is used, the hidden state It will be passed to the next layer of LSTM units for deeper time series feature extraction.

[0157] Step d: predict trajectory output;

[0158] At the last time step of the last LSTM layer, the hidden state Mapped to the predicted trajectory of the next m frames through the fully connected layer (FC):

[0159] ;

[0160] in, represents the predicted trajectory; Indicates the offset.

[0161] The features of the adjacent nodes of the time series graph are used to smooth outliers; the position of the current abnormal node is adjusted to the weighted average of its neighboring nodes, while ensuring that neighbors closer to the current frame have a greater impact on the correction result.

[0162] GNN predicts the future location of a target by representing its time series as a graph and leveraging the information of neighboring nodes.

[0163] The input of GNN includes:

[0164] Node features: graph structure data node The initial feature is the coordinate of the target, which is recorded as: .

[0165] Edge features: edge weights .

[0166] Output of GNN:

[0167] Predicted trajectory: .

[0168] GNN updates node features through information transmission and uses the information of adjacent nodes to predict trajectories.

[0169] In each layer, the node features are updated as follows:

[0170] ;

[0171] Representation node In the l Feature representation of the layer;

[0172] is a node The neighbor set of (usually the nodes of the previous and next adjacent frames);

[0173] is the edge weight, indicating the node and degree of association;

[0174] and It is l The layer's weight matrix and bias vector;

[0175] is the ReLU activation function.

[0176] Adjacent feature aggregation: features of neighboring nodes By weight A weighted sum is performed to represent the information received by the node from its neighbors.

[0177] Node feature update: The aggregated features are updated through linear transformation and activation function to update the features of the node itself.

[0178] (1) Use updated node features to predict future locations

[0179] go through l After the layer message is passed, the node The final feature representation is This feature contains the local trajectory information of the target, and the predicted position can be obtained by Get:

[0180] ;

[0181] MLP is a multi-layer perceptron that maps high-dimensional features to the coordinates of the next time step. .

[0182] (2) Recursively predict multi-step trajectories;

[0183] Predicting the trajectory of the next m frames can be achieved recursively:

[0184] use The predicted value of is taken as input and calculated .

[0185] Repeat the above steps until the positions of all m frames are predicted.

[0186] Specifically, the target is tracked according to the target prediction coordinates transmitted to the spatial tracking target model by the trajectory prediction model, and the attention around the predicted target coordinates is enhanced based on the local attention mechanism.

[0187] In SiamRPN, the attention mechanism is integrated into the template image and search images During the feature extraction process, the network weights the area near the predicted location are dynamically adjusted. First, at the predicted location An attention window is constructed near the target area; then, a weight is assigned to each position in the attention window, and the size of the weight is dynamically calculated by the attention module; finally, the features are fused in a weighted manner to highlight the target area and important background features.

[0188] The process of constructing the local attention area is as follows:

[0189] (1) Construct an attention window near the predicted position:

[0190] Based on the predicted position of the previous frame , select a local window (such as 5×5 or 7×7) on the search feature map. The center of the local window is the predicted position Indicates the area that needs to be paid attention to, with a size of k×k, which controls the range of the area that the attention module focuses on, and is usually set to 1~2 times the target size.

[0191] (2) Extract local features:

[0192] Use a cropping operation to remove the features of the attention window area from the search graph Extract it and get local features :

[0193] ;

[0194] The “:” in the above formula indicates slicing the matrix or array to select all rows.

[0195] The process of attention weight calculation is:

[0196] The local features generate weight distribution through the attention module:

[0197] Input fusion: template features and local search features Calculate similarity by dot product And fuse them. Output a similarity graph , and The height and width of the local window.

[0198] Indicates that the template feature and search feature are at position similarity.

[0199] (2) Normalization:

[0200] right Use the Softmax function to perform normalization and calculate the attention weight of each position : ;

[0201] is the position in the local window The attention weight indicates the importance of each position in the local area of ​​the search graph to the target prediction of the current frame. The closer to the center of the target, the higher the weight.

[0202] The calculation process of weighted fusion features is:

[0203] Attention weighting: Using the above-obtained attention weights to weight the local search features, we can obtain:

[0204] ;

[0205] Local search features integrated with attention weights.

[0206] Feature replacement: Replace the original search image features with The corresponding area is replaced by , which enables the model to focus more on the target area during target tracking and reduce the interference of irrelevant background, so that the algorithm can better capture target information and improve the accuracy and robustness of target positioning.

[0207] SiamRPN integrated attention mechanism:

[0208] Template and search feature extraction:

[0209] Extracting template images using Siamese networks and search images The depth features of .

[0210] Attention module introduction:

[0211] In the search graph According to the predicted location Extracting local features At the same time, the attention mechanism is used to generate weights , and weightedly fuse local features to obtain weighted local features .

[0212] Feature matching and target position prediction:

[0213] Search graph And the weighted local features Input to the cross-correlation module to obtain the final feature map :

[0214] ;

[0215] in, represents the final feature map;

[0216] according to The response map determines the target location and size.

[0217] S104, after the interference, the spatial tracking target model is used to detect the target self-tracking, and the target tracking position transmitted by the spatial tracking target model to the trajectory prediction model and the graph neural network model is used to continuously update the trajectory prediction model.

[0218] It should be noted here that "after interference" refers to the situation where the target is unable to be identified and tracked due to strong interference such as sudden changes in illumination or star clusters.

[0219] Specifically, in the process of continuously updating the trajectory prediction model, the graph neural network model is used to compare and analyze the error between the target predicted trajectory output by the trajectory prediction model and the target actual trajectory output by the spatial tracking target model, and the weight of the trajectory prediction model is dynamically adjusted according to the error.

[0220] By comparing LSTM prediction trajectories and GNN predicted trajectory Error :

[0221] ;

[0222] when When it is greater than the set threshold, it is considered that there is an error in the LSTM prediction.

[0223] By dividing the error With a feedback factor Combined generation of feedback information :

[0224] ;

[0225] Feedback coefficient Controls the strength of feedback and determines the magnitude of LSTM weight adjustment, usually taking a value between [0, 1].

[0226] When the error When it is small, the feedback information Smaller, indicating that LSTM prediction is more accurate and feedback has less impact on the LSTM model; when the error When it is large, feedback information A larger value indicates that the LSTM prediction deviates significantly from the actual position, and the feedback has a greater impact on the LSTM.

[0227] Dynamically adjust the weights of LSTM:

[0228] In one or more embodiments, GNN uses feedback information Dynamically adjust the weights of the LSTM model , optimize LSTM's ability to predict target trajectories. During LSTM training, the feedback information generated by error analysis is used , dynamically adjust the weight of LSTM:

[0229] ;

[0230] in, is the weight of the LSTM model at time t+1; The weight of the LSTM model at time t determines the degree to which the historical trajectory influences the current prediction. If it is large, the LSTM will rely more on past trajectory data; conversely, the LSTM will reduce its reliance on past trajectories and enhance its ability to predict the current target location. The learning rate controls the step size of weight adjustment and determines the speed of LSTM weight update. A larger learning rate will make the weight update faster, while a smaller learning rate will make the adjustment more gradual. and learning rate All of them are obtained by using LSTM to train prediction models using historical data. It is the feedback information generated by LSTM and GNN prediction errors.

[0231] In some other optional embodiments, the local prediction of GNN is used to modify the result of LSTM, and the fusion weight is introduced β Dynamically adjust the LSTM trajectory:

[0232] ;

[0233] : LSTM predicts future positions based on global motion laws.

[0234] : GNN short-term prediction results based on local trajectory features.

[0235] : Control the fusion of the two, usually relying more on GNN when occlusion is severe.

[0236] Weight fusion parameters Determines the contribution ratio of each when fusing LSTM and GNN prediction results.

[0237] LSTM confidence : Confidence evaluation obtained through historical trajectory training.

[0238] ;

[0239] in, Represents the error rate of LSTM, which is a value less than 1.

[0240] GNN confidence : The average value of edge weights based on GNN output reflects the local smoothness of the trajectory; ;

[0241] The calculation formula is:

[0242] ;

[0243] The effect of visual interference target tracking based on trajectory prediction is shown in the figure Figure 8 As shown, Figure 8 The green track is the actual track of the target, the blue track is the track tracked by SiamRPN when the target is not occluded, and the red track is the predicted track of LSTM after GNN correction when the target is visually disturbed.

[0244] Figure 7 This is a schematic diagram of the structure of a low-orbit space visual interference target tracking system based on trajectory prediction in an embodiment of the present invention. Figure 1 The low-orbit space visual interference target tracking method based on trajectory prediction corresponds to Figure 7 As shown, the low-orbit space visual interference target tracking system based on trajectory prediction in this embodiment may include:

[0245] Online training module 701, which is used to track the target and obtain the target position and extract target features based on the target image frame and the pre-trained spatial tracking target model before interference, and to train the trajectory prediction model and the graph neural network model online;

[0246] The occlusion judgment module 702 is used to calculate the peak response within the selected target frame area and determine whether the target is occluded by comparing the relative distance between the highest peak and the second highest peak, so as to determine whether to call the trajectory prediction model and the graph neural network model;

[0247] The occluded target tracking module 703 is used to call the graph neural network model to perform local correction on the target position when the target is occluded, and transmit the locally corrected target position to the trajectory prediction model for target trajectory prediction, and then track the target according to the target prediction coordinates transmitted to the spatial tracking target model by the trajectory prediction model;

[0248] The unobstructed target tracking module 704 is used to detect the target self-tracking using the spatial tracking target model after interference, and at the same time use the target tracking position transmitted by the spatial tracking target model to the trajectory prediction model and the graph neural network model to continuously update the trajectory prediction model.

[0249] It should be noted here that, Figure 7 The various modules in the low-orbit space visual interference target tracking system based on trajectory prediction in Figure 1 The various steps in the low-orbit space visual interference target tracking method based on trajectory prediction in correspondence one to one, and the specific implementation process is the same, which will not be repeated here.

[0250] In one or more embodiments, an electronic device is provided that includes a central processing unit (CPU) capable of executing various appropriate actions and processes based on programs stored in a read-only memory (ROM) or programs loaded from a storage unit into a random access memory (RAM). The RAM also stores various programs and data required for system operation. The CPU, ROM, and RAM are interconnected via a bus. An input / output (I / O) interface is also connected to the bus.

[0251] The following components are connected to the I / O interface: an input section including a keyboard and mouse; an output section including cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; a storage section including a hard disk; and a communication section including network interface cards such as local area network (LAN) cards and modems. The communication section performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as needed. Removable media such as magnetic disks, optical disks, magneto-optical disks, and semiconductor memories are installed in the drive as needed, allowing computer programs read from these media to be installed in the storage section as needed.

[0252] When the central processing unit in the electronic device of this embodiment executes the program, the following is achieved: Figure 1 The steps in the low-orbit space visual interference target tracking method based on trajectory prediction are shown.

[0253] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, the computer program including a computer program for executing Figure 1 In such an embodiment, the computer program can be downloaded and installed from a network via the communication portion and / or installed from a removable medium. When the computer program is executed by the central processing unit, the various functions defined in the apparatus of the present application are performed.

[0254] in, Figure 1 The computer program instructions corresponding to the method shown can also be stored in a computer readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0255] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0256] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A low-orbit space visual interference target tracking method based on trajectory prediction, characterized in that: include: Before interference, the target is tracked and its position and features are obtained based on the target image frame and the pre-trained spatial tracking target model, and the trajectory prediction model and graph neural network model are trained online. Calculate the peak response within the selected target frame area and determine whether the target is blocked by comparing the relative distance between the highest peak and the second highest peak to determine whether to call the trajectory prediction model and graph neural network model; When the target is occluded, the graph neural network model is called to perform local correction on the target position, and the locally corrected target position is transmitted to the trajectory prediction model for target trajectory prediction. The target is then tracked based on the target prediction coordinates transmitted to the spatial tracking target model by the trajectory prediction model. After the interference, the spatial tracking target model is used to detect the target and track it by itself. At the same time, the target tracking position transmitted by the spatial tracking target model to the trajectory prediction model and the graph neural network model is used to continuously update the trajectory prediction model.

2. The low-orbit space visual interference target tracking method based on trajectory prediction according to claim 1 is characterized in that: The process of calling the graph neural network model to perform local correction on the target position is: The graph neural network model is used to regard the local feature map of the target's historical trajectory as a graph node, construct a time series graph and correct the target position.

3. The low-orbit space visual interference target tracking method based on trajectory prediction according to claim 2, characterized in that: The features of the adjacent nodes of the time series graph are used to smooth outliers; the position of the current abnormal node is adjusted to the weighted average of its neighboring nodes, while ensuring that neighbors closer to the current frame have a greater impact on the correction result.

4. The low-orbit space visual interference target tracking method based on trajectory prediction according to claim 1, characterized in that: The target is tracked according to the target predicted coordinates transmitted to the spatial tracking target model by the trajectory prediction model, and the attention around the predicted target coordinates is enhanced based on the local attention mechanism.

5. The low-orbit space visual interference target tracking method based on trajectory prediction according to claim 1, characterized in that: In the process of continuously updating the trajectory prediction model, the graph neural network model is used to compare and analyze the error between the target predicted trajectory output by the trajectory prediction model and the target actual trajectory output by the spatial tracking target model, and the weight of the trajectory prediction model is dynamically adjusted according to the error.

6. The low-orbit space visual interference target tracking method based on trajectory prediction according to claim 1, characterized in that: The spatial tracking target model is a SiamRPN model.

7. The low-orbit space visual interference target tracking method based on trajectory prediction according to claim 1, characterized in that: The trajectory prediction model adopts the LSTM model.

8. A low-orbit space visual interference target tracking system based on trajectory prediction, characterized in that: include: An online training module, which is used to track the target and obtain the target position and extract target features based on the target image frame and the pre-trained spatial tracking target model before interference, and to train the trajectory prediction model and graph neural network model online; The occlusion judgment module is used to calculate the peak response within the selected target frame area and determine whether the target is occluded by comparing the relative distance between the highest peak and the second highest peak, so as to determine whether to call the trajectory prediction model and the graph neural network model; The occluded target tracking module is used to call the graph neural network model to perform local correction on the target position when the target is occluded, and transmit the locally corrected target position to the trajectory prediction model for target trajectory prediction. The target is then tracked based on the target prediction coordinates transmitted by the trajectory prediction model to the spatial tracking target model; The unobstructed target tracking module is used to detect the target and track it by itself after interference using the spatial tracking target model. At the same time, it uses the target tracking position transmitted by the spatial tracking target model to the trajectory prediction model and the graph neural network model to continuously update the trajectory prediction model.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the low-orbit space visual interference target tracking method based on trajectory prediction are implemented as described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the low-orbit space visual interference target tracking method based on trajectory prediction are implemented as described in any one of claims 1-7.

Images