Low-orbit space visual interference target tracking method and system based on trajectory prediction

Abstract

The invention belongs to the technical field of target tracking, and provides a low-orbit space visual interference target tracking method and system based on trajectory prediction in order to solve the problem that the tracking accuracy and continuity are reduced. The low-orbit space visual interference target tracking method based on trajectory prediction comprises the steps that before interference, a target is tracked, the position of the target is acquired, the features of the target are extracted, and a trajectory prediction model and a graph neural network model are trained online; calculating peak response in the selected target frame area, and judging whether the target is shielded or not by comparing the relative distance between the highest peak and the secondary peak; when the target is shielded, calling the trajectory prediction model and the graph neural network model and respectively predicting a long-term trajectory and a short-term trajectory of the target; tracking the target according to the target prediction coordinates transmitted to the space tracking target model by the track prediction model; after interference, a space tracking target model is used to detect target self-tracking. The tracking accuracy and continuity can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of target tracking, and particularly relates to a method and system for tracking a low-orbit space visual interference target based on trajectory prediction. Background Art

[0002] The statements in this part only provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] Visual interference refers to the phenomenon that the features of a target cannot be completely extracted or the target briefly disappears from the field of view due to the interference of a dynamic background or the influence of other objects. Currently, most technologies rely on filtering algorithms in traditional tracking algorithms such as Kalman filtering and its variants, etc., which are all used for transient trajectory prediction. In the aerospace field, when a target (such as a satellite or a spacecraft, etc.) passes through a star cluster, it is severely blocked for a long time, which will cause the target to be lost and unable to be tracked; at the same time, the target is affected by the star cluster, resulting in the loss of target features, and it is impossible to track based on the extracted historical target features, reducing the accuracy and continuity of tracking. Summary of the Invention

[0004] In order to solve the technical problem that the target is lost due to the interference of a dynamic background or other objects during the target tracking process, resulting in a decrease in the accuracy and continuity of tracking, the present invention provides a method and system for tracking a low-orbit space visual interference target based on trajectory prediction, which can improve the accuracy and continuity of tracking.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: The first aspect of the present invention provides a method for tracking a low-orbit space visual interference target based on trajectory prediction.

[0006] In one or more embodiments, a method for tracking a low-orbit space visual interference target based on trajectory prediction is provided, including: Before interference, track the target based on the target image frame and the pre-trained space tracking target model, obtain the target position and extract the target features, and online train the trajectory prediction model and the graph neural network model; Calculate the peak response within the selected target box area, and determine whether the target is occluded by comparing the relative distance between the highest peak and the second highest peak, so as to determine whether to call the trajectory prediction model and the graph neural network model; When the target is occluded, call the graph neural network model to locally correct the target position, and transmit the locally corrected target position to the trajectory prediction model for target trajectory prediction, and then track the target according to the target prediction coordinates transmitted by the trajectory prediction model to the space tracking target model; After interference, the spatial tracking target model is used to detect the target for self-tracking, and at the same time, the target tracking position transmitted by the spatial tracking target model to the trajectory prediction model and the graph neural network model is used to continuously update the trajectory prediction model.

[0007] As an implementation, the process of calling the graph neural network model to perform local correction on the target position is as follows: The graph neural network model is used to regard the local feature map of the target historical trajectory as a graph node, construct a time series graph, and correct the target position.

[0008] As an implementation, the features of adjacent nodes of the time series graph are used to smooth outliers; the position of the current abnormal node is adjusted to the weighted average of its neighbor nodes, while ensuring that neighbors closer to the current frame have a greater impact on the correction result.

[0009] As an implementation, while tracking the target according to the target prediction coordinates transmitted by the trajectory prediction model to the spatial tracking target model, the attention degree around the predicted target coordinates is also enhanced based on the local attention mechanism.

[0010] As an implementation, during the process of continuously updating the trajectory prediction model, the graph neural network model is used to compare and analyze the error between the target prediction trajectory output by the trajectory prediction model and the target actual trajectory output by the spatial tracking target model, and the weight of the trajectory prediction model is dynamically adjusted according to the error.

[0011] As an implementation, the spatial tracking target model is a SiamRPN model.

[0012] As an implementation, the trajectory prediction model uses an LSTM model.

[0013] The second aspect of the present invention provides a low-earth orbit space visual interference target tracking system based on trajectory prediction.

[0014] In one or more embodiments, a low-earth orbit space visual interference target tracking system based on trajectory prediction includes: An online training module, which is used to track the target based on the target image frame and the pre-trained spatial tracking target model before interference, obtain the target position and extract the target features, and online train the trajectory prediction model and the graph neural network model; An occlusion judgment module, which is used to calculate the peak response within the selected target box area, judge whether the target is occluded by comparing the relative distance between the highest peak and the second highest peak, so as to determine whether to call the trajectory prediction model and the graph neural network model; The occlusion target tracking module is used to, when the target is occluded, call the graph neural network model to perform local correction on the target position, and transmit the locally corrected target position to the trajectory prediction model for trajectory prediction of the target, and then track the target according to the target prediction coordinates transmitted by the trajectory prediction model to the space tracking target model; The non-occlusion target tracking module is used to, after interference, use the space tracking target model to detect and track the target by itself, and continuously update the trajectory prediction model by using the target tracking positions transmitted by the space tracking target model to the trajectory prediction model and the graph neural network model.

[0015] The third aspect of the present invention provides a computer-readable storage medium.

[0016] A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the steps in the above-mentioned low-orbit space vision interference target tracking method based on trajectory prediction.

[0017] The fourth aspect of the present invention provides an electronic device.

[0018] An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the steps in the above-mentioned low-orbit space vision interference target tracking method based on trajectory prediction.

[0019] Compared with the prior art, the beneficial effects of the present invention are: The present invention uses the peak value of the peak response in the selected target box area to judge whether the target is occluded, and then determines whether to call the trajectory prediction model and the graph neural network model. When the target is occluded, the graph neural network model is called to perform local correction on the target position, and the locally corrected target position is transmitted to the trajectory prediction model. Moreover, after interference, the space tracking target model is used to detect and track the target by itself, which improves the accuracy and continuity of target tracking. Description of the Drawings

[0020] The specification drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention.

[0021] Figure 1 is a flowchart of the low-orbit space vision interference target tracking method based on trajectory prediction according to an embodiment of the present invention; Figure 2 is a schematic diagram of the target being occluded when passing through a star cluster according to an embodiment of the present invention; Figure 3It is a process diagram of low-earth orbit space visual interference target tracking based on trajectory prediction according to an embodiment of the present invention; Figure 4 It is a comparison graph of the peaks without interference and with interference according to an embodiment of the present invention; Figure 5 It is a judgment flow of whether the target is interfered according to an embodiment of the present invention; Figure 6 It is the SiamRPN model architecture according to an embodiment of the present invention; Figure 7 It is a schematic structural diagram of a low-earth orbit space visual interference target tracking system based on trajectory prediction according to an embodiment of the present invention; Figure 8 It is an effect diagram of visual interference target tracking based on trajectory prediction according to an embodiment of the present invention. Detailed implementation manners

[0022] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0023] It should be noted that the following detailed descriptions are all illustrative and are intended to provide further explanations of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0024] It should be noted that the terms used herein are only for describing specific implementation manners and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0025] Term explanation: SiamRPN is a lightweight model based on the Siamese network, which can meet the real-time requirements of space target tracking while ensuring accuracy; SiamRPN can quickly generate multiple candidate regions and select the most likely target position through a scoring mechanism, so it can still show strong robustness in the face of complex space backgrounds.

[0026] The LSTM (Long Short-Term Memory) is good at processing and predicting time series data. Through its special gating mechanism (input gate, forget gate, and output gate), it can effectively learn and remember long-term dependencies, which is beneficial for predicting future positions using historical motion states. The GNN (Graph Neural Network Model) conducts a comparative analysis of the predicted trajectory of the LSTM and the actual trajectory of the SiamRPN, generates feedback information, dynamically adjusts the weights of the LSTM or introduces correction terms to improve the prediction accuracy of the LSTM.

[0027] Figure 1 It is a schematic flowchart of a low-earth orbit space visual interference target tracking method based on trajectory prediction in an embodiment of the present invention. Combining Figure 1 and Figure 3 , the low-earth orbit space visual interference target tracking method in this embodiment may include steps S101 to S104: S101, before interference, track the target based on the target image frame and the pre-trained space tracking target model, obtain the target position, and extract the target features, and online train the trajectory prediction model and the graph neural network model.

[0028] Figure 2 In (a) of Figure 2 is the state diagram of the target (such as a satellite or other spacecraft, etc.) when it is near the star cluster;

[0029] In this embodiment, the space tracking target model is the SiamRPN model. The trajectory prediction model uses the LSTM model.

[0030] Before interference, use SiamRPN to track the target, obtain the target position, and extract the target features.

[0031] The structure of SiamRPN is as shown in Figure 6 . The process of tracking the target by SiamRPN is as follows: Step (1.1): First, outline the target in the first frame of the video, denoted as the template image z; then the area in each subsequent frame that may contain the target position is denoted as the search image x.

[0032] Step (1.2): SiamRPN uses a Siamese network with two shared-weight sub-networks (Convolutional Neural Network CNN) to extract features (edges, brightness, shape) from the template image and the search image respectively, denoted as and 。

[0033] Step (1.3): The Siamese network uses convolutional layers to fuse the template image and the search image to obtain the correlation between them.

[0034] ; represents the similarity between the template and each pixel in the search area, that is, the matching degree; is the depth feature of the th channel of the template image; is the depth feature of the th channel of the search image.

[0035] Step (1.4): The Region Proposal Network (RPN) of SiamRPN generates a series of candidate regions (anchors), performs object matching and confidence scoring on each region to determine the most likely position of the object. At the same time, the exact position of the object is predicted through the regression layer, and the position and size parameters of the object are output.

[0036] Step (1.5): Update the position of the object in the current frame according to the exact position obtained in step (1.4) to achieve the tracking of the object.

[0037] The process of obtaining the object position through SiamRPN is as follows: Step (2.1): Extract the template image features and search image features through the Siamese network, where, ; ; is the number of channels of the feature map, and H and W are the height and width of the template image feature map respectively; and are the height and width of the search image feature map respectively; represents the set of real numbers.

[0038] Step (2.2): Calculate the and similarity : ; 、 respectively represent the features of the i-th channel in the template image and the features of the j-th channel in the search image; represents the similarity between the template and each pixel in the search area, that is, the matching degree.

[0039] Step (2.3): The RPN network outputs a score map and a bounding box regression prediction; The score map represents the confidence of each candidate box, denoted as P(x, y), where x and y are the abscissa and ordinate of the candidate region respectively.

[0040] Bounding box regression: Each candidate box has a set of regression parameters, 、 representing the horizontal displacement and vertical displacement respectively; 、 representing the width change and height change of the target respectively.

[0041] Step (2.4): Determining the target position; Confidence selection: Using the maximum score function to select the candidate box with the highest confidence: ; where represents the maximum score function; Boundary regression adjustment: Adjusting the candidate box coordinates according to the regression offset: ; ; and represent the initial position and initial size of the selected candidate box respectively; and represent the adjusted position and adjusted size of the selected candidate box respectively.

[0042] Extracting target features through SiamRPN includes but is not limited to: Magnitude feature: Magnitude usually refers to the radiation intensity emitted by celestial bodies, which is represented as irregular dots with different brightness on the image.

[0043] Edge feature: Describing the boundary information in the image, which is the dividing line between the target and the background. Edge features can help capture the contour shape of the target.

[0044] Shape feature: Describing the geometric shape of the target, used to describe the boundary, size, contour, etc. of the target.

[0045] Temporal feature: Describing the changes of the target in the time series, capable of capturing the dynamic change information of the target between multiple frames.

[0046] In this embodiment, the core of the GNN lies in modeling with a graph structure. Representing the position coordinates of each frame as nodes in a graph (denoted as ), each node can contain multiple types of information: position attribute , time attribute t, velocity attribute , acceleration attribute etc. Represent the node features as vectors , then: ; Position attribute: the position coordinates of the target in the current frame; Time attribute: the timestamp of the frame. This information is used to construct the graph structure so that the position information in the time series can reflect the movement trajectory of the target.

[0047] The edges of the graph structure reflect the smoothness of the position change between two frames. The nodes of each pair of adjacent frames are connected by edges to form a time series graph.

[0048] The weight of the edge is usually represented by the Euclidean distance between the targets in adjacent frames: ; is a tuning parameter used to control the weight decay rate. The larger the weight, the higher the similarity.

[0049] The process of constructing the time series graph is as follows: Step (3.1): Add graph nodes: When processing each frame of the video, record the target position and add a new node to the graph.

[0050] Step (3.2): Update graph edges: The newly added node is connected to the node in the previous frame by a new edge.

[0051] Global update rule: Step (3.3): The updated state of the graph is represented as G=(V, E), where: V={ , }: The set of nodes represents the position information of each frame in the time series.

[0052] E={ }: The set of edges represents the correlation information between adjacent frames in the time series.

[0053] SiamRPN transmits the obtained position coordinates to LSTM and GNN. LSTM uses a dataset of every 30 frames, and the coordinate data of the first 27 frames is used as the training set to online train the trajectory prediction model, and the last three frames are used as the validation set to verify the effect of the model. GNN obtains the coordinate data, regards the position coordinates of each frame as a graph node, constructs a time series graph and corrects local anomalies.

[0054] Among them, using the local structure of the time series graph, it is judged whether the position of a certain frame is abnormal. The main methods for anomaly detection include: (1) Velocity change detection: If the velocity change between two frames exceeds the preset threshold, then the node is considered abnormal.

[0055] (2) Graph signal smoothing: Use graph convolution to smooth the coordinate information of nodes and determine whether the local position change is abnormal. The graph signal smoothing formula is: ; is the smoothed position; is the node 's neighbor set; is the edge weight between nodes; are the coordinates of neighbor nodes.

[0056] The process of using the time series graph to correct local anomalies is as follows: Use the features of adjacent nodes to smooth the outliers: ; where, represents the position of the corrected node .

[0057] Adjust the position of the current abnormal node to the weighted average of its neighbor nodes, while ensuring that neighbors closer to the current frame have a greater impact on the correction result.

[0058] This embodiment utilizes the strong tracking ability of SiamRPN and the strong prediction ability of LSTM to achieve tracking during target interference.

[0059] S102, Calculate the peak response within the selected target box area, and determine whether the target is occluded by comparing the relative distance between the highest peak and the second highest peak, so as to determine whether to call the trajectory prediction model and the graph neural network model.

[0060] When the target is not interfered, since the space background imaging is black, the peak of the target in the RPN area is as Figure 4 shown in (a) of Figure 4 , and there is a large gap between the highest peak and the second highest peak; when the target is interfered by the star cluster light, the peak in the RPN is as Figure 5 shown in (b) of

[0061] S103, When the target is occluded, call the graph neural network model to locally correct the target position, and transmit the locally corrected target position to the trajectory prediction model for target trajectory prediction, and then track the target according to the target prediction coordinates transmitted by the trajectory prediction model to the space tracking target model.

[0062] Among them, the process of calling the graph neural network model to perform local correction on the target position is as follows: The graph neural network model is used to regard the local feature map of the target historical trajectory as a graph node, construct a time series graph, and correct the target position.

[0063] LSTM performs temporal modeling on the historical trajectory of the target, captures long-term dependencies using memory cells, and predicts the future target position. Its core is the gating mechanism (input gate, forget gate, output gate) and the update logic of the memory cell.

[0064] The input of LSTM is the historical trajectory of the target; Denote the historical trajectory of the target as: represents the position coordinates of the target in the two-dimensional space, and k is the number of time steps of the input trajectory.

[0065] The output of LSTM is the predicted trajectory of the target; The predicted trajectory of the target in the future is: .

[0066] The forget gate of LSTM determines which past information needs to be discarded. For example, if the motion pattern of the target suddenly changes, the forget gate will choose to discard the outdated trajectory information. Its expression is: ; represents the output of the forget gate, with a value range of [0, 1], indicating the forgetting ratio; represents the weight matrix of the forget gate; represents the hidden state of the previous time step; represents the input of the current time step; represents the bias term of the forget gate; represents the Sigmoid activation function, which maps the value to [0, 1].

[0067] The input gate of LSTM determines which new information needs to be added to the memory cell to update the motion state of the target: ; ; is the output of the input gate, with a value range of [0, 1], indicating the proportion of new information; is the memory cell state; The weight matrices of the input gate and the memory unit, respectively; The bias terms of the input gate and the memory unit, respectively; is the hyperbolic tangent function, which is used to map values to [−1, 1].

[0068] The output gate of the LSTM determines which information is output as the hidden state. The hidden state stores the trajectory feature information of the current time step and is an important basis for predicting future trajectories: ; The final hidden state is: ; represents the hidden state of the current time step; represents the intermediate parameter; represents the bias parameter; is the element-wise multiplication; represents the intermediate weight matrix.

[0069] The memory unit of the LSTM accumulates the long-term information of the historical trajectory and is used to capture the global law of the target movement. The memory unit combines the forget gate and the input gate to update the memory state: .

[0070] Among them, is the candidate memory cell state, indicating the new information that can be added; The process of the LSTM for target trajectory prediction is as follows: Step a: Data preprocessing; Standardize the historical trajectory coordinates using the StandardScaler function to reduce the impact of the numerical range on model training. Perform sliding window segmentation of the trajectory sequence with a fixed length, and each segment of the trajectory is used as an input sample. The StandardScaler function is: ; is the original trajectory coordinate; is the mean of the original trajectory coordinates; is the standard deviation of the original trajectory coordinates; is the data after standardization.

[0071] Step b: Input at each time step; At each time step, input a historical coordinate into the LSTM cell, combined with the hidden state and the memory unit , update the hidden state and the memory unit .

[0072] Step c: Multi-layer LSTM processing; If a multi-layer LSTM is adopted, the hidden state will be passed to the next layer of LSTM units for deeper time series feature extraction.

[0073] Step d: Predictive trajectory output; At the last time step of the last layer of LSTM, the hidden state is mapped to the predictive trajectory for the next m frames through a fully connected layer (FC): ; where, represents the predicted trajectory; represents the offset.

[0074] Use the features of adjacent nodes in the time series graph to smooth outliers; adjust the position of the current outlier node to the weighted average of its neighbor nodes, while ensuring that neighbors closer to the current frame have a greater impact on the correction result.

[0075] The GNN predicts the future position of the target by representing the time series of the target as a graph and using the information of neighboring nodes.

[0076] The input of the GNN includes: Node features: The initial feature of the graph structure data node is the coordinate of the target, denoted as: .

[0077] Edge features: The weight of the edge .

[0078] The output of the GNN: Predictive trajectory: .

[0079] The GNN updates the node features through information passing and predicts the trajectory using the information of adjacent nodes.

[0080] In each layer, the node feature is updated as follows: ; represents the feature representation of node at the l th layer; is the neighbor set of node (usually the nodes of adjacent frames before and after); is the edge weight, representing the node and degree of association; and are the weight matrix and bias vector of the l th layer; is the ReLU activation function.

[0081] Adjacent feature aggregation: Aggregate the features of neighbor nodes weighted by to obtain a weighted sum, representing the information received by the node from its neighbors.

[0082] Node feature update: Update the node's own features by passing the aggregated features through a linear transformation and an activation function.

[0083] (1) Predict the future position using the updated node features After l layers of message passing, the final feature representation of node is . This feature contains the local trajectory information of the target, and the predicted position can be obtained through : ; The MLP is a multi-layer perceptron used to map high-dimensional features to the coordinates of the next time step .

[0084] (2) Recursively predict multi-step trajectories; Predicting the trajectory of the next m frames can be achieved recursively: Use the predicted value of as the input to calculate .

[0085] Repeat the above steps until the positions of all m frames are predicted.

[0086] Specifically, while tracking the target according to the target prediction coordinates transmitted from the trajectory prediction model to the spatial tracking target model, the attention degree around the predicted target coordinates is enhanced based on the local attention mechanism.

[0087] In SiamRPN, the attention mechanism is integrated into the feature extraction processes of the template image and the search image to dynamically adjust the weights of the network for the area near the predicted position. First, construct an attention window near the predicted position ; then, assign a weight to each position in the attention window, and the size of the weight is dynamically calculated by the attention module; finally, fuse the features in a weighted manner to highlight the target area and important background features.

[0088] The process of constructing the local attention area is as follows: (1) Construct an attention window near the predicted position: Based on the predicted position of the previous frame , a local window (such as 5×5 or 7×7) is selected on the search feature map. The center of the local window is the predicted position represents the area that needs to be focused on, with a size of k×k, which controls the area range that the attention module focuses on, usually set to 1 to 2 times the target size.

[0089] (2) Extract local features: Use a cropping operation to extract the features of the attention window area from the search map to obtain local features : ; The ":" in the above formula indicates slicing a matrix or array to select all rows.

[0090] The process of calculating the attention weight is as follows: The local features generate a weight distribution through the attention module: Input fusion: The template feature and the local search feature calculate the similarity through dot product and fuse them. Output a similarity map , and are the height and width of the local window.

[0091] represents the similarity between the template feature and the search feature at position .

[0092] (2) Normalization: Perform a normalization operation on using the Softmax function to calculate the attention weight at each position : ; is the attention weight at position in the local window, indicating the importance of each position in the local area of the search map for the target prediction of the current frame. The closer to the center position of the target, the higher the weight.

[0093] The process of calculating the weighted fusion feature is as follows: Attention weighting: Use the obtained attention weights above to weight the local search features, and we can get: ; Local search features integrated with attention weights.

[0094] Feature replacement: Replace the corresponding region of the original search image features with , so that the model focuses more on the target area during target tracking, reduces the interference of irrelevant backgrounds, enables the algorithm to better capture target information, and improves the accuracy and robustness of target localization.

[0095] SiamRPN integrated attention mechanism: Template and search feature extraction: Use the Siamese network to extract the deep features of the template image and the search image .

[0096] Introduction of the attention module: In the search graph , according to the predicted position extract local features . At the same time, use the attention mechanism to generate weights , and weighted fuse the local features to obtain the weighted local features .

[0097] Feature matching and target position prediction: Input the search graph and the weighted local features into the cross-correlation module to obtain the final feature map : ; Among them, represents the final feature map; According to 's response map, determine the target position and size.

[0098] S104, after interference, use the spatial target tracking model to detect the target for self-tracking, and at the same time use the target tracking position transmitted by the spatial target tracking model to the trajectory prediction model and the graph neural network model to continuously update the trajectory prediction model.

[0099] It should be noted here that "after interference" means that the target cannot be recognized and tracked due to strong interferences such as sudden light changes and star clusters.

[0100] Specifically, during the process of continuously updating the trajectory prediction model, use the graph neural network model to compare and analyze the error between the target prediction trajectory output by the trajectory prediction model and the target actual trajectory output by the spatial target tracking model, and dynamically adjust the weights of the trajectory prediction model according to the error.

[0101] By comparing the predicted trajectories of the LSTM and the predicted trajectories of the GNN for errors : ; When is greater than the set threshold, it is considered that there is an error in the LSTM prediction.

[0102] By combining the error with a feedback coefficient to generate feedback information : ; The feedback coefficient controls the intensity of the feedback and determines the amplitude of the LSTM weight adjustment, usually taking values in [0, 1].

[0103] When the error is small, the feedback information is small, indicating that the LSTM prediction is relatively accurate and the impact of the feedback on the LSTM model is small; when the error is large, the feedback information is large, indicating that the deviation between the LSTM prediction and the actual position is large and the impact of the feedback on the LSTM is large.

[0104] Dynamically adjust the weights of the LSTM: In one or more embodiments, the GNN dynamically adjusts the weights of the LSTM model according to the feedback information , optimizing the prediction ability of the LSTM for the target trajectory. During the training of the LSTM, the feedback information generated by error analysis is used to dynamically adjust the weights of the LSTM: ; where is the weight of the LSTM model at time t + 1; is the weight of the LSTM model at the current time t, which determines the degree of influence of the historical trajectory on the current prediction. If it is large, the LSTM will rely more on the past trajectory data; conversely, the LSTM will reduce its dependence on the past trajectory and enhance its prediction ability for the current target position. is the learning rate, which controls the step size of the weight adjustment and determines the speed of the LSTM weight update. A larger learning rate will cause the weights to be updated quickly, while a smaller learning rate will result in a smoother adjustment. The weights and the learning rate are both obtained by training the prediction model using historical data through the LSTM. It is feedback information generated from the prediction errors of LSTM and GNN.

[0105] In some other alternative embodiments, the result of LSTM is corrected by using the local prediction of GNN, and the fusion weight is introduced β to dynamically adjust the trajectory of LSTM: ; : The future position predicted by LSTM according to the global motion law.

[0106] : The short-term prediction result of GNN based on local trajectory features.

[0107] : Control the fusion of the two, usually relying more on GNN when occlusion is severe.

[0108] Weight fusion parameter Determines the respective contribution ratios when fusing the prediction results of LSTM and GNN.

[0109] LSTM confidence : Confidence evaluation obtained through training with historical trajectories.

[0110] ; Among them, represents the error rate of LSTM, which is a value less than 1.

[0111] GNN confidence : Based on the average value of the edge weights output by GNN, reflecting the smoothness of the local trajectory; ; The calculation formula of is: ; The visual interference target tracking effect diagram based on trajectory prediction is as Figure 8 shown, Figure 8 In, the green trajectory is the actual trajectory of the target, the blue trajectory is the trajectory tracked by SiamRPN during the period when the target is not occluded, and the red trajectory is the predicted trajectory of LSTM corrected by GNN when the target is visually interfered.

[0112] Figure 7 This is the structural schematic diagram of a low-earth orbit space visual interference target tracking system based on trajectory prediction in an embodiment of the present invention. This embodiment corresponds to the Figure 1 low-earth orbit space visual interference target tracking method based on trajectory prediction. As Figure 7 shown, the low-earth orbit space visual interference target tracking system in this embodiment may include: An online training module 701, which is used to track the target based on the target image frame and the pre-trained spatial tracking target model before interference, obtain the target position, extract the target features, and online train the trajectory prediction model and the graph neural network model; An occlusion judgment module 702, which is used to calculate the peak response within the selected target box area, and judge whether the target is occluded by comparing the relative distance between the highest peak and the second highest peak, so as to determine whether to call the trajectory prediction model and the graph neural network model; An occluded target tracking module 703, which is used to locally correct the target position by calling the graph neural network model when the target is occluded, and transmit the locally corrected target position to the trajectory prediction model for target trajectory prediction, and then track the target according to the target prediction coordinates transmitted by the trajectory prediction model to the spatial tracking target model; An unoccluded target tracking module 704, which is used to detect and track the target by itself using the spatial tracking target model after interference, and continuously update the trajectory prediction model using the target tracking position transmitted by the spatial tracking target model to the trajectory prediction model and the graph neural network model.

[0113] It should be noted here that Figure 7 each module in the low-earth orbit space vision interference target tracking system based on trajectory prediction in Figure 1 corresponds one by one to each step in the low-earth orbit space vision interference target tracking method based on trajectory prediction in

[0114] In one or more embodiments, an electronic device is provided, including a central processing unit (CPU), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage section into a random access memory (RAM). In the RAM, various programs and data required for system operation are also stored. The central processing unit, ROM, and RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0115] The following components are connected to the I / O interface: an input part including a keyboard, a mouse, etc.; an output part including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a local area network (LAN) card, a modem, etc. The communication part performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as needed. A removable medium, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive as needed, so that a computer program read from it can be installed into the storage part as needed.

[0116] When the central processing unit in the electronic device of this embodiment executes the program, it implements the steps in the low-earth orbit space vision interference target tracking method based on trajectory prediction as Figure 1 shown.

[0117] Specifically, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing Figure 1 the method shown. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit, various functions defined in the device of the present application are executed.

[0118] Among them, Figure 1 the computer program instructions corresponding to the method shown can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one process or multiple processes and / or Figure 1 one block or multiple blocks.

[0119] Those of ordinary skill in the art can understand that to implement all or part of the processes in the above-mentioned embodiment methods, it can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned various methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0120] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for tracking a low-earth orbit space visual interference target based on trajectory prediction, characterized in that Including: Before interference, based on the target image frame and the pre-trained spatial target tracking model, track the target to obtain the target position and extract the target features, and online train the trajectory prediction model and the graph neural network model; Calculate the peak response within the selected target box area, and determine whether the target is occluded by comparing the relative distance between the highest peak and the second highest peak, so as to determine whether to call the trajectory prediction model and the graph neural network model; When the target is occluded, call the graph neural network model to locally correct the target position, and transmit the locally corrected target position to the trajectory prediction model for target trajectory prediction, and then track the target according to the target prediction coordinates transmitted by the trajectory prediction model to the spatial target tracking model; After interference, use the spatial target tracking model to detect the target for self-tracking, and at the same time use the target tracking positions transmitted by the spatial target tracking model to the trajectory prediction model and the graph neural network model to continuously update the trajectory prediction model.

2. The method for tracking a low-orbit space visual interference target based on trajectory prediction according to claim 1, wherein The process of calling the graph neural network model to locally correct the target position is as follows: Use the graph neural network model to regard the local feature map of the target historical trajectory as a graph node, construct a time series graph and correct the target position.

3. The method for tracking a low-orbit space visual interference target based on trajectory prediction according to claim 2, wherein Use the features of adjacent nodes of the time series graph to smooth the outliers; adjust the position of the current outlier node to the weighted average of its neighbor nodes, while ensuring that neighbors closer to the current frame have a greater impact on the correction result.

4. The method for tracking a low-orbit space visual interference target based on trajectory prediction according to claim 1, characterized in that, When tracking the target according to the target prediction coordinates transmitted by the trajectory prediction model to the spatial target tracking model, also enhance the attention degree around the predicted target coordinates based on the local attention mechanism.

5. The method for tracking a low-orbit space visual interference target based on trajectory prediction according to claim 1, wherein During the process of continuously updating the trajectory prediction model, use the graph neural network model to compare and analyze the error between the target prediction trajectory output by the trajectory prediction model and the target actual trajectory output by the spatial target tracking model, and dynamically adjust the weight of the trajectory prediction model according to the error.

6. The method for tracking a low-orbit space vision interference target based on trajectory prediction according to claim 1, wherein, The spatial target tracking model is the SiamRPN model.

7. The method for tracking a low-orbit space visual interference target based on trajectory prediction according to claim 1, wherein The trajectory prediction model adopts the LSTM model.

8. A low-earth orbit space visual interference target tracking system based on trajectory prediction, characterized in that, Including: An online training module, which is used to track the target based on the target image frame and the pre-trained spatial target tracking model before interference to obtain the target position and extract the target features, and online train the trajectory prediction model and the graph neural network model; An occlusion judgment module, which is used to calculate the peak response within the selected target box area, and determine whether the target is occluded by comparing the relative distance between the highest peak and the second highest peak, so as to determine whether to call the trajectory prediction model and the graph neural network model; An occluded target tracking module, which is used to call the graph neural network model to locally correct the target position when the target is occluded, and transmit the locally corrected target position to the trajectory prediction model for target trajectory prediction, and then track the target according to the target prediction coordinates transmitted by the trajectory prediction model to the spatial target tracking model; An unoccluded target tracking module, which is used to use the spatial target tracking model to detect the target for self-tracking after interference, and at the same time use the target tracking positions transmitted by the spatial target tracking model to the trajectory prediction model and the graph neural network model to continuously update the trajectory prediction model.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps in the low-earth orbit space vision interference target tracking method based on trajectory prediction according to any one of claims 1-7.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the low-earth orbit space vision interference target tracking method based on trajectory prediction according to any one of claims 1-7.

Images