Moving vehicle tracking model construction method in satellite video

Abstract

The invention discloses a method for constructing a moving vehicle tracking model in a satellite video. The method comprises a correlation filtering target tracking module construction method, a neural network target tracking compensation module construction method and a correlation parameter feedback module construction method. In a correlation filtering target tracking module, in order to cope with a complex background in a satellite video, a histogram of oriented gradient feature, a gray feature and a CN color feature are adopted to carry out fusion training on a correlation filter, and the discrimination rate between a target vehicle and a similar object and the algorithm robustness are improved. And meanwhile, a shielding judgment standard is established by utilizing an APCE value of the correlation filtering response diagram after fusion training, so that the shielding state of the vehicle target is effectively detected, and the target in the shielding state is subjected to driving track prediction by adopting correlation filtering. And after shielding is detected to be finished, connection and correction of a correlation filtering prediction trajectory and a correlation filter output trajectory are realized, so that continuous tracking is realized. In the neural network target tracking compensation module, the output position of a filter is subjected to secondary correction through a Siamese network, and meanwhile, a KM algorithm and a region-based target recovery network are used for forming a tracking trajectory, recovering a shielding target, reducing a label conversion error and improving the visual perception and understanding ability of an image. In the related parameter feedback module, an incidence matrix is established by using an augmented path algorithm, a feedback value is obtained by calculating an area intersection-to-union ratio, and related parameters of Kalman filtering and a Siamese network are updated, so that the detection precision and robustness are improved, the system flow is simplified, and the efficiency is improved. According to the method provided by the invention, large-range vehicle tracking can be carried out by using satellite videos, and the method has relatively high tracking precision and efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of target tracking. More specifically, the present invention relates to a method for constructing a moving vehicle tracking model in satellite video. Background Art

[0002] With the rapid development of computer technology, vehicle tracking technology has broad application prospects and important significance in many fields such as intelligent transportation, military security and environmental monitoring. The emergence of video satellites has made up for the inadequacy of traditional satellites in obtaining dynamic information of moving vehicles, and provided a new opportunity for real-time and continuous large-scale observation of the ground. The discriminant tracking algorithm based on correlation filtering has been widely used in the field of target tracking. It not only has the advantages of high efficiency and high precision, but also does not require a large amount of training data. Through the application of kernel techniques, circulant matrices and Fourier transform methods, it has low computational complexity and low computing power requirements for equipment, which can improve the performance of the model. The output results of the correlation filter are compensated and optimized using a neural network to improve the recognition accuracy and robustness of the model. By establishing a correlation matrix to provide feedback to the above network modules, the accuracy, parameter adjustment and real-time feedback of the model are continuously optimized, thereby improving the performance and application effect of the model.

[0003] At present, when tracking moving vehicle targets in satellite videos, the applicant has found at least the following problems:

[0004] First, the complex background information in satellite videos and the occlusion of buildings, vegetation and their shadows can easily lead to tracking errors and tracking losses. How to track moving vehicles efficiently and accurately under the above circumstances has become the key to current research.

[0005] Second, the application of discriminative tracking algorithms based on correlation filtering in relatively new satellite video data still faces the problem of tracking errors caused by similar objects and tracking losses caused by buildings, vegetation and their shadows.

[0006] At present, there is no clear and effective solution to the problem of tracking errors caused by complex background areas and similar objects in satellite videos in related technologies. How to improve the target detection effect and target detection accuracy in satellite videos is an urgent problem to be solved in this field. Summary of the invention

[0007] The present application example provides a method for constructing a moving vehicle tracking model in satellite video, which can track moving vehicles in satellite video with high efficiency and high precision.

[0008] A method for constructing a moving vehicle tracking model in satellite video comprises the following steps:

[0009] Step S1: Read satellite video images, manually select the vehicle target to be tracked or read a file with tracking vehicle target frame information;

[0010] Manually selecting the vehicle target to be tracked means using a mouse to select the target to be tracked in the first frame of the satellite video, and the frame selection method is a circumscribed rectangle formed from the upper left corner to the lower right corner of the target;

[0011] The file containing the target frame information of the tracking vehicle is read to store the file containing the target label and the coordinates of the upper left corner and the lower right corner of its circumscribed rectangle;

[0012] Furthermore, the file with the tracking vehicle target frame information is a file in which the vehicle target frame information is manually written; the file format includes but is not limited to .txt file, .xml file and .xlsx file, etc.;

[0013] Step S2: Extract the features of the selected tracking vehicle target to train the kernel correlation filter to obtain the filter template M n ;

[0014] The filter templates include directional gradient histogram feature filter templates, grayscale feature filter templates and CN color feature filter templates;

[0015] Furthermore, the features of the tracking target are extracted from the target image block after cyclic shift processing.

[0016] Step S3: Filter the template M according to the previous frame n-1 Determine the search area of ​​the current frame target and extract features within the search area of ​​the current frame target;

[0017] The features include oriented gradient histogram features, grayscale features and CN color features;

[0018] Step S4: performing cross-correlation calculation on the directional gradient histogram feature of the target search area of ​​the current frame and the directional gradient histogram feature filter template of the previous frame to obtain a directional gradient histogram feature response map;

[0019] The grayscale feature in the target search area of ​​the current frame is cross-correlated with the grayscale feature filter template of the previous frame to obtain a grayscale feature response map;

[0020] The CN color features in the target search area of ​​the current frame are cross-correlated with the CN color feature filter template of the previous frame to obtain a CN color feature response map;

[0021] Step S5: Calculate the average peak correlation energy (APCE) value of the directional gradient histogram feature response map, the grayscale feature response map, and the CN color feature response map;

[0022] The directional gradient histogram feature response map, the grayscale feature response map and the CN color feature response map are adaptively weighted and added to obtain the final correlation filter response map F(z);

[0023] Through the above-mentioned adaptive feature response fusion method, the accuracy of target tracking can be effectively improved;

[0024] Furthermore, the adaptive weighted addition refers to calculating the corresponding combination weight according to the maximum value and average peak correlation energy (APCE) of the directional gradient histogram feature response map, the grayscale feature response map and the CN color feature response map.

[0025] Step S6: judging whether the occlusion condition is met according to the APCE value of the final correlation filter response graph;

[0026] Furthermore, the shielding condition is: C APCE <R APCE , where C APCE Represents the APCE value of the final correlation filter response calculated in each frame of video image, R APCE Indicates the occlusion determination threshold;

[0027] Step S7: If the occlusion condition is not met, then the maximum value area of ​​the final correlation filter response map is the target location of the current frame filter output;

[0028] Extract the features of the target location in the current frame, and use the ridge regression objective function to train the kernel correlation filter to obtain the updated filter template M′ of the current frame. n ;

[0029] The updated filter templates include a directional gradient histogram feature filter template, a grayscale feature filter template, and a CN color feature filter template;

[0030] Step S8: If the occlusion condition is met, Kalman filtering is enabled to predict and track the driving trajectory of the vehicle target, and the trajectory points predicted by the Kalman filter are used to replace the tracking position output by the filter;

[0031] Extract the target features in the neighborhood of the predicted trajectory point and stop updating the filter template;

[0032] The target features include oriented gradient histogram features, grayscale features and CN color features;

[0033] Further, in step S8, the Kalman filter predicts the neighborhood of the trajectory point to be a tracking target box size S selected with the trajectory point as the center. k ;

[0034] Step S9: judging whether the occlusion end condition is met according to the APCE value of the final correlation filter response graph;

[0035] Step S10: If the occlusion end condition is met, the directional gradient histogram feature of the target in the neighborhood of the trajectory point predicted by the Kalman filter of the current frame is cross-correlated with the directional gradient histogram feature filter template of the previous frame that meets the occlusion condition to obtain a directional gradient histogram feature response map;

[0036] The grayscale feature response map is obtained by performing cross-correlation calculation on the grayscale feature of the target in the neighborhood of the trajectory point predicted by the Kalman filter of the current frame and the grayscale feature filter template of the previous frame that meets the occlusion condition;

[0037] The CN color features of the target in the neighborhood of the trajectory point predicted by the Kalman filter of the current frame are cross-correlated with the CN color feature filter template of the previous frame that meets the occlusion condition to obtain the CN color feature response map;

[0038] Enable the correlation filter to calculate the position of the vehicle target in the next frame, and connect the Kalman prediction trajectory of the current frame with the filter output tracking trajectory;

[0039] Continue step S3 to step S9;

[0040] Step S11: If the occlusion end condition is not met, continue with step S8 to step S9;

[0041] Step S12: output and save the tracking target position;

[0042] Step S13: using a deep learning target detection algorithm to detect and identify the tracking target;

[0043] Furthermore, representative algorithms of deep learning target detection include YOLO series, Faster-RCNN series, etc.

[0044] Step S14: Calculate the similarity score y between the candidate frame image detected in step S13 and the vehicle target to be tracked manually annotated in step S1 df ;

[0045] Furthermore, the Siamese network is used to calculate the similarity score. The Siamese network can be expanded through transfer learning. By pre-training on a large-scale dataset, the learned feature representation can be transferred to the new target tracking task, reducing the need for a large amount of labeled data. It can then quickly adapt to different target categories and scenarios and achieve better target tracking performance.

[0046] Furthermore, the Siamese network uses a twin network structure with symmetrical shared weights to compare the similarity between the target and the candidate box for target matching and tracking; the choice of threshold in the Siamese network will vary according to the specific application scenario and needs, and needs to be adjusted and optimized according to the characteristics of the specific data set and task, and finally the determined threshold is verified using cross-validation and evaluation indicators.

[0047] Furthermore, the calculation of similarity scores usually uses the Euclidean distance to measure the similarity between two samples;

[0048] Step S15: judging whether the target tracking condition is met according to the similarity score;

[0049] Furthermore, we set the threshold y in the Siamese network sia ;

[0050] Furthermore, if the output similarity score y df >Threshold y sia , you can start tracking and output the tracking target position; if the output similarity score y df <Threshold y sia , then repeat steps S13 to S15;

[0051] Step S16: performing target assignment and data association on the position output by the Kalman filter in step S10 and the output position in step S15 to form a tracking trajectory;

[0052] Furthermore, the KM algorithm is used to assign targets and associate data to the output positions and form tracking trajectories. The matching scheme is gradually optimized through multiple iterations to obtain the optimal association between targets and data, as well as the corresponding tracking trajectories.

[0053] Furthermore, the basic idea of ​​the KM algorithm is to find a set of optimal allocations that minimize the total cost. Here, the positions of the Kalman filter output can be used as rows and the positions of the Siamese network output can be used as columns to construct a cost matrix by calculating the cost or distance between them. The KM algorithm is then used to convert this cost matrix into the best target allocation solution, that is, to associate the position of each Kalman filter output with the position of the most matching Siamese network output.

[0054] Step S17: Combine the final result output in step S16, restore the occluded target, reduce the label conversion error, and output the final prediction result;

[0055] Furthermore, the output result is restored using a region-based occluded object recovery network, which generates an image of the restored object and fuses it with the result of the occlusion segmenter. The result of the occlusion segmentation can be used to obtain an accurate occluded region mask, which is then applied to the restored target image to restore the fully occluded area. The restored object is then fused with the original object through image fusion technology to make the boundary smoother and more natural;

[0056] Furthermore, by jointly training and reasoning the region-based occlusion target recovery network with the target detection or recognition network, more accurate and complete target detection or recognition results can be obtained. Specifically, the region-based occlusion target recovery network can generate some possible occlusion restoration candidates based on information such as the location and characteristics of the occlusion, and then fuse the restored image area with the original image to obtain the corrected target area, and then output the final prediction result;

[0057] Step S18: After obtaining the final prediction result outputted in step S17, establish a correlation matrix, obtain a feedback value to be fed back to the Kalman filter in step S8 and the twin network in step S14, and update their related parameters;

[0058] Furthermore, a feedback mechanism is established through the augmenting path algorithm, in which the actual output result and the final prediction result are combined to establish a correlation matrix according to the area intersection and union ratio, and finally a feedback value is obtained and fed back to the Kalman filter and Siamese network.

[0059] Furthermore, the feedback mechanism in the target detection system based on the augmenting path algorithm can improve the performance of the target tracking or target recognition system through parameter updating and adjustment by establishing the correlation matrix and feeding its information back to the Kalman filter and Siamese network. This feedback mechanism allows the system to continuously optimize the prediction results based on the actual detection results to achieve more accurate and robust target detection.

[0060] Step S19: Repeat steps S3 to S18 for multiple iterations until all target images are processed.

[0061] The beneficial effects of the present invention are:

[0062] In order to cope with the complex background in the satellite video, the correlation filter target tracking module in the present invention uses the directional gradient histogram feature, the grayscale feature and the CN color feature to perform fusion training on the correlation filter, thereby improving the discrimination rate and algorithm robustness between the target vehicle and similar objects. At the same time, the APCE value of the correlation filter response map after fusion training is used to establish an occlusion discrimination standard, thereby effectively detecting the occlusion state of the vehicle target, and using the correlation filter to predict the driving trajectory for the target in the occlusion state. After the occlusion is detected, the connection and correction of the correlation filter prediction trajectory and the correlation filter output trajectory are realized, thereby realizing continued tracking.

[0063] The neural network target tracking compensation module in the present invention performs secondary correction on the filter output position through the Siamese network, and uses the KM algorithm and the region-based target recovery network to form a tracking trajectory, restore the occluded target, reduce the label conversion error, and improve the visual perception and understanding ability of the image.

[0064] The relevant parameter feedback module in the present invention uses an augmenting path algorithm to establish a correlation matrix, and obtains feedback values ​​by calculating the area intersection ratio to update the relevant parameters of the Kalman filter and the Siamese network, thereby improving the detection accuracy and robustness, simplifying the system process and improving efficiency.

[0065] The method provided by the present invention can utilize satellite video to track vehicles over a large area, and has high tracking accuracy and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0067] Figure 1 The present invention is a flow chart of the construction of the moving vehicle target tracking model in the satellite video

[0068] Figure 2 This is a flowchart of the correlation filtering target tracking module of the present invention.

[0069] Figure 3 This is the principle diagram of the kernel correlation filter tracking algorithm described in the present invention.

[0070] Figure 4 This is a visualization principle diagram of the occlusion discrimination threshold setting of the present invention.

[0071] Figure 5 This is a flowchart of the neural network tracking compensation module of the present invention.

[0072] Figure 6 This is a flow chart of the construction of the relevant parameter feedback module of the present invention. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0074] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0075] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a kind of", "the" and the like involved in this application do not indicate a quantitative limitation and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices.

[0076] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] like Figure 1 As shown, the method for constructing a moving vehicle tracking model in satellite video of the present invention comprises the following steps:

[0078] Input the actual tracking target manually marked in step S1 into the correlation filter target tracking module and the neural network target tracking compensation module;

[0079] The input data enters the correlation filter target tracking module in step S1 to step S12, through Figure 2 After a series of processing in step S12, the target position is output;

[0080] The input data enters the neural network tracking compensation module in step S13 to step S17, through Figure 5 After a series of processing in step S17, the prediction result is output;

[0081] In step S17, the prediction result is output to compensate the target position output in step S12 to improve the model accuracy and recognition rate;

[0082] The prediction result output in step S17 and the target position output in step S12 are input into the relevant parameter feedback module in step S18, and the actual detection result and the final prediction result are used to establish a correlation matrix according to the area intersection and union ratio through the augmenting path algorithm, and then the feedback value is output to update the relevant parameters in the relevant filter target tracking module and the neural network tracking compensation module. This feedback mechanism continuously optimizes the prediction result according to the actual detection result to achieve more accurate and robust target detection;

[0083] Finally, in step S19, multiple iterations are performed until all target images are processed.

[0084] like Figure 2 As shown, the method for constructing the correlation filtering target tracking module of the present invention comprises the following steps:

[0085] Step S1: read the satellite video image, manually select the vehicle target to be tracked or read the file with the tracking vehicle target frame information; the manual selection of the vehicle target to be tracked refers to using the mouse to select the target to be tracked in the first frame of the satellite video, and the frame selection method is the circumscribed rectangle formed from the upper left corner to the lower right corner of the target; the reading of the file containing the tracking vehicle target frame information refers to a file storing the target label and the coordinates of the upper left corner and the lower right corner of its circumscribed rectangle.

[0086] Step S2: Figure 3 As shown, the features of the selected vehicle target are extracted to train the kernel correlation filter to obtain the filter template M n ; The filtering template includes a directional gradient histogram feature filtering template, a grayscale feature filtering template and a CN color feature filtering template.

[0087] Step S3: Figure 3 As shown, according to the previous frame filtering template M n-1Determine the search area of ​​the current frame target and extract the features in the current frame target search area; the features include directional gradient histogram features, grayscale features and CN color features; the directional gradient histogram features of the target are denoted as M HOG , including N image channels, specifically represented by M HOG ={m HOG_1 ,m HOG_2 ,…,m HOG_N}; The grayscale feature of the target is recorded as M GRAY , including N image channels, specifically represented by M GRAY ={m GRAY_1 ,m GRAY_2 ,…,m GRAY_N}; The CN color feature of the target is recorded as M CN , including N image channels, specifically represented by M CN ={m CN_1 ,m CN_2 ,…,m CN_N}.

[0088] Step S4: Figure 3 As shown, the directional gradient histogram features of the target search area of ​​the current frame are cross-correlated with the directional gradient histogram feature filtering template of the previous frame to obtain the directional gradient histogram feature response map; the grayscale features in the target search area of ​​the current frame are cross-correlated with the grayscale feature filtering template of the previous frame to obtain the grayscale feature response map; the CN color features in the target search area of ​​the current frame are cross-correlated with the CN color feature filtering template of the previous frame to obtain the CN color feature response map.

[0089] The cross-correlation calculation method is: Where F represents the calculated correlation filter response graph; m c Represents the features of the cth image channel of the current frame, including the directional gradient histogram feature m HOG , grayscale feature m GRAY And CN color feature m CN ;h c Represents the feature filter template of the c-th image channel in the previous frame, including the directional gradient histogram feature filter template h HOG , grayscale feature filter template h GRAY And CN color feature filter template h CN .

[0090] After calculating the feature response graphs of N image channels of each feature, the corresponding feature response graph F is obtained by summing them up. HOG (z), F GRAY (z) and F CN (z).

[0091] Step S5: Figure 3 As shown, the average peak correlation energy (APCE) values ​​of the directional gradient histogram feature response map, the grayscale feature response map and the CN color feature response map are calculated; the directional gradient histogram feature response map, the grayscale feature response map and the CN color feature response map are adaptively weighted and added to obtain the final correlation filter response map; the above-mentioned adaptive feature response fusion method can effectively improve the accuracy of target tracking;

[0092] The final correlation filter response map is obtained by adaptively weighted addition of the directional gradient histogram feature response map, the grayscale feature response map, and the CN color feature response map. The calculation formula is:

[0093]

[0094] Among them, F(z) represents the final correlation filter response map, and They represent the directional gradient histogram feature response map F HOG (z), grayscale feature response map F GRAY (z) and CN color feature response map F CN (z) the corresponding combination weight;

[0095] The adaptive weights of the directional gradient histogram feature response map, the grayscale feature response map, and the CN color feature response map can be calculated by the following formula: Among them, f M (t) = f max ×APCE; by f max and APCE value to calculate f M (t), and through all previous frames f M The mean of (t) is normalized, and the weight of the directional gradient histogram feature response map is finally obtained: The grayscale feature response map weight is The CN color feature response weight is

[0096] Step S6: Figure 4 As shown, whether the occlusion condition is met is determined based on the APCE value of the final correlation filter response graph.

[0097] The occlusion condition is: C APCE <R APCE ; Among them, C APCE Represents the APCE value of the final response graph calculated in the current frame; R APCE Represents the occlusion discrimination threshold, which is calculated as follows: in, represents the average APCE value, σ APCE Indicates the standard deviation APCE value, Max APCE Indicates the maximum APCE value on each frame, m is the total number of satellite video frames, γ and β are adjustable parameters used to control the looseness of the threshold. Usually, the value range of γ is between 1.5 and 2.5, and the value range of β is between 0.5 and 1.

[0098] Step S7: If the occlusion condition is not met, then the maximum value area of ​​the final correlation filter response map is the target location of the current frame filter output; extract the features of the target location of the current frame, and use the ridge regression objective function to train the kernel correlation filter to obtain the updated filter template M′ of the current frame n ; The updated filter template includes a directional gradient histogram feature filter template, a grayscale feature filter template and a CN color feature filter template;

[0099] The updated filtering template M′ n The calculation formula is:

[0100] M′ n =(1-α)M ′ n-1 +αM n

[0101] Among them, α is a constant representing the learning rate, which is calculated as follows:

[0102]

[0103] Among them, α0 is the initial learning rate, k is the decay coefficient, and p is the number of decay rounds;

[0104] In this formula, linear decay and exponential decay are multiplied together to quickly reduce the learning rate in the early stage of training, and then gradually reduce the learning rate in the later stage, which helps to better adapt to the initial stage and stabilize the later training of the model. By adjusting the values ​​of α0 and k, the speed and amplitude of the decline of the learning rate can be controlled, further improving the convergence performance of the model;

[0105] Step S8: If the occlusion condition is met, Kalman filtering is enabled to predict the driving trajectory of the tracking vehicle target, and the trajectory points predicted by the Kalman filter are used to replace the tracking position output by the filter; the target features in the neighborhood of the predicted trajectory points are extracted and the updating of the filter template is stopped; the target features include directional gradient histogram features, grayscale features and CN color features;

[0106] The Kalman filter predicts the neighborhood of the trajectory point to be the size of the tracking target box S selected with the trajectory point as the center. k , the calculation formula for the tracking target box size is:

[0107] S k =H+(V×T+0.5×A×T 2 )×M

[0108] Among them, S k Indicates the size of the tracking target box, H is the height of the target, V is the speed of the target, T is the time step, indicating the predicted time interval, and M is the multiplication factor used to adjust the size of the box;

[0109] Step S9: judging whether the occlusion end condition is met according to the APCE value of the final correlation filter response graph;

[0110] The occlusion end condition is: C APCE >R APCE

[0111] Step S10: If the occlusion end condition is met, the directional gradient histogram feature of the target in the neighborhood of the Kalman filter prediction trajectory point of the current frame is cross-correlated with the directional gradient histogram feature filter template of the previous frame that meets the occlusion condition to obtain a directional gradient histogram feature response map; the grayscale feature of the target in the neighborhood of the Kalman filter prediction trajectory point of the current frame is cross-correlated with the grayscale feature filter template of the previous frame that meets the occlusion condition to obtain a grayscale feature response map; the CN color feature of the target in the neighborhood of the Kalman filter prediction trajectory point of the current frame is cross-correlated with the CN color feature filter template of the previous frame that meets the occlusion condition to obtain a CN color feature response map; enable the correlation filter to calculate the position of the vehicle target in the next frame, and connect the Kalman prediction trajectory of the current frame with the filter output tracking trajectory; continue steps S3 to S9;

[0112] Step S11: If the occlusion end condition is not met, continue with steps S8 to S9;

[0113] Step S12: output and save the tracking target position;

[0114] like Figure 5 As shown, the method for constructing a neural network tracking compensation module of the present invention comprises the following steps:

[0115] Step S13: using a deep learning target detection algorithm to detect and identify the tracking target;

[0116] Representative algorithms for deep learning target detection include the YOLO series, Faster-RCNN series, etc.

[0117] Step S14: Calculate the similarity score between the candidate frame image detected in step S13 and the vehicle target to be tracked manually marked in step S1

[0118] The calculation of similarity score usually uses Euclidean distance to measure the similarity between two samples. The calculation relationship between similarity score and Euclidean distance is as follows:

[0119]

[0120] in is the Euclidean distance between samples x1 and x2, and α is a positive constant used to adjust the curve shape of the similarity score. For a given α value, the training set is used for model training and Euclidean distance calculation, and then the validation set is used to calculate the similarity score. By comparing the model performance under different α values, the corresponding α value is selected.

[0121] Step S15: judging whether the target tracking condition is met according to the similarity score;

[0122] Set the threshold in the Siamese network, the calculation formula is:

[0123]

[0124] l is the recognition accuracy under different satellite videos, and its calculation method is:

[0125]

[0126] Where: g is the number of frames identified; G is the total number of frames of video data; f is the resolution of video data; t is the total duration of video data;

[0127] Determine the relationship between the output similarity score and the set threshold:

[0128] If the output similarity score y df >Threshold y sia , you can start tracking and output the tracking target position;

[0129] If the output similarity score y df <Threshold y sia , then repeat steps S13 to S15;

[0130] Step S16: performing target assignment and data association on the position output by the Kalman filter in step S10 and the output position in step S15 to form a tracking trajectory;

[0131] The KM algorithm is used to perform target assignment and data association on the output position of the correlation filter target tracking module and the predicted position of the neural network tracking compensation module to form a tracking trajectory. The matching scheme is gradually optimized through multiple iterations to obtain the optimal association between the target and the data, as well as the corresponding tracking trajectory.

[0132] The basic idea of ​​the KM algorithm is to find a set of optimal allocations that minimize the total cost. Here, the positions of the Kalman filter output can be used as rows and the positions of the Siamese network output can be used as columns to construct a cost matrix by calculating the cost or distance between them. The KM algorithm is then used to convert this cost matrix into the best target allocation solution, that is, to associate the position of each Kalman filter output with the position of the most matching Siamese network output.

[0133] Step S17: Combine the final result output in step S16, restore the occluded target, reduce the label conversion error, and output the final prediction result;

[0134] The output is restored using a region-based occluded object recovery network, which generates an image of the recovered object and fuses it with the result of the occlusion segmenter. The result of the occlusion segmentation can be used to obtain an accurate occluded region mask, which is then applied to the restored target image to restore the fully occluded area. The restored object is then fused with the original object through image fusion technology to make the boundary smoother and more natural;

[0135] In step S17, by jointly training and reasoning the region-based occlusion target recovery network with the target detection or recognition network, a more accurate and complete target detection or recognition result can be obtained. Specifically, the region-based occlusion target recovery network can generate some possible occlusion restoration candidates based on information such as the location and characteristics of the occlusion, and then fuse the restored image region with the original image to obtain the corrected target region, and then output the final prediction result;

[0136] like Figure 6 As shown, the method for constructing the relevant parameter feedback module of the present invention includes the following steps:

[0137] Step S18: After obtaining the final prediction result outputted in step S17, establish a correlation matrix, obtain a feedback value to be fed back to the Kalman filter in step S8 and the twin network in step S14, and update their related parameters;

[0138] The feedback mechanism in the target detection system based on the augmenting path algorithm can improve the performance of the target tracking or target recognition system through parameter update and adjustment by establishing the correlation matrix and feeding its information back to the Kalman filter and Siamese network. This feedback mechanism allows the system to continuously optimize the prediction results based on the actual detection results to achieve more accurate and robust target detection.

[0139] This method based on the augmenting path algorithm aims to match the actual detection results with the final prediction results, establish an association matrix by calculating the area intersection ratio, and use the augmenting path algorithm to find the best match. By comparing the accuracy of the match, the feedback value is calculated and used to update the parameters of the Kalman filter and Siamese network. Such a feedback mechanism helps to improve the accuracy and robustness of target detection and prediction, and further optimize the performance of the algorithm.

[0140] Step S19: Repeat steps S3 to S18 for multiple iterations until all target images are processed.

[0141] It should be noted that the terms "comprises", "includes" or any other variations thereof in this article are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus.

[0142] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for constructing a moving vehicle tracking model in satellite video, characterized in that: The following steps are involved: Step S1: Read satellite video images, manually select the vehicle target to be tracked or read a file with tracking vehicle target frame information; Manually selecting the vehicle target to be tracked means using a mouse to select the vehicle target to be tracked in the first frame of the satellite video, and the frame selection method is a circumscribed rectangle formed from the upper left corner to the lower right corner of the target; The file containing the target frame information of the tracking vehicle is read to store the file containing the target label and the coordinates of the upper left corner and the lower right corner of its circumscribed rectangle; Step S2: Extract the features of the selected tracking vehicle target to train the kernel correlation filter to obtain the filter template M n ; The filter templates include directional gradient histogram feature filter templates, grayscale feature filter templates and CN color feature filter templates; Step S3: Filter the template M according to the previous frame n-1 Determine the search area of ​​the current frame target and extract features within the search area of ​​the current frame target; The features include oriented gradient histogram features, grayscale features and CN color features; Step S4: performing cross-correlation calculation on the directional gradient histogram feature of the target search area of ​​the current frame and the directional gradient histogram feature filter template of the previous frame to obtain a directional gradient histogram feature response map; The grayscale feature in the target search area of ​​the current frame is cross-correlated with the grayscale feature filter template of the previous frame to obtain a grayscale feature response map; The CN color features in the target search area of ​​the current frame are cross-correlated with the CN color feature filter template of the previous frame to obtain a CN color feature response map; Step S5: Calculate the average peak correlation energy (APCE) value of the directional gradient histogram feature response map, the grayscale feature response map, and the CN color feature response map; The directional gradient histogram feature response map, the grayscale feature response map and the CN color feature response map are adaptively weighted and added to obtain the final correlation filter response map; Step S6: judging whether the occlusion condition is met according to the APCE value of the final correlation filter response graph; Step S7: If the occlusion condition is not met, then the maximum value area of ​​the final correlation filter response map is the target location of the current frame filter output; Extract the features of the target location in the current frame, and use the ridge regression objective function to train the kernel correlation filter to obtain the updated filter template M′ of the current frame. n ; The updated filter templates include a directional gradient histogram feature filter template, a grayscale feature filter template, and a CN color feature filter template; Step S8: If the occlusion condition is met, Kalman filtering is enabled to predict and track the driving trajectory of the vehicle target, and the trajectory points predicted by the Kalman filter are used to replace the tracking position output by the correlation filter; Extract the target features in the neighborhood of the predicted trajectory point and stop updating the filter template; The target features include oriented gradient histogram features, grayscale features and CN color features; Step S9: judging whether the occlusion end condition is met according to the APCE value of the final correlation filter response graph; Step S10: If the occlusion end condition is met, the directional gradient histogram feature of the target in the neighborhood of the trajectory point predicted by the Kalman filter of the current frame is cross-correlated with the directional gradient histogram feature filter template of the previous frame that meets the occlusion condition to obtain a directional gradient histogram feature response map; The grayscale feature response map is obtained by performing cross-correlation calculation on the grayscale feature of the target in the neighborhood of the trajectory point predicted by the Kalman filter of the current frame and the grayscale feature filter template of the previous frame that meets the occlusion condition; The CN color features of the target in the neighborhood of the Kalman filter predicted trajectory point in the current frame are cross-correlated with the CN color feature filter template of the previous frame that meets the occlusion condition to obtain the CN color feature response map; the correlation filter is enabled to calculate the position of the vehicle target in the next frame, and the Kalman predicted trajectory of the current frame is connected with the tracking position output by the correlation filter; Continue step S3 to step S10; Step S11: If the occlusion end condition is not met, continue with step S8 to step S9; Step S12: output and save the tracking target position; Step S13: using a deep learning target detection algorithm to detect and identify the tracking target; Step S14: Calculate the similarity score between the candidate frame image detected in step S13 and the vehicle target to be tracked manually marked in step S1; Step S15: Determine whether the target tracking condition is met based on the similarity score: Step S16: performing target assignment and data association on the position output by the Kalman filter in step S10 and the output position in step S15 to form a tracking trajectory; Step S17: Combine the output results in step S16, restore the occluded target, reduce the label conversion error, and output the final prediction result; Step S18: After obtaining the final prediction result outputted in step S17, establish a correlation matrix, obtain a feedback value to be fed back to the Kalman filter in step S8 and the twin network in step S14, and update their related parameters; Step S19: Repeat steps S3 to S18 for multiple iterations until all target images are processed.

2. The method for constructing a moving vehicle tracking model in satellite video according to claim 1, characterized in that: In step S1, the file with the tracking vehicle target frame information is a file in which the vehicle target frame information is manually written; the file format includes but is not limited to .txt file, .xml file, and .xlsx file.

3. The method for constructing a moving vehicle tracking model in satellite video according to claim 1, characterized in that: In step S2, the features of the tracking target are extracted from the target image block after cyclic shift processing.

4. The method for constructing a moving vehicle tracking model in satellite video according to claim 1, characterized in that: In step S5, the adaptive weighted addition refers to calculating the corresponding combination weights according to the maximum values ​​and average peak correlation energy (APCE) of the directional gradient histogram feature response map, the grayscale feature response map and the CN color feature response map. Among them, F(z) represents the final correlation filter response map, and They represent the directional gradient histogram feature response map F HOG (z), grayscale feature response map F GRAY (z) and CN color feature response map F CN (z) The corresponding combination weight.

5. The method for constructing a moving vehicle tracking model in satellite video according to claim 1, characterized in that: In step S6, the shielding condition is: C APCE <R APCE , where C APCE Represents the APCE value of the final correlation filter response map calculated in each frame of video image, R APCE Indicates the occlusion determination threshold.

6. The method for constructing a moving vehicle tracking model in satellite video according to claim 1, characterized in that: In step S7, the updated filter template M′ n The calculation method of M' is: n =(1-α)M′ n-1 +αM n , where α is a constant representing the learning rate.

7. The method for constructing a moving vehicle tracking model in satellite video according to claim 1, characterized in that: In step S8, the Kalman filter predicts the neighborhood of the trajectory point to select a tracking target box S with the trajectory point as the center. k .

8. The method for constructing a moving vehicle tracking model in satellite video according to claim 1, characterized in that: In step S14, a Siamese network is used to calculate the similarity score. The Siamese network can be expanded through transfer learning. By pre-training on a large-scale data set, the learned feature representation can be transferred to a new target tracking task, reducing the need for a large amount of labeled data. It can then quickly adapt to different target categories and scenarios to achieve better target tracking performance. The calculation of the similarity score usually uses the Euclidean distance to measure the similarity between two samples. The Euclidean distance is calculated, and the value obtained by the Euclidean distance calculation is converted based on the relationship between the similarity score and the Euclidean distance to obtain the similarity score y required by the present invention. df .

9. The method for constructing a moving vehicle tracking model in satellite video according to claim 1, characterized in that: In step S15, the threshold in the Siamese network is set to determine whether the target tracking condition is met. The threshold is represented by the symbol y sia express. If the output similarity score y df >Threshold y sia , you can start tracking and output the tracking target position; If the output similarity score y df <Threshold y sia , then repeat steps S13 to S15.

10. The method for constructing a moving vehicle tracking model in satellite video according to claim 1, characterized in that: In step S14 to step S15, the Siamese network plays a role in compensating the frame-by-frame prediction of the Kalman filter, so as to correct the deviation between the Kalman filter prediction result and the actual result, so as to improve the accuracy and robustness of the model.

11. The method for constructing a moving vehicle tracking model in satellite video according to claim 1, characterized in that: In step S16, the KM algorithm is used to perform target assignment and data association on the output position and form a tracking trajectory. The matching scheme is gradually optimized through multiple iterations to obtain the optimal association between the target and the data, as well as the corresponding tracking trajectory.

12. The method for constructing a moving vehicle tracking model in satellite video according to claim 1, characterized in that: In step S17, the output result is restored using a region-based occlusion target recovery network, which generates an image of the restored target and fuses it with the result of the occlusion segmenter. The result of the occlusion segmentation can be used to obtain an accurate occlusion region mask, and the mask is applied to the restored target image to restore the completely occluded area. The restored target is then fused with the original target through image fusion technology to make the boundary smoother and more natural.

13. The method for constructing a moving vehicle tracking model in satellite video according to claim 1, characterized in that: In step S18, a feedback mechanism is established through an augmenting path algorithm, in which a correlation matrix is ​​established between the actual output result and the final prediction result according to the area intersection and union ratio, and finally a feedback value is obtained and fed back to the Kalman filter and the Siamese network.

Images