Multi-time-sensitive individual trajectory generation acceleration method based on satellite video data
Through parallel prediction and feature extraction at the main node, combined with multi-node distributed computing, the problem of low time-saving data processing in traditional methods is solved, and efficient multi-time sensitive individual trajectory generation is achieved, which is suitable for the limited resource environment of domestic chips. .
Patent Information
- Application Number
- CN202411506281.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-28
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-10-28
AI Technical Summary
The traditional multi-time sensitive individual trajectory generation method has low time-efficiency in processing high-resolution satellite video data and high hardware resource requirements, making it difficult to meet practical application scenarios, especially under the limited computing power of domestic chips.
The position and motion state prediction and motion feature extraction of the main node parallel time-sensitive individuals are used, and the Mahalanobis distance and Cosine distance calculation are realized through multi-node distributed parallel calculation. Combined with adaptive acceleration Kalman filtering and SpeedNet detector, multi-time-sensitive individual trajectory is performed Generate acceleration.
The processing speed and accuracy of multi-time sensitive individual trajectory generation has been significantly improved, the hardware resource requirements have been reduced, and efficient trajectory generation under limited resources has been achieved, and the processing timeliness has been improved by about 2.5 times.
Smart Images

Figure CN119027458A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of near real-time target tracking based on satellite video data, and in particular relates to a multi-time-sensitive individual trajectory generation acceleration method based on satellite video data. Background Art
[0002] In recent years, various methods for generating multi-temporal-sensitive individual trajectories based on satellite video data have emerged, which has greatly promoted the development of this field. However, as the resolution, size, and bit depth of satellite video data increase, the traditional multi-temporal-sensitive individual trajectory generation methods have bottleneck problems such as complex business processes, large amounts of memory reading and writing, and huge parameter processing, resulting in low processing timeliness. In order to meet the needs of fast multi-temporal-sensitive individual trajectory generation based on satellite video data, the requirements for hardware resources are extremely high. Many devices in the production environment are difficult to meet these hardware resource requirements, resulting in the multi-temporal-sensitive individual trajectory generation based on satellite video data being difficult to meet actual application scenarios. Therefore, the research on acceleration methods for the generation of multi-temporal-sensitive individual trajectories based on satellite video data is of great practical significance for promoting the development of the field.
[0003] With the rapid development of domestic chips, the timeliness of generating multi-time-sensitive individual rapid trajectories using traditional satellite video data is greatly limited. Therefore, how to use the limited computing power of domestic chips to quickly and efficiently generate multi-time-sensitive individual trajectories of satellite video data within an acceptable range of accuracy loss has become a frontier and practical problem in current research. Summary of the invention
[0004] The purpose of the present invention is to provide a method for accelerating the generation of multi-time-sensitive individual trajectories based on satellite video data, which realizes the acceleration of the generation of multi-time-sensitive individual trajectories in satellite video data by accelerating means such as predicting the position and motion state of time-sensitive individuals in parallel at the main node, extracting motion features, and initially matching time-sensitive individuals at multiple nodes.
[0005] The technical solution adopted by the present invention is a multi-time-sensitive individual trajectory generation acceleration method based on satellite video data, comprising the following steps: S1, the master node receives the location and confidence extraction results of the time-sensitive individual in the current frame of satellite video data; S2, creates two threads and starts working at the same time. The working mode of one thread is to use the historical trajectory information of the time-sensitive individual to apply the adaptive acceleration Kalman filter to complete the prediction of the position and motion state on the CPU, and the working mode of the other thread is to use the SpeedNet detector to complete the time-sensitive individual motion feature extraction on the domestic graphics computing chip; Adaptive acceleration Kalman filter is composed of 12-dimensional motion parameters and an adaptive noise covariance matrix Composition, among which ( ) is the target position of the current frame of satellite video data D The horizontal and vertical positions of the center point, the aspect ratio of the target, and the target width, ( ) represents the velocity vector of the target in the current frame of satellite video data S , ( ) represents the acceleration vector P ; The core of SpeedNet detector is to use self-supervised lightweight and compact detection head to complete the near-real-time extraction of target motion features in the current frame of satellite video data with a small number of parameters and low power consumption of domestic chips; S3, judging whether it is in the tracking state according to the predicted results of the time-sensitive individual position and motion state, if it is in the tracking state, combining the time-sensitive individual motion feature extraction results, perform multi-node time-sensitive individual initial matching; if it is in the non-tracking state or the tracking state initial matching fails, perform EfficientIou time-sensitive individual rematch; Multi-node time-sensitive individual initial matching uses multi-node distributed parallel computing to achieve the minimum association cost distance matrix MinCostMatching Calculation, Matrix MinCostMatching The value of each element in is obtained by weighting the Mahalanobis distance and the Cosine distance. The master node divides the Mahalanobis distance and the Cosine distance calculation into multiple tasks through the multi-task distribution mode, leaving one task to be executed on the master node, and the rest of the tasks are distributed to the slave nodes for execution. After the slave node tasks are completed, the results of each slave node are merged to the master node to complete the matrix MinCostMatching Reconstruction of EfficientIou time-sensitive individual weight matching uses multi-threading on the master node to implement the EfficientIou time-sensitive individual weight matching cost matrix EffiouCost The Hungarian algorithm is used to calculate the optimal match, where the cost matrix EffiouCost Each element in is the EfficientIou distance between the current frame extraction result of the satellite video data and the target or non-tracking trajectory of the multi-node time-sensitive individual that failed to match initially; S4, if the initial matching of multi-node time-sensitive individuals is successful or the re-matching of EfficientIou time-sensitive individuals is successful, the trajectory of the matched extraction results is updated; if the re-matching of EfficientIou time-sensitive individuals fails, a new trajectory ID is assigned to the unmatched extraction results, and the trajectory generation results of the time-sensitive individuals are output.
[0006] Furthermore, the specific steps of S2 are: B1, based on t-The state information of the time-sensitive individual in 1 frame is used to predict the state of the time-sensitive individual in the current frame of satellite video data using the adaptive acceleration Kalman filter. The specific calculation formula is: in, t Represents the current frame of satellite video data; t- 1 represents the previous frame of the current frame of satellite video data; for t The motion state of the frame is determined by the target position D、 Velocity Vector S and the acceleration vector P constitute; represents the prediction process noise; Satellite video data t Frame and t- 1 frame imaging time difference; B2, using the position information of the time-sensitive individual to extract the pixel matrix corresponding to the current frame of the satellite video data, performing Lanczos interpolation size scaling on the pixel matrix to unify the input size, and using the SpeedNet detector to complete the extraction of the time-sensitive individual motion features on the domestic graphics computing chip; The order of steps B1 and B2 is adjustable, regardless of priority.
[0007] Furthermore, the core of the SpeedNet detector is the self-supervised lightweight and compact detection head. The specific implementation steps of the self-supervised lightweight and compact detection head are: b1, input characteristics of time-sensitive individuals at different scales; b2, use dilated convolution to unify the number of input channels of each branch; b3, the SpeedHost module is composed of two different groups of convolutions in series. The first group of convolutions consists of one depth-wise separable convolution + HSwish activation function, and the second group of convolutions consists of two depth-wise separable convolutions + HSwish activation functions. Finally, projection convolution is used to achieve the fusion of the two groups of convolutions at the channel scale. b4, perform projection convolution + HSwish activation function again to further compress and reduce the dimension of features; b5, using three different groups of convolutions to map the feature map to a higher dimension to obtain stronger fitting ability, where each group of convolutions is composed of a different number of depthwise separable convolutions + HSwish activation functions in series, and using feature splicing to achieve the fusion of different receptive field features, which can more accurately align and extract key features in the image, especially when the shape and size of the same target vary in different frames of satellite video data.
[0008] Furthermore, in S3, the specific steps of performing the initial matching of multi-node time-sensitive individuals are: c11, the master node divides the time-sensitive individual extraction results, the time-sensitive individual trajectory of the tracking state, and the appearance features extracted by the SpeedNet detector into multiple tasks, leaving one task on the master node and distributing the remaining tasks to each slave node for execution; c12, the master node and the slave node calculate the Mahalanobis distance cost matrix and the Cosine distance cost matrix on their respective nodes; The calculation formula of the Mahalanobis distance cost matrix is: in, For the m The time-sensitive individual extraction result matrix; For the n The time-sensitive individual trajectory matrix of the tracking state; H The time-sensitive individual covariance matrix predicted by the adaptive acceleration Kalman filter; The calculation formula of the Cosine distance cost matrix is: in, Indicates n A detection box appearance feature matrix; represents the first m The appearance characteristics of each trajectory; c13, the master node receives the results of the Mahalanobis distance cost matrix and the Cosine distance cost matrix from the slave node and completes the aggregation of the two cost matrices to reconstruct the minimum association cost distance matrix MinCostMatching , the specific calculation formula is: in DisMah is the overall Mahalanobis distance cost matrix; DisCos is the overall Cosine distance cost matrix; g represents the weight coefficient; c14, the minimum association cost distance matrix according to the maximum threshold MaxDistance MinCostMatching To filter, assign infinity to the elements in the matrix that exceed the threshold, and use the Hungarian matching algorithm to MinCostMatching Complete the initial matching of multi-node time-sensitive individuals.
[0009] Furthermore, in S3, the specific steps of performing EfficientIou time-sensitive individual weight matching are: c21, calculate the EfficientIou time-sensitive individual weight matching cost matrix EffiouCost ; c22, use the Hungarian matching algorithm to match the cost matrix EffiouCost Make the best match.
[0010] Furthermore, in step c21, the EfficientIou time-sensitive individual weight matching cost matrix is calculated. EffiouCost The formula is: in, Represents the current frame of satellite video data n The time-sensitive target extraction results and multi-node m The intersection and union ratio of the trajectories of the target or non-tracking state that the time-sensitive individuals failed to match initially; Represents the current frame of satellite video data m Time-sensitive target extraction results; Represents multiple nodes n The trajectory of a time-sensitive individual that initially fails to match the target or is in a non-tracking state; , They represent the width and height of the minimum circumscribed rectangular frame of the two frames covering the extraction result of the current frame of satellite video data and the trajectory of the multi-node time-sensitive individual that fails to match the initial or non-tracking state; , Respectively represent m The width of the current frame extraction result of satellite video data, n The width of the target or non-tracking state where the multi-node time-sensitive individual initially fails to match; , Respectively represent m The height of the current frame extraction result of the satellite video data, n The height of the target or non-tracking state where the multi-node time-sensitive individual fails to initially match; Represents the Euclidean distance.
[0011] The beneficial effects of the present invention are: (1) The present invention implements a multi-time-sensitive individual trajectory generation acceleration method based on satellite video data, splits the steps of the multi-time-sensitive individual trajectory generation algorithm and reconstructs the logic adaptability. By completing the prediction of the time-sensitive individual position and motion state and the motion feature extraction in parallel on the main node, and completing the relatively time-consuming Mahalanobis distance calculation and Cosine distance calculation in parallel on multiple nodes, the multi-node hardware architecture characteristics of the terminal device are fully utilized to solve the bottleneck problems of low tracking accuracy and poor processing timeliness of the existing traditional processing algorithms under resource-constrained conditions. At the same time, more time-sensitive individuals can be tracked quickly and correctly, which also provides strong support for the engineering deployment and application of the multi-time-sensitive individual trajectory generation algorithm.
[0012] (2) The SpeedNet detector is used to extract time-sensitive individual motion features. The core of the detector is the use of a self-supervised lightweight and compact detection head. The grouped convolutional parallel network structure is used to fuse features of different receptive fields, allowing a single detection head to use features of different scales. This simplifies the entire processing flow, significantly reduces the number of parameters, and significantly improves the processing speed, ultimately achieving the acceleration of the generation of multiple time-sensitive individual trajectories in satellite video data. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 It is a schematic diagram of the overall process of the method of the present invention.
[0014] Figure 2 Local display of the trajectory generation results of the method of the present invention Figure 1 .
[0015] Figure 3 Local display of the trajectory generation results of the method of the present invention Figure 2 . DETAILED DESCRIPTION
[0016] In order to make the purpose, technical solution and advantages of the present invention clearer and more understandable, the technical solution of the present invention will be further described in detail below with reference to the accompanying drawings.
[0017] A method for accelerating the generation of multiple time-sensitive individual trajectories based on satellite video data is proposed. The purpose is to accelerate the generation of multiple time-sensitive individual trajectories based on satellite video data in order to meet the high timeliness and high precision requirements of time-sensitive individual trajectory generation under the condition of limited terminal computing resources. Figure 1 As shown, the following steps are included: S1, the master node receives the location and confidence extraction results of the time-sensitive individual in the current frame of satellite video data.
[0018] S2, creates two threads and starts working at the same time. The working mode of one thread is to use the historical trajectory information of the time-sensitive individual to apply the adaptive acceleration Kalman filter to complete the prediction of the position and motion state on the CPU (FT-D2000), and the working mode of the other thread is to use the SpeedNet detector to complete the time-sensitive individual motion feature extraction on the domestic graphics computing chip (Atlas200).
[0019] Adaptive acceleration Kalman filter is composed of 12-dimensional motion parameters and an adaptive noise covariance matrix Composition, among which ( ) is the target position of the current frame of satellite video data D The horizontal and vertical positions of the center point, the aspect ratio of the target, and the target width, ( ) represents the velocity vector of the target in the current frame of satellite video dataS , ( ) represents the acceleration vector P .
[0020] The core of the SpeedNet detector is the use of a self-supervised lightweight and compact detection head, which can complete the near-real-time extraction of target motion features in the current frame of satellite video data with a small number of parameters and low power consumption of domestic chips.
[0021] The specific steps of using the time-sensitive individual historical trajectory information to apply adaptive acceleration Kalman filtering to complete the prediction of position and motion state on the CPU and using the SpeedNet detector to complete the extraction of time-sensitive individual motion features on the domestic graphics chip are as follows: B1, based on t- The state information of the time-sensitive individual in 1 frame is used to predict the state of the time-sensitive individual in the current frame of satellite video data using the adaptive acceleration Kalman filter. The specific calculation formula is: in, t Represents the current frame of satellite video data, t- 1 represents the previous frame of the current frame of satellite video data, for t The motion state of the frame is determined by the target position D、 Velocity Vector S and the acceleration vector P constitute; represents the prediction process noise; is the imaging time difference between the two frames of images; B2, use the position information of the time-sensitive individual to extract the pixel matrix corresponding to the current frame of the satellite video data, perform Lanczos interpolation size scaling on the pixel matrix, uniformly scale it to 1024×1024 pixel size, and use the SpeedNet detector to complete the extraction of time-sensitive individual motion features on the domestic graphics computing chip.
[0022] The core of SpeedNet detector is to use self-supervised lightweight and compact detection head. The specific implementation steps of self-supervised lightweight and compact detection head are as follows: b1, input characteristics of time-sensitive individuals at different scales; b2, using dilated convolution to unify the number of input channels of each branch, significantly improving the feature extraction capability and reducing network parameters; b3, the SpeedHost module is composed of two groups of different convolution methods in series. The first group of convolutions consists of one depth-wise separable convolution + HSwish activation function, and the second group of convolutions consists of two depth-wise separable convolutions + HSwish activation functions. Finally, projection convolution is used to achieve the fusion of the two groups of convolutions at the channel scale. By segmentation feature extraction and channel fusion, the number of parameters is significantly reduced; b4, perform projection convolution + HSwish activation function again to further compress and reduce the features, effectively reducing the number of model parameters and computational complexity; b5, using three different groups of convolutions to map the feature map to a higher dimension to obtain stronger fitting ability, where each group of convolutions is composed of a different number of depthwise separable convolutions + HSwish activation functions in series, and using feature splicing to achieve the fusion of different receptive field features, which can more accurately align and extract key features in the image, especially when the shape and size of the same target vary in different frames of satellite video data.
[0023] It should be noted that the order of steps B1 and B2 is adjustable and there is no particular order.
[0024] S3, judge whether it is in the tracking state according to the predicted results of the time-sensitive individual position and motion state. If it is in the tracking state, perform multi-node time-sensitive individual initial matching based on the time-sensitive individual motion feature extraction results; if it is in the non-tracking state or the initial tracking state matching fails, perform EfficientIou time-sensitive individual re-matching.
[0025] Multi-node time-sensitive individual initial matching uses multi-node distributed parallel computing to achieve the minimum association cost distance matrix MinCostMatching Calculation, Matrix MinCostMatching The value of each element in is obtained by weighting the Mahalanobis distance and the Cosine distance. The master node divides the Mahalanobis distance and the Cosine distance calculation into multiple tasks through the multi-task distribution mode, leaving one task to be executed on the master node, and the rest of the tasks are distributed to the slave nodes for execution. After the slave node tasks are completed, the results of each slave node are merged to the master node to complete the matrix MinCostMatching The specific steps of reconstruction are: c11, the master node divides the time-sensitive individual extraction results, the time-sensitive individual trajectory of the tracking state, and the appearance features extracted by the SpeedNet detector into multiple tasks, leaving one task on the master node and distributing the rest of the tasks to each slave node for execution; c12, the master node and the slave node calculate the Mahalanobis distance cost matrix and the Cosine distance cost matrix on their respective nodes; The calculation formula of the Mahalanobis distance cost matrix is: in, For the m The time-sensitive individual extraction result matrix; For the n The time-sensitive individual trajectory matrix of the tracking state; H The time-sensitive individual covariance matrix predicted by the adaptive acceleration Kalman filter; The calculation formula of the Cosine distance cost matrix is: in, Indicates n A detection box appearance feature matrix; represents the first m The appearance characteristics of each trajectory; c13, the master node receives the results of the Mahalanobis distance cost matrix and the Cosine distance cost matrix from the slave node and completes the aggregation of the two cost matrices to reconstruct the minimum association cost distance matrix MinCostMatching , the specific calculation formula is: in DisMah is the overall Mahalanobis distance cost matrix; DisCos is the overall Cosine distance cost matrix; g represents the weight coefficient; c14, the minimum association cost distance matrix according to the maximum threshold MaxDistance MinCostMatching To filter, assign infinity to the elements in the matrix that exceed the threshold, and use the Hungarian matching algorithm to MinCostMatching Complete the initial matching of multi-node time-sensitive individuals.
[0026] EfficientIou time-sensitive individual weight matching uses multi-threading on the master node to implement the EfficientIou time-sensitive individual weight matching cost matrix EffiouCos The Hungarian algorithm is used to calculate the optimal match, where the cost matrix EffiouCos Each element in is the EfficientIou distance between the current frame extraction result of the satellite video data and the target or non-tracking trajectory of the multi-node time-sensitive individual that failed to match the initial match. The specific steps are: c21, calculate the EfficientIou time-sensitive individual weight matching cost matrix EffiouCos , the calculation formula is: in, Represents the current frame of satellite video data n The time-sensitive target extraction results and multi-node m The intersection and union ratio of the trajectories of the target or non-tracking state that the time-sensitive individuals failed to match initially; Represents the current frame of satellite video data m Time-sensitive target extraction results; Represents multiple nodes n The trajectory of a time-sensitive individual that initially fails to match the target or is in a non-tracking state; , They represent the width and height of the minimum circumscribed rectangular frame of the two frames covering the extraction result of the current frame of satellite video data and the trajectory of the multi-node time-sensitive individual that fails to match the initial or non-tracking state; , Respectively represent m The width of the current frame extraction result of satellite video data, n The width of the target or non-tracking state where the multi-node time-sensitive individual initially fails to match; , Respectively represent m The height of the current frame extraction result of the satellite video data, n The height of the target or non-tracking state where the multi-node time-sensitive individual fails to initially match; Represents the Euclidean distance.
[0027] c22, use the Hungarian matching algorithm to match the cost matrix EffiouCos Make the best match.
[0028] S4, if the multi-node time-sensitive individual is initially matched successfully or the EfficientIou time-sensitive individual is re-matched successfully, the trajectory of the matched extraction result is updated; if the EfficientIou time-sensitive individual re-match fails, a new trajectory ID is assigned to the unmatched extraction result, and the trajectory generation result of the time-sensitive individual is output. The trajectory generation processing time of the time-sensitive individual is 2.1 seconds (processing 100 frames of satellite video data), which is about 2.5 times faster than the traditional method. The output trajectory generation result of the time-sensitive individual is as follows: Figure 2 , Figure 3 shown.
[0029] The contents not described in detail in this specification belong to the prior art known to those skilled in the art.
Claims
1. A multi-time-sensitive individual trajectory generation acceleration method based on satellite video data, characterized in that: The following steps are involved: S1, the master node receives the location and confidence extraction results of the time-sensitive individual in the current frame of satellite video data; S2, creates two threads and starts working at the same time. The working mode of one thread is to use the historical trajectory information of the time-sensitive individual to apply the adaptive acceleration Kalman filter to complete the prediction of the position and motion state on the CPU, and the working mode of the other thread is to use the SpeedNet detector to complete the time-sensitive individual motion feature extraction on the domestic graphics computing chip; Adaptive acceleration Kalman filter is composed of 12-dimensional motion parameters and an adaptive noise covariance matrix Composition, among which ( ) is the target position of the current frame of satellite video data D The horizontal and vertical positions of the center point, the aspect ratio of the target, and the target width, ( ) represents the velocity vector of the target in the current frame of satellite video data S , ( ) represents the acceleration vector P ; The core of SpeedNet detector is to use self-supervised lightweight and compact detection head to complete the near-real-time extraction of target motion features in the current frame of satellite video data with a small number of parameters and low power consumption of domestic chips; S3, judging whether it is in the tracking state according to the predicted results of the time-sensitive individual position and motion state, if it is in the tracking state, combining the time-sensitive individual motion feature extraction results, perform multi-node time-sensitive individual initial matching; if it is in the non-tracking state or the tracking state initial matching fails, perform EfficientIou time-sensitive individual rematch; Multi-node time-sensitive individual initial matching uses multi-node distributed parallel computing to achieve the minimum association cost distance matrix MinCostMatching Calculation, Matrix MinCostMatching The value of each element in is obtained by weighting the Mahalanobis distance and the Cosine distance. The master node divides the Mahalanobis distance and the Cosine distance calculation into multiple tasks through the multi-task distribution mode, leaving one task to be executed on the master node, and the rest of the tasks are distributed to the slave nodes for execution. After the slave node tasks are completed, the results of each slave node are merged to the master node to complete the matrix MinCostMatching Reconstruction of EfficientIou time-sensitive individual weight matching uses multi-threading on the master node to implement the EfficientIou time-sensitive individual weight matching cost matrix EffiouCost The Hungarian algorithm is used to calculate the optimal match, where the cost matrix EffiouCost Each element in is the EfficientIou distance between the current frame extraction result of the satellite video data and the target or non-tracking trajectory of the multi-node time-sensitive individual that failed to match initially; S4, if the initial matching of multi-node time-sensitive individuals is successful or the re-matching of EfficientIou time-sensitive individuals is successful, the trajectory of the matched extraction results is updated; if the re-matching of EfficientIou time-sensitive individuals fails, a new trajectory ID is assigned to the unmatched extraction results, and the trajectory generation results of the time-sensitive individuals are output.
2. The method for accelerating the generation of multi-time-sensitive individual trajectories based on satellite video data according to claim 1 is characterized in that: The specific steps of S2 are: B1, based on t- The state information of the time-sensitive individual in 1 frame is used to predict the state of the time-sensitive individual in the current frame of satellite video data using the adaptive acceleration Kalman filter. The specific calculation formula is: in, t Represents the current frame of satellite video data; t- 1 represents the previous frame of the current frame of satellite video data; for t The motion state of the frame is determined by the target position D、 Velocity Vector S and the acceleration vector P constitute; represents the prediction process noise; Satellite video data t Frame and t- 1 frame imaging time difference; B2, using the position information of the time-sensitive individual to extract the pixel matrix corresponding to the current frame of the satellite video data, performing Lanczos interpolation size scaling on the pixel matrix to unify the input size, and using the SpeedNet detector to complete the extraction of the time-sensitive individual motion features on the domestic graphics computing chip; The order of steps B1 and B2 is adjustable, regardless of priority.
3. The method for accelerating the generation of multi-time-sensitive individual trajectories based on satellite video data according to claim 2 is characterized in that: The core of SpeedNet detector is to use self-supervised lightweight and compact detection head. The specific implementation steps of self-supervised lightweight and compact detection head are as follows: b1, input characteristics of time-sensitive individuals at different scales; b2, use dilated convolution to unify the number of input channels of each branch; b3, the SpeedHost module is composed of two different groups of convolutions in series. The first group of convolutions consists of one depth-wise separable convolution + HSwish activation function, and the second group of convolutions consists of two depth-wise separable convolutions + HSwish activation functions. Finally, projection convolution is used to achieve the fusion of the two groups of convolutions at the channel scale. b4, perform projection convolution + HSwish activation function again to further compress and reduce the dimension of features; b5, using three different groups of convolutions to map the feature map to a higher dimension to obtain stronger fitting ability, where each group of convolutions is composed of a different number of depthwise separable convolutions + HSwish activation functions in series, and using feature splicing to achieve the fusion of different receptive field features, which can more accurately align and extract key features in the image, especially when the shape and size of the same target vary in different frames of satellite video data.
4. The method for accelerating the generation of multi-time-sensitive individual trajectories based on satellite video data according to claim 2, characterized in that: In S3, the specific steps of performing the initial matching of multi-node time-sensitive individuals are: c11, the master node divides the time-sensitive individual extraction results, the time-sensitive individual trajectory of the tracking state, and the appearance features extracted by the SpeedNet detector into multiple tasks, leaving one task on the master node and distributing the rest of the tasks to each slave node for execution; c12, the master node and the slave node calculate the Mahalanobis distance cost matrix and the Cosine distance cost matrix on their respective nodes; The calculation formula of the Mahalanobis distance cost matrix is: in, For the m The time-sensitive individual extraction result matrix; For the n The time-sensitive individual trajectory matrix of the tracking state; H The time-sensitive individual covariance matrix predicted by the adaptive acceleration Kalman filter; The calculation formula of the Cosine distance cost matrix is: in, Indicates n A detection box appearance feature matrix; represents the first m The appearance characteristics of each trajectory; c13, the master node receives the results of the Mahalanobis distance cost matrix and the Cosine distance cost matrix from the slave node and completes the aggregation of the two cost matrices to reconstruct the minimum association cost distance matrix MinCostMatching , the specific calculation formula is: in DisMah is the overall Mahalanobis distance cost matrix; DisCos is the overall Cosine distance cost matrix; g represents the weight coefficient; c14, the minimum association cost distance matrix according to the maximum threshold MaxDistance MinCostMatching To filter, assign infinity to the elements in the matrix that exceed the threshold, and use the Hungarian matching algorithm to MinCostMatching Complete the initial matching of multi-node time-sensitive individuals.
5. The method for accelerating the generation of multi-time-sensitive individual trajectories based on satellite video data according to claim 1, characterized in that: In S3, the specific steps of performing EfficientLou time-sensitive individual weight matching are: c21, calculate the EfficientIou time-sensitive individual weight matching cost matrix EffiouCost ; c22, use the Hungarian matching algorithm to match the cost matrix EffiouCost Make the best match.
6. The method for accelerating the generation of multi-time-sensitive individual trajectories based on satellite video data according to claim 5, characterized in that: In step c21, the EfficientIou time-sensitive individual weight matching cost matrix is calculated EffiouCost The formula is: in, Represents the current frame of satellite video data n The time-sensitive target extraction results and multi-node m The intersection and union ratio of the trajectories of the target or non-tracking state that the time-sensitive individuals failed to match initially; Represents the current frame of satellite video data m Time-sensitive target extraction results; Represents multiple nodes n The trajectory of a time-sensitive individual that initially fails to match the target or is in a non-tracking state; , They represent the width and height of the minimum circumscribed rectangular frame of the two frames covering the extraction result of the current frame of satellite video data and the trajectory of the multi-node time-sensitive individual that fails to match the initial or non-tracking state; , Respectively represent m The width of the current frame extraction result of satellite video data, n The width of the target or non-tracking state where the multi-node time-sensitive individual initially fails to match; , Respectively represent m The height of the current frame extraction result of the satellite video data, n The height of the target or non-tracking state where the multi-node time-sensitive individual fails to initially match; Represents the Euclidean distance.
Citation Information
Patent Citations
Multi-target tracking positioning and motion state estimation method based on unmanned aerial vehicle
CN113269098A
Multi-target tracking method for intelligent driving
CN116402850A
Correlation filtering satellite video target tracking method based on trajectory prediction
CN116977869A
Multi-target tracking method based on pseudo depth estimation and online trajectory classification
CN118196715A
Method and device for processing pixels contained in a video sequence
US20130016784A1
