Space-borne video moving vehicle tracking detection method based on space-time characteristic constraint

Abstract

The invention belongs to the technical field of moving target tracking, and particularly relates to a space-time characteristic constraint-based satellite-borne video moving vehicle tracking detection method, which comprises the following steps of: performing road extraction on a single-frame image of a satellite-borne video to obtain a road mask containing space information; generating a motion heat map containing time information based on the satellite-borne video; fusing the road mask and the motion heat map to obtain a combination result of the road mask and the motion heat map, and taking the combination result as a constructed constraint area; the method comprises the following steps: constructing a constraint area, detecting the constructed constraint area to obtain a detection result of a moving vehicle, and tracking the vehicle according to the detection result by using a vehicle association method combining vehicle relative position information and historical motion information. Therefore, the method not only extracts the road mask containing the space information, but also generates the motion heat map containing the time information, and fuses the road mask and the motion heat map to generate the constraint condition of the region of interest, thereby improving the detection accuracy of the moving vehicle.

Description

Technical Field

[0001] The invention belongs to the technical field of moving target tracking, and in particular relates to a satellite-borne video moving vehicle tracking detection method based on time-space characteristic constraints. Background Art

[0002] In recent years, low-orbit video satellites have gradually become the cutting-edge technology in the current development of remote sensing because they can quickly obtain large-scale high-resolution video data. Compared with traditional optical remote sensing satellites, the biggest feature of video satellites is that they can obtain dynamic information on the surface. The rich vehicle dynamic information in satellite-borne videos is of great significance in urban traffic planning, road construction and smart city construction. Therefore, the extraction and tracking of moving vehicles in satellite-borne videos has become one of the research hotspots in the current remote sensing field.

[0003] Compared with ordinary ground videos, satellite videos need to consider their unique properties when detecting and tracking moving vehicles. For example, satellite videos involve fewer scale changes and occlusion issues faced by ground videos, but the characteristics of video satellites shooting at high altitudes make satellite videos have a wide imaging range, resulting in a large number of complex objects in the background of satellite videos, and moving targets appear dim and tiny. At the same time, the staring imaging characteristics of video satellites cause stationary targets to appear in motion between video frames, that is, parallax pseudo-motion. This makes it impossible for traditional moving target detection algorithms to effectively identify real moving targets, pseudo-moving targets, and noise. At the same time, due to the similar characteristics of vehicles in the tracking stage, the identity of the vehicle cannot be effectively identified. Therefore, how to accurately extract and track moving vehicles from satellite videos is still one of the difficult technologies currently being studied.

[0004] To this end, the prior art proposes to use road prior knowledge to optimize the target detection algorithm, specifically by establishing a road mask to eliminate a large amount of background noise and pseudo-motion interference, thereby significantly improving the detection accuracy. However, this type of method will face various problems when establishing constraints: for example, the method based on motion information accumulation uses time domain information, but the algorithm has poor stability. When the traffic volume is small, it is impossible to effectively accumulate a complete mask, and a lot of time is required for accumulation before detection. The method based on single-frame object-oriented classification uses spatial domain information. However, this type of method is complex to calculate, and there are problems such as road omissions and breaks. For example, the method based on loading existing road network information uses existing achievements. This type of method occupies a large amount of storage space and requires manual establishment of road network data in advance.

[0005] In the tracking phase, the main issue is to solve the problem of inter-frame association of detected targets. Traditional technologies mainly associate data by constructing motion features, but with the rapid development of neural networks, methods based on deep features have gradually become mainstream. However, based on the characteristics of small size, huge number, and similar features of moving vehicles in satellite-borne videos, deep features alone cannot stably track vehicles between frames.

[0006] Based on the above analysis, there are three types of problems in the existing moving target tracking technology: 1. Most algorithms only consider time domain information or space domain information when constructing prior constraints, and do not fully combine the two, resulting in poor algorithm stability. 2. Most algorithms are poor at handling dim and small moving targets. 3. In the tracking stage, most algorithms rely too much on deep features. That is, the existing algorithms cannot show good applicability when faced with the characteristics of small size, large number, and similar features of moving vehicles in satellite videos. Summary of the invention

[0007] The purpose of the present invention is to provide a satellite-borne video moving vehicle tracking detection method based on spatiotemporal characteristic constraints, so as to solve the problem that the existing moving target tracking technology has poor algorithm stability when constructing constraint conditions, resulting in inaccurate vehicle tracking detection results based on the constructed constraint conditions.

[0008] In order to solve the above technical problems, the present invention provides a space-borne video moving vehicle tracking detection method based on spatiotemporal characteristic constraints, comprising the following steps:

[0009] 1) Extract roads from single-frame images of satellite-borne video to obtain road masks containing spatial information; generate motion heat maps containing temporal information based on satellite-borne video;

[0010] 2) Compare the road mask with the motion heat map, perform conditional discrimination on the connected areas in the motion heat map in turn, combine the connected areas that meet the conditional discrimination requirements with the road mask, and remove the connected areas that do not meet the conditional discrimination requirements from the motion heat map to obtain the combination result of the road mask and the motion heat map, and use the combination result as the constructed constraint area;

[0011] 3) Use the target detection algorithm to detect the constructed constraint area and obtain the detection result of the moving vehicle. According to the detection result, the vehicle association method combining the vehicle relative position information and historical motion information is used to track the vehicle.

[0012] Its beneficial effects are as follows: the method of the present invention improves the robustness of the constructed constraint conditions by fusing the road mask and the motion heat map. In this method, the road mask makes full use of the spatial domain information and can provide a relatively good initial value for the motion heat map, while the motion heat map uses the time domain information to reversely supplement the broken and missed parts of the road mask. At the same time, this method occupies less memory and does not require a large amount of manual labeling. Therefore, the method of the present invention not only extracts the road mask containing spatial information, but also generates a motion heat map containing time information, and fuses the two to generate the constraint conditions of the region of interest, thereby improving the accuracy of moving vehicle detection. And in the present invention, the constructed constraint area is detected by using the target detection algorithm, so as to obtain a more excellent moving vehicle detection effect. Finally, the detected moving vehicles are associated between frames in combination with the improved data association method to complete vehicle tracking.

[0013] Furthermore, in step 1), roads are extracted from single-frame images of satellite-borne video through the D-LinkNet network.

[0014] The D-LinkNet road extraction network fully considers the spatial performance characteristics of roads in single-frame images: narrowness, connectivity, complexity and long span. At the same time, the network uses jump connection mode and hole convolution module to multiply the model receptive field, making it more capable of road extraction. In addition, the D-LinkNet road extraction network uses ResNet34 as an encoder to effectively solve the problem of image feature information loss during the convolution process. Therefore, when the network faces satellite-borne video data, it shows a relatively excellent effect in road extraction. Therefore, the present invention uses the D-LinkNet network to extract roads from single-frame images to achieve the initialization of constraint conditions.

[0015] Furthermore, in step 1), a motion heat map containing time information is cumulatively generated by using an inter-frame difference method.

[0016] Furthermore, the method of accumulating and generating a motion heat map containing time information by using the inter-frame difference method includes: using the inter-frame difference method to detect the video frames, then performing mathematical morphological processing on the detection results of each frame, and finally superimposing and accumulating the processing results to generate a rough motion heat map, and preprocessing the rough motion heat map to obtain a motion heat map containing time information.

[0017] The inter-frame difference method makes full use of the target motion information between two consecutive frames of images, and can effectively supplement the road mask generated using only a single frame of image. Due to the high detection efficiency of the inter-frame difference method, the process of mathematical morphological processing of each frame detection result can eliminate the influence of random noise and pseudo-moving targets to a certain extent.

[0018] Furthermore, the preprocessing includes: removing pseudo-moving target connected domains from connected domains in the rough motion heat map based on area, aspect ratio, perimeter-to-area ratio and roughness.

[0019] In the present invention, the influence of pseudo-moving targets is eliminated to a certain extent through the preprocessing method. Specifically, when the area, aspect ratio, perimeter to area ratio and roughness of the connected domain all meet the corresponding thresholds, it is considered to be in compliance. If not, the connected domain is eliminated, thereby achieving the elimination of pseudo-moving target connected domains in the rough motion heat map.

[0020] Furthermore, in step 2), the compliance judgment requirement is: the connected domain of the motion heat map intersects with the road mask, or the historical aspect ratio and length growth rate of the motion heat map simultaneously meet the corresponding thresholds.

[0021] The quality of the discrimination conditions directly determines the effect and stability of the constraint construction. Therefore, the present invention comprehensively considers the topological relationship between the two and the growth rate of the motion heat map. When the connected domain of the motion heat map intersects with the road mask, or the historical aspect ratio and length growth rate of the motion heat map simultaneously meet one of the corresponding thresholds, the condition discrimination requirements are met.

[0022] Furthermore, the expression that meets the conditional discrimination requirements is:

[0023]

[0024] Among them, I d represents the connected domain of the dth road mask, I i represents the i-th connected domain in the motion heat map, represents the historical aspect ratio of the i-th connected domain in the motion heat map, R represents the growth rate of the length of the connected domain, and T A represents the connected domain aspect ratio threshold, T R represents the growth rate threshold of the length of the connected domain, m is the number of connected domains in the road mask, and n is the number of connected domains in the motion heat map.

[0025] Furthermore, in step 3), the target detection algorithm is a KNN moving target detection algorithm.

[0026] Furthermore, the process of using the KNN target detection algorithm to detect the constructed constrained area includes: using the Lab color space local contrast enhancement algorithm to enhance the contrast of the constructed constrained area, and then using the KNN moving target detection algorithm to detect the constrained area.

[0027] The Lab color space local contrast enhancement algorithm is used to enhance the contrast of the constructed constraint area, thereby highlighting the moving vehicles with similar colors to the road, which can greatly improve the detection integrity of moving vehicles. Therefore, on this basis, the KNN moving target detection algorithm is used to detect the area within the constraint conditions, thereby obtaining a better detection effect.

[0028] Furthermore, in step 3), the vehicle association method combining the relative position information of the vehicle and the historical motion information is as follows: the moving vehicle is defined as a vector with position and velocity using Kalman filtering, the next frame state is predicted from the current frame state of the target using the covariance matrix, and the prediction result is matched with the detection result of the next frame through data association to achieve vehicle tracking.

[0029] In satellite-borne videos, moving vehicles only occupy a few pixels, and their shapes, colors and other features are similar. This results in a large number of mismatches when only using features such as appearance and color to match vehicles between frames. However, in satellite-borne videos, vehicle motion can be approximated as two-dimensional planar motion, so the two-dimensional position information of moving vehicles in satellite-borne videos is more conducive to inter-frame vehicle association. Therefore, based on the detection of moving vehicles, the present invention proposes a vehicle association method that combines vehicle relative position information and historical motion information, which is conducive to inter-frame vehicle association and thus improves the effect of moving vehicle tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of the satellite-borne video moving vehicle tracking detection method based on time-space characteristic constraints of the present invention;

[0031] Figure 2 is a flow chart of a method for automatically acquiring a rough motion heat map of the present invention;

[0032] Figure 3 is a flow chart of the process of fusing the initial road mask and the rough motion heat map of the present invention;

[0033] Figure 4 It is a schematic diagram of the discrimination conditions of the present invention;

[0034] Figure 5 It is a schematic diagram of the angle difference of the movement direction of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0036] Embodiment of the satellite-borne video moving vehicle tracking detection method based on spatiotemporal characteristic constraints:

[0037] The overall process of the method of this embodiment consists of four parts: video data preprocessing, building spatiotemporal constraints, moving vehicle detection, and inter-frame vehicle association. Figure 1 As shown in the figure. For the original satellite video data, firstly, preprocessing such as intra-frame denoising and inter-frame stabilization is performed. Then, the D-LinkNet network is used to extract the road mask containing spatial information in the fixed interval frame, and the inter-frame difference method is used to accumulate and generate the motion heat map containing time information, and the two are fused to generate the constraint conditions of the region of interest, so as to improve the accuracy of moving vehicle detection. In addition, the local contrast enhancement algorithm based on Lab color space is used to optimize KNN moving target detection in the region of interest, so as to obtain a better moving vehicle detection effect. Finally, the detected moving vehicles are associated between frames by combining the improved data association method to complete vehicle tracking.

[0038] Specifically, when constructing the spatiotemporal constraints in this embodiment (i.e., fusing the road mask containing spatial information and the motion heat map containing temporal information to generate the constraints of the region of interest), it is considered that among the existing constraint construction methods, the method based on motion accumulation has poor stability and requires a large number of video frames to accumulate, the method of extracting the road mask by single-frame object-oriented classification is prone to breakage and missed detection, and the method of loading the existing road network has too high memory usage and requires a lot of manual labeling work. Therefore, the constraint construction method of this embodiment improves the robustness of the constructed constraints by fusing the road mask and the motion heat map. In this method, the road mask makes full use of the spatial domain information and can provide a relatively good initial value for the motion heat map, while the motion heat map uses the temporal domain information to reversely supplement the broken and missed detection parts of the road mask. At the same time, this method takes up less memory and does not require a lot of manual labeling.

[0039] In this embodiment, the D-LinkNet network is used to extract roads from a single-frame image to achieve the initialization of constraint conditions, that is, the road mask containing spatial information is extracted by using the D-LinkNet network. Because the D-LinkNet road extraction network fully considers the spatial performance characteristics of roads in single-frame images: narrowness, connectivity, complexity and long span. At the same time, the network uses jump connections and hole convolution modules to multiply the model receptive field, making it more capable of road extraction, and the D-LinkNet road extraction network uses ResNet34 as an encoder to effectively solve the problem of image feature information loss during the convolution process. Therefore, when the network is faced with satellite-borne video data, it shows a relatively excellent effect in road extraction. In this embodiment, the D-LinkNet network is used to extract roads from a single-frame image, and the road mask containing spatial information is obtained more accurately.

[0040] Based on the above-mentioned use of the D-LinkNet network to extract roads from a single-frame image, in order to avoid using only spatial characteristics to extract road masks on a single-frame image, without fully utilizing the advantages of video data in the time domain, resulting in the unavoidable problem of road breakage, missing, etc., this embodiment uses the inter-frame difference method to accumulate and generate a motion heat map to supplement it (i.e., a road mask containing spatial information). The inter-frame difference method makes full use of the target motion information between two consecutive frames of images, and can effectively supplement the road mask generated using only a single-frame image. Figure 2 As shown in the figure, the process of generating a rough motion heat map by using the frame difference method (i.e., the inter-frame difference method) in this embodiment is as follows: first, the video frames are quickly detected using the inter-frame difference method with higher detection efficiency; then, mathematical morphology processing is performed on the detection results of each frame to eliminate the influence of random noise and pseudo-moving targets to a certain extent; finally, the processing results are superimposed and accumulated to generate a rough motion heat map.

[0041] Based on the acquired road mask containing spatial information and the motion heat map containing temporal information, the process of fusing the two to generate the constraint conditions of the region of interest is as follows: Figure 3 As shown in the figure: First, the connected domain of the rough motion heat map is preprocessed based on the area (A), aspect ratio (A r ), perimeter to area ratio (R ca ), surface roughness (S r ) and other parameters to eliminate the influence of pseudo-moving targets to a certain extent. In this embodiment, when the area, aspect ratio, perimeter to area ratio, surface roughness and other parameters of the connected domain meet the threshold, the connected domain meets the conditions, otherwise it does not meet the conditions, and the connected domain that does not meet the conditions is eliminated. The calculation method of the aspect ratio and the surface roughness is shown in formula (1) and formula (2):

[0042] A r = l / w (1)

[0043] S r =C mech / C a (2)

[0044] In formula (1), l represents the length of the minimum circumscribed rectangle of the connected domain, w represents the width of the minimum circumscribed rectangle of the connected domain, and C in formula (2) mech represents the minimum circumscribed convex hull perimeter of the connected domain, C a Represents the perimeter of the connected domain.

[0045] Then, the preprocessed motion heat map is graphically compared with the road mask, and the remaining connected domains in the motion heat map are conditionally judged in turn. If they meet the judgment requirements, they are included in the road mask, otherwise the connected domain is discarded. This embodiment takes into account that the quality of the judgment conditions can directly determine the effect and stability of the constraint condition construction. Therefore, this embodiment comprehensively considers the topological relationship between the two and the growth rate of the motion heat map, and sets the judgment conditions to: ① The connected domain of the motion heat map intersects with the road mask; ② The historical aspect ratio and length growth rate of the motion heat map both meet the corresponding thresholds. If one of them is met, the connected domain is considered to meet the conditions. Among them, condition ① effectively improves the priority of moving targets around the road and retains their cumulative results, while suppressing moving targets far away from the road area to make them enter the judgment condition ②; the length growth rate in condition ② can effectively identify the connected domain formed by noise and moving vehicles. The historical aspect ratio represents the aspect ratio of the connected domain during the motion accumulation process, which can eliminate the impact of the surge in the length of the connected domain due to huge noise in a certain frame during the motion accumulation process. Figure 4 As shown ( Figure 4 The left picture in the middle is a schematic diagram of the tth frame, and the right picture is a schematic diagram of the t+1th frame. The bottom layer is the actual video frame image, the middle layer is the road mask, and the top layer is the motion heat map connected domain area). Connected domains B and C both intersect with the road mask, meeting condition ①. The length growth rate and historical aspect ratio of connected domain A meet the threshold requirements, meeting condition ②, and should be retained. However, connected domain D does not meet both conditions and is selected to be eliminated. The specific judgment condition can be described as formula (3):

[0046]

[0047] Where I d represents the connected domain of the dth road mask, I i represents the i-th connected domain in the motion heat map, represents the historical aspect ratio of the i-th connected domain in the motion heat map, R represents the growth rate of the length of the connected domain, and T A represents the connected domain aspect ratio threshold, T R represents the growth rate threshold of the length of the connected domain, m is the number of connected domains in the road mask, and n is the number of connected domains in the motion heat map.

[0048] Based on the above constraints of this embodiment, when detecting moving vehicles, it is considered that the moving vehicles appear as dim and tiny targets in the satellite video. Directly using the traditional background subtraction method for detection will result in serious missed detection. Therefore, in order to solve the problem that the contrast between the vehicle and the background is small and difficult to distinguish, the KNN moving vehicle detection algorithm is optimized by using the Lab color space local contrast enhancement algorithm in this embodiment.

[0049] Specifically, due to the rich and diverse colors of the background area, the overall contrast enhancement effect of the image will be greatly reduced, and even produce side effects. Therefore, contrast enhancement is first performed within the constructed constraint area: specifically, the average value I of the brightness (L) and color (a, b) feature components in the Lab model is extracted within the constraint area. u =[L u ,a u ,b u ] T , as shown in formula (4):

[0050]

[0051] Then, a Gaussian low-pass filter is applied to the image to obtain the filtered image I g , as shown in formula (5):

[0052]

[0053] Convert the low-pass filtered image to Lab color space and separate the L, a, and b channels to obtain three features I ωhc , as shown in formula (6):

[0054] I ωhc =[L g ,a g ,b g ] T (6)

[0055] The features are fused to obtain a significant value image. This will highlight the moving vehicles with a color similar to the road, which can greatly improve the detection integrity of moving vehicles. On this basis, the KNN moving target detection algorithm is used to detect the area within the constraint conditions, thereby obtaining a better detection effect.

[0056] Based on the above-mentioned moving vehicle detection method of this embodiment, the detection results are obtained to associate vehicles between frames and thus realize the moving vehicle tracking process. Because the moving vehicles in the satellite video only occupy a few pixels, and the shapes, colors and other features are similar, there will be a large number of mismatches when only using the appearance, color and other features to match the vehicles between frames. In the satellite video, the vehicle movement can be approximated as a two-dimensional plane movement, so the two-dimensional position information of the moving vehicle in the satellite video is more conducive to the association of vehicles between frames. Therefore, in this embodiment, based on the detection of moving vehicles, a vehicle association method combining the relative position information of the vehicle and the historical motion information is proposed:

[0057] First, the Kalman filter is used to define the moving vehicle as a vector with position and velocity, and the next frame state, that is, the trajectory position of the moving vehicle, is inferred from the current frame state of the target through the covariance matrix. The Kalman filter prediction equation is as follows:

[0058]

[0059] In the formula represents the state quantity of the target k-th frame; A is the state transfer matrix; P is the state covariance matrix; Q is the process noise covariance matrix. After associating the detection result of the next frame, the target state is updated. The update equation at this time is shown in formula (8):

[0060]

[0061] Where z k is the state value observation vector; H is the state observation matrix; R is the measurement noise covariance matrix; K is the Kalman gain; I is the unit matrix.

[0062] Since the video contains multiple targets, it is necessary to match the prediction results with the detection results of the next frame through data association to achieve vehicle tracking. For the t+1th frame, the moving vehicle detection method of this embodiment can detect m moving targets, and the Kalman filter predicts the target position of the t+1th frame based on the tracking results of the tth frame to obtain n prediction results. In this way, the data association matrix is ​​formed:

[0063]

[0064] Where C is the data association matrix; c mn It represents the matching degree between the mth detection frame and the nth prediction frame. Finally, the Hungarian algorithm is used to find the optimal solution for the data association matrix and complete the inter-frame moving vehicle association.

[0065] Matching degree is a measure of the similarity between vehicles in frames, so the quality of matching degree construction determines the success of vehicle association between frames. In this embodiment, the characteristics of satellite-borne vehicles are fully considered, and the characteristics such as position and motion topology are integrated to propose a matching degree construction method combining intersection-over-union ratio and motion direction angle difference. The intersection-over-union ratio effectively describes the position relationship between the target detection frame and the prediction frame, and the motion direction angle difference makes full use of the target's historical position information to evaluate it. Figure 5 As shown in the figure, the green dots and the red dots represent the positions of the two moving vehicles in multiple frames in the video, the solid line represents its real trajectory, and the dotted line represents the predicted trajectory. Among them, α1 represents the angle between the green target detection frame and the prediction frame, α2 represents the angle between the green target prediction frame and the red target detection frame, and α3 represents the angle between the red target prediction frame and the detection frame. When the intersection and union ratios of the green target prediction frame and the two detection frames are the same, the angle differences α1 and α2 can be used to effectively eliminate the influence of the wrong target. Therefore, the linear combination of the two can more robustly describe the similarity of vehicle matching between frames. The matching degree construction equation can be described as:

[0066] C ij =βC diou (i,j)+εC top (i,j) (10)

[0067] In the formula, C diou is the intersection and union ratio of the two distances. top =cos(α) is the cosine value of the direction of the target prediction frame and the detection frame. Where α is the angle between the detection frame of frame t and the prediction frame relative to the detection frame of frame t+1, as Figure 5 As shown in Figure 2. β and ε are weight coefficients, and their sum is 1.

[0068] In this embodiment, a satellite video moving vehicle tracking detection method based on spatiotemporal characteristic constraints is proposed to solve the problems of complex satellite video background, serious interference of noise and pseudo-moving targets on moving vehicle detection and tracking. Compared with the traditional method, the method of this embodiment can more robustly construct constraint conditions and eliminate background noise interference, improve the recall rate of moving target extraction, and achieve obvious advantages in moving vehicle detection. Figure 5 The results shown can indicate that the method of this embodiment is also relatively stable during the tracking phase.

[0069] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. The patent protection scope of the present invention shall be based on the claims. All equivalent structural changes made using the contents of the description and drawings of the present invention should also be included in the protection scope of the present invention.

Claims

1. A method for tracking and detecting moving vehicles in satellite-borne video based on spatiotemporal characteristic constraints, characterized in that: The steps include: 1) Extract roads from single-frame images of satellite-borne video to obtain road masks containing spatial information; generate motion heat maps containing temporal information based on satellite-borne video; 2) Compare the road mask with the motion heat map, perform conditional discrimination on the connected areas in the motion heat map in turn, combine the connected areas that meet the conditional discrimination requirements with the road mask, and remove the connected areas that do not meet the conditional discrimination requirements from the motion heat map to obtain the combination result of the road mask and the motion heat map, and use the combination result as the constructed constraint area; 3) Use the target detection algorithm to detect the constructed constraint area and obtain the detection result of the moving vehicle. According to the detection result, the vehicle association method combining the vehicle relative position information and historical motion information is used to track the vehicle.

2. The method for tracking and detecting moving vehicles using satellite-borne video based on spatiotemporal characteristic constraints according to claim 1, characterized in that: In step 1), roads are extracted from single-frame images of satellite-borne video through the D-LinkNet network.

3. The method for tracking and detecting moving vehicles using satellite-borne video based on spatiotemporal characteristic constraints according to claim 1, characterized in that: In step 1), a motion heat map containing time information is cumulatively generated by the inter-frame difference method.

4. The method for tracking and detecting moving vehicles using satellite-borne video based on spatiotemporal characteristic constraints according to claim 3, characterized in that: The method for cumulatively generating a motion heat map containing time information by using an inter-frame difference method includes: using an inter-frame difference method to detect video frames, then performing mathematical morphological processing on the detection results of each frame, and finally superimposing and accumulating the processing results to generate a rough motion heat map, and preprocessing the rough motion heat map to obtain a motion heat map containing time information.

5. The method for tracking and detecting moving vehicles in satellite-borne video based on time-space characteristic constraints according to claim 4, characterized in that: The preprocessing includes: removing pseudo-moving target connected domains from connected domains in a rough motion heat map based on area, aspect ratio, perimeter-to-area ratio and roughness.

6. The method for tracking and detecting moving vehicles using satellite-borne video based on spatiotemporal characteristic constraints according to claim 1, characterized in that: In step 2), the compliance judgment requirement is: the connected domain of the motion heat map intersects with the road mask, or the historical aspect ratio and length growth rate of the motion heat map meet the corresponding thresholds at the same time.

7. The method for tracking and detecting moving vehicles using satellite-borne video based on spatiotemporal characteristic constraints according to claim 6, characterized in that: The expression that meets the conditional discrimination requirements is: Among them, I d represents the connected domain of the dth road mask, I i represents the i-th connected domain in the motion heat map, represents the historical aspect ratio of the i-th connected domain in the motion heat map, R represents the growth rate of the length of the connected domain, and T A represents the connected domain aspect ratio threshold, T R represents the growth rate threshold of the length of the connected domain, m is the number of connected domains in the road mask, and n is the number of connected domains in the motion heat map.

8. The method for tracking and detecting moving vehicles in satellite-borne video based on time-space characteristic constraints according to claim 1, characterized in that: In step 3), the target detection algorithm is a KNN moving target detection algorithm.

9. The method for tracking and detecting moving vehicles in satellite-borne video based on time-space characteristic constraints according to claim 8, characterized in that: The process of using the KNN target detection algorithm to detect the constructed constrained area includes: using the Lab color space local contrast enhancement algorithm to enhance the contrast of the constructed constrained area, and then using the KNN moving target detection algorithm to detect the constrained area.

10. The method for tracking and detecting moving vehicles in satellite-borne video based on time-space characteristic constraints according to claim 1, characterized in that: In step 3), the vehicle association method that combines the vehicle relative position information and historical motion information is as follows: the moving vehicle is defined as a vector with position and velocity using Kalman filtering, the next frame state is predicted from the target current frame state using the covariance matrix, and the prediction result is matched with the detection result of the next frame through data association to achieve vehicle tracking.

Images