Moving Target Extraction Method Based on Multi-source Information of Satellite Super-temporal Data

Abstract

The present invention relates to a method for extracting moving targets based on multi-source information of satellite super-phase data, comprising: a. collecting super-phase images of a region and pre-processing the images; b. preliminarily extracting moving targets in the images, and extracting road targets on which the moving targets in the images depend; c. further extracting moving targets in the images based on the preliminarily extracted moving targets and road targets; d. morphologically processing the target result image extracted in step c to obtain the final result. Based on the characteristics of video satellite spectrum, texture, time series, etc., the present invention extracts the spatial geometric shape, movement speed, and road environment on which the moving targets depend, and uses them together for the detection of moving targets, avoiding the influence of some pseudo-moving targets caused by parallax and false moving targets caused by registration errors and random noise, effectively improving the accuracy of moving target detection, and reducing the false detection rate.

Description

Technical Field

[0001] The invention relates to a moving target extraction method based on multi-source information of satellite super-temporal phase data. Background Art

[0002] Moving target detection is a technology formed by the cross-integration of computer vision, remote sensing image processing, artificial intelligence and other fields, and is an important research content of situational awareness. It can not only perceive the existence of things, but also perceive the dynamic change trend of things. This plays an extremely important role in real-time reconnaissance, real-time monitoring, and real-time control in both military and civilian fields. Video satellite is a new type of earth observation technology that can capture continuous images from a moving satellite platform, providing a reliable data source for moving target detection and situational awareness. The characteristics of video satellites such as large-area observation, high spatial resolution, and continuous video imaging enable them to quickly obtain real-time dynamic information on the surface.

[0003] Due to the relative motion between the satellite platform and the ground, the ups and downs of the ground, and other factors, there are deformations such as translation, rotation, distortion, and stretching between different frames of the video. Therefore, targets such as high-rise buildings and towers will show obvious pseudo-motion characteristics. If the moving targets of the video satellite are directly detected using traditional motion target detection algorithms such as inter-frame difference method, background modeling method, and optical flow method, these high-rise targets may be misjudged as moving targets. Moreover, it is difficult to complete the recognition of the pseudo-motion of high-altitude targets caused by the edge of the ground object caused by image translation and parallax changes by simply improving the traditional methods. Summary of the invention

[0004] The object of the present invention is to provide a method for extracting moving targets based on multi-source information of satellite super-temporal data.

[0005] To achieve the above-mentioned object of the invention, the present invention provides a moving target extraction method based on multi-source information of satellite super-temporal data, comprising the following steps:

[0006] a. Collecting a super-phase image of a region and preprocessing the image;

[0007] b. Preliminarily extract the moving target in the image, and extract the road target on which the moving target in the image depends;

[0008] c. Further extraction of moving targets in the image is completed based on the initially extracted moving targets and road targets;

[0009] d. Perform morphological processing on the target result image extracted in step c to obtain the final result.

[0010] According to one aspect of the present invention, in step a, a current frame image is captured and one frame image is captured each within a period of time before and after the current frame to form a three-frame sequence of a previous frame, a current frame and a subsequent frame;

[0011] The time interval between the acquisition of three frames of images is determined by the speed and length of the moving target and the frame rate of the video. The movement amplitude of the moving target between the extracted adjacent frames is between 10m and 100m.

[0012] According to one aspect of the present invention, the preprocessing in step a is to perform inter-frame registration on a frame of image extracted before and after the current frame based on the current frame;

[0013] The inter-frame registration includes reading the current frame image and the previous or subsequent frame image into an array, and using SIFT, SURF, ORB or AKAZE algorithm to perform key point detection and feature description on the two frames of images respectively;

[0014] The matcher is used to perform feature matching on the key points of the two frames. The matching method is to calculate the distance between the descriptors of each pair of key points and return the minimum distance among the k best matches to each key point.

[0015] The homography matrix of the two images is calculated based on the matching point pairs, and the image is deformed for the image before or after the current frame. The RANSAC algorithm is used to remove abnormal point pairs during deformation.

[0016] According to one aspect of the present invention, in step b, the moving target in the image is initially extracted based on the speed attribute and the time series attribute of the moving target respectively;

[0017] When extracting moving targets based on their speed attributes, the optical flow method is used to perform dense optical flow calculations on the current frame and the previous frame to obtain the optical flow state of each pixel. If the speed and direction of the pixel remain unchanged, it is the background, otherwise it is the foreground target.

[0018] When extracting moving targets based on their time series attributes, the three-frame difference method is used in combination with the moving characteristics of the moving targets in the time series to preliminarily divide the foreground targets and background targets.

[0019] The D_LinkNet network based on deep learning is used to extract road targets that are dependent on moving targets in images.

[0020] According to one aspect of the present invention, when extracting a moving target based on the speed attribute of the moving target, a previous frame image and a current frame image for which optical flow calculation is required are sequentially input, and a pyramid is constructed for each image at a specified image ratio;

[0021] Determine the number of pyramid levels, the size of the averaging window, the number of iterations of the algorithm at each level of the image pyramid, the number of neighboring pixels at each pixel for which the polynomial expansion is calculated, the Gaussian standard deviation used for smoothing derivatives, and the initial flow approximation;

[0022] Convert the calculated optical flow from the Cartesian coordinate system to the polar coordinate system to obtain the speed and direction of each pixel;

[0023] According to the speed and direction of the optical flow solution, the pixel whose speed and direction values ​​are both 0 is represented as a background target, otherwise, it is a foreground target.

[0024] According to one aspect of the present invention, when extracting the moving target based on the time series attribute of the moving target in step b, all three frames of the collected images are read, and the images are converted from RGB images to grayscale images;

[0025] Perform inter-frame difference between the grayscale image of the current frame and the grayscale images of the previous and next frames to obtain two difference images;

[0026] A threshold is set to binarize the two difference images respectively, and two binary images are obtained that distinguish the foreground and background objects;

[0027] Perform an AND operation on two binary images and extract the moving target in the intersected image.

[0028] According to one aspect of the present invention, when extracting the road target on which the moving target depends in step b, firstly, a sample data set for network training and testing is constructed to generate a road extraction network;

[0029] Select some remote sensing images from target satellites and mark the road targets and other targets in the images;

[0030] The data set is divided into training set, validation set and test set according to a certain ratio. The training set is used to iteratively train the network parameters, and the validation set is used to verify whether the trained model can achieve the expected accuracy.

[0031] The current frame image is used as a test data set and input into the trained road extraction network to obtain the final road segmentation result.

[0032] According to one aspect of the present invention, the road extraction network is a U-type network based on an encoder-bridge-decoder structure;

[0033] The encoder part of the network is a ResNet34, and the bridge part is five convolutional blocks;

[0034] The decoder part is the inverse operation of the encoder part, and uses upsampling and superimposed with the encoder part of the same level.

[0035] According to one aspect of the present invention, in step c, the three types of results extracted in step b are binary stored as 0 and 1, where 0 represents background target and 1 represents foreground;

[0036] Targets whose values ​​are all 1 after binary storage are extracted as the accurate extraction results of moving targets.

[0037] According to one aspect of the present invention, the morphological processing in step d is to use a 3*3 circular template structure to perform a morphological opening operation on the image extracted by the final target to eliminate spots in the image;

[0038] A 3*3 circular template structure is used to perform morphological closing operation on the image after morphological opening operation to eliminate holes in the image;

[0039] It also includes connectivity analysis of the morphologically processed image and extracting the final moving target according to the following rules:

[0040] When the target size is between 4 pixels and 2000 pixels, the target aspect ratio is less than 8, the ratio of the target area to the minimum circumscribed rectangle area is greater than 0.2, and the target's average pixel value is between 10 and 250, it is determined to be a vehicle target.

[0041] According to the present invention, based on the characteristics of video satellite spectrum, texture, time series, etc., the spatial geometry of the moving target, the moving speed, the road environment on which it depends, and other information are extracted and used together for the detection of the moving target, thereby avoiding the influence of some pseudo-moving targets caused by parallax and false moving targets caused by alignment errors and random noise, effectively improving the accuracy of moving target detection and reducing the false detection rate.

[0042] According to one solution of the present invention, a deep learning algorithm is used to extract road targets that are dependent on moving targets in an image. Thus, objects with pseudo-motion features that are mistaken for moving targets in the initial moving target extraction can be screened out, so as to more accurately extract moving targets of video satellites, thereby solving the defects in the prior art.

[0043] According to one solution of the present invention, the optical flow method and the improved three-frame difference method are used to extract moving targets in an image respectively, and then the extraction results of the two methods are superimposed and analyzed together with the extraction results of road targets, so that the two methods can be corrected with each other to further improve the accuracy of the final moving target extraction.

[0044] According to one solution of the present invention, the collected images are preprocessed before the extraction of moving targets and road targets. The preprocessing step mainly uses the current frame to perform inter-frame registration on the two frames of images before and after the current frame, respectively, so as to avoid the image distortion caused by satellite jitter and other reasons affecting the subsequent target extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart schematically showing a method for extracting moving targets based on multi-source information of satellite super-temporal data according to an embodiment of the present invention;

[0046] Figure 2 A schematic diagram schematically showing the best matching key point pair between the current frame and the previous frame during the image registration process;

[0047] Figure 3 A schematic diagram schematically showing the speed of a moving target obtained by the optical flow method;

[0048] Figure 4 A schematic diagram schematically showing a moving target obtained by an improved three-frame difference method;

[0049] Figure 5 A schematic diagram schematically showing a road target (road) extracted based on a deep learning method;

[0050] Figure 6 Schematic diagram showing the current frame (left) and the final target extraction on the current frame (right). DETAILED DESCRIPTION

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments cannot be described one by one here, but the embodiments of the present invention are not therefore limited to the following embodiments.

[0053] See also Figure 1The present invention provides a moving target extraction method based on multi-source information of satellite super-phase data. First, a super-phase image (or multi-phase image) of a certain area is collected. The super-phase image is image data with a continuous time series (also understood as video data). Then, the collected image is preprocessed by a data preprocessing module (image reading can also be completed by this module). After preprocessing, the moving target in the image can be initially extracted by a target information extraction module. Considering that objects at high places may be misjudged as moving targets, according to the concept of the present invention, road targets that the moving targets in the image depend on are additionally extracted based on the background environment dependency attributes of the moving targets. Taking vehicles and roads as an example, vehicles are moving targets, and roads are road targets that vehicles depend on. The initially extracted moving targets may include vehicles and high-rise buildings at the same time. However, these moving targets are only moving targets that should be extracted when they appear on the road. Therefore, the present invention additionally considers the environmental dependency attributes of the moving targets, and further extracts the image on the basis of the initial extraction, thereby eliminating the defect of the prior art that objects with pseudo-motion features such as high-rise buildings or iron towers are mistakenly judged as moving targets.

[0054] The following Figure 1 The video of Jilin-1-03 satellite in the Atlanta airport area is shown as an embodiment to illustrate the method of the present invention in detail.

[0055] When the present invention collects images, the collected super-phase images include the current frame and one frame of image collected in a period of time before and after the current frame, forming a three-frame sequence of the previous frame, the current frame and the next frame. The time interval of data reading is determined by the speed and length of the target to be extracted, and the frame rate of the video, so as to maximize the guarantee that the moving target does not overlap between adjacent frames. Specifically, for these three frames of images, the collection time interval between adjacent frames should ensure that the movement amplitude of the moving target is between 10m and 100m, so as to avoid the overlap of the moving target between different frames. Specifically, the previous frame, the current frame and the next frame can be read in sequence. After the first two frames of images are initialized, the next frame of image is read as a frame after the current frame. For this embodiment, the frame rate of Jilin No. 1 video satellite is 10 frames / s, the movement speed of ground vehicles is usually 20-160m / s, and the size of ground vehicle targets is usually between 3-15m. Therefore, in this embodiment, 0.5s is set as the time interval of image collection (or image reading), so that most vehicle targets can be detected. There are only a few very long trucks and vehicles with very low speeds, which will produce holes due to the difference, and these can also be eliminated in the post-processing process. The above steps can obtain three frames that are not continuous in time, but the time interval of the selected images is determined by the speed of the moving target and the frame rate of the super-phase data.

[0056] The idea of ​​the image preprocessing step of the present invention is to use the current frame as a reference and successively align (transform) the two frames of images extracted before and after it. This can avoid the image distortion caused by factors such as the satellite's own jitter affecting subsequent target extraction. For details, see Figure 2 , first read the current frame and the previous frame into the array, and use SIFT, SURF, ORB, AKAZE and other algorithms to perform key point detection and feature description on the two frames of images respectively. In this embodiment, the AKAZE algorithm is used, and a total of 644 key points are identified and screened. After the key points of the two frames of images are identified, a matcher is used to perform feature matching on the key points on the two frames of images. The matching method is to calculate the distance between the descriptors of each pair of key points, and return the minimum distance among the k best matches for each key point. Specifically, the distance between the descriptors of each pair of key points is measured, and the k best matching key point pairs with the smallest distance to each key point are obtained. Figure 2 As shown, in this embodiment, a total of 515 best matching key point pairs are returned. Then, the homography matrix (i.e., Homographies) of the transformation of the two images can be calculated based on the matching point pairs, and the image before the current frame can be deformed. In this embodiment, in order to ensure the optimal deformation effect, the RANSAC algorithm is used to remove abnormal point pairs during deformation. After the final deformation, the registered front frame image can be obtained. Then, the current frame and the frame image after it are read into the array, and the registered rear frame image can also be obtained using the same steps as above.

[0057] The above steps complete the inter-frame registration of the image (i.e., preprocessing), which can ensure that the objects in the image will not be distorted and affect the subsequent detection and recognition process. After that, the moving target to be extracted in the image can be preliminarily extracted. For this, the present invention adopts two methods to preliminarily extract the moving target, which are based on the speed attribute and time series attribute of the moving target. When extracting the moving target based on the speed attribute of the moving target, the optical flow method is used to perform dense optical flow solution on the current frame and the previous frame image to obtain the optical flow state of each pixel. The pixel whose speed and direction are unchanged is considered to be the background, otherwise, it is considered to be a foreground target. In this way, the moving speed of the target at the pixel level is obtained, thereby realizing the speed measurement of the target. When extracting the moving target based on the time series attribute of the moving target, the improved three-frame difference method is used, combined with the mobility characteristics of the target in the time series, to perform a preliminary division of the foreground target and the background target.

[0058] When using the optical flow method to extract moving targets, it should be based on the assumption of constant image gradient and constant local optical flow. That is, when the same point changes with time, its brightness will not change. This is the assumption of the basic optical flow method, and all optical flow method variants must meet it to obtain the basic equation of the optical flow method. In addition, it should also be ensured that the change in time will not cause a drastic change in position, so that the grayscale can be partial derivative of the position. In this way, the grayscale change caused by the unit position change between the previous and next frames can be used to approximate the partial derivative of the grayscale to the position. In this embodiment, the Gunnar Farneback algorithm is used to calculate dense optical flow. This method matches all points on the image point by point, calculates the offset of all points, obtains the optical flow field, and then performs registration. In this embodiment, the previous frame and the current frame images that need to be calculated for optical flow are input in sequence, the image ratio is specified to be 0.5, and a (image) pyramid is constructed for each image. The number of layers of the pyramid is determined to be 3, the size of the average window is 12, the number of iterations of the algorithm in each layer of the image pyramid is determined to be 3, and the number of adjacent pixels for calculating the polynomial expansion at each pixel point is determined to be 7. The Gaussian standard deviation for smoothing the derivative is then determined and used as the basis for the polynomial expansion, which is set to 1.5 in this embodiment. In addition, this embodiment uses the input flow as the initial flow approximation, and of course Gaussian filters can also be used in other embodiments.

[0059] After completing the above settings, the calculated optical flow is converted from the Cartesian coordinate system to the polar coordinate system to obtain the speed and direction of the moving target at each pixel. Figure 3 As shown in the figure, the higher the brightness in the optical flow map, the higher its speed. Then a certain threshold can be set to determine the moving target in the optical flow map. For example, if the speed value is 0, 1, 2, 3, ..., then the points greater than or equal to 1 in the optical flow map can be regarded as moving targets, that is, the threshold can be set to 1. In this way, the moving target can be preliminarily extracted according to the speed attribute of the moving target, that is, the target with a speed greater than or equal to 1 can be extracted. Specifically, color can be used to characterize the direction of the moving target. According to the speed and direction solved by the optical flow, the pixel point with a speed and direction value of 0 is represented as a background target, otherwise, it is a foreground target.

[0060] When the three-frame difference method is used to extract the moving target, the three frames of images collected in the image acquisition step are read. These images are converted from RGB images to grayscale images, and the grayscale image of the current frame and the grayscale images of the previous and subsequent frames are subjected to inter-frame difference to obtain two difference images. Subsequently, a threshold can be set, and the difference images of the above two frames are binarized to obtain two binary images. In this embodiment, the threshold is set to 40. The threshold is the grayscale of the image. The higher the value, the brighter it is, and the lower the value, the darker it is. That is, when binarized, the pixel point greater than this threshold is 1, and vice versa. 0. In this way, the foreground (i.e., the bright spot area) and the background (i.e., the black area) can be distinguished, thereby obtaining the moving targets extracted from the current frame and its previous and subsequent frame images respectively. Subsequently, the two binary images need to be subjected to an "and" (i.e., intersection) operation. This operation can be understood as obtaining the intersection of the two binary images (also known as overlay analysis). For example, the points that are simultaneously 1 are moving targets (bright spots), so that the moving targets in the image can be preliminarily extracted from the intersecting binary images, such as Figure 4 shown.

[0061] Through the above steps, the moving target in the image is preliminarily extracted according to the speed attribute and time series attribute of the moving target by using the optical flow method and the three-frame difference method respectively. According to the above concept of the present invention, it is also necessary to extract the road target from the moving target. In this regard, the present invention uses a target (road) recognition method based on deep learning to extract target-dependent environmental elements (i.e., road targets). The extraction of ground moving target-dependent environment mainly refers to the extraction of road targets (vehicles) from high-resolution satellite remote sensing images. The present invention uses a D_LinkNet network to extract road targets. In this regard, it is first necessary to construct a road extraction network. Since the present embodiment uses vehicles as moving targets and roads as road targets. Therefore, the road extraction network is a U-shaped road extraction network based on an encoder-bridge-decoder structure. The encoder part in the network is a ResNet34, the bridge part is five (conventional) convolution blocks, and the decoder part is the inverse operation of the encoder part. In addition, the decoder part adopts upsampling and superimposition with the encoder part of the same level to achieve feature fusion of different spatial scales. When constructing a road extraction network, firstly, a sample data set for network training and testing is used to generate a road extraction network. Specifically, it is necessary to select some representative remote sensing images of some target satellites in a targeted manner. Among them, representative data can be selected by grayscale attributes, such as the typical grayscale values ​​of road targets at different times in historical data. The data set constructed using these images can ensure that the trained model can identify various types of road targets at all times. Of course, other spatial, spectral and other attributes of the target object can also be used. In short, it can reflect the characteristics of the target object to facilitate model recognition. After selecting representative images, it is necessary to annotate the road targets and other targets therein (data set), and divide the data set into training set, validation set and test set according to a certain ratio. The training set is used to iteratively train the network parameters, and the validation set is used to verify whether the trained model can achieve the desired accuracy. The training set is input into the road extraction network, and the network parameters are iteratively trained until the network reaches the ideal loss and accuracy on the training set and validation set. Finally, the image frame of the video satellite (i.e., the current frame) is used as the test data set, input into the trained network, and the final road segmentation result is output, i.e., the road map in the image corresponding to the current frame (i.e., the extraction result), as shown in Figure 5 shown.

[0062] Therefore, according to the above steps, the moving targets extracted by the two methods and the road targets obtained by the deep learning algorithm can be obtained. Subsequently, the three extracted targets should be overlaid and analyzed, that is, the extraction results of the three methods are used to jointly complete the final extraction of the moving targets. Specifically, the three types of results, the moving targets extracted by the optical flow method and the three-frame difference method and the extraction results of the road targets extracted based on deep learning (Figure), are all stored as 0 and 1 to form a binary image. When storing, the moving targets are also stored as 1 (i.e., bright spot), that is, 0 represents background targets and 1 represents foreground. In this way, in the current frame image, the targets whose values ​​are all 1 after binary storage are extracted, and the accurate (further) extraction results of the moving targets obtained by fusing multi-source information attributes can be obtained.

[0063] Therefore, the present invention essentially uses the optical flow method and the three-frame difference method to extract the moving target from the image, that is, it combines the target speed attribute obtained by the optical flow method and the foreground target obtained by the three-frame difference method. These two methods have their own advantages and disadvantages, and combining the two methods can make the two correct each other, and combined with the consideration of the environmental attributes that the moving target depends on, the most accurate moving target extraction result can be obtained. Of course, although this method has basically completed the final extraction of the moving target, its extraction result is still only an initial result, and still needs to be improved through post-processing steps.

[0064] Specifically, Figure 6 As shown, the data post-processing module is used to perform morphological processing on the image after the target information extraction, so as to eliminate the influence of small spots and small holes (i.e., random noise, image registration error) in the image on the integrity of the moving target (vehicle) and obtain the final result. Specifically, a 3*3 circular template structure is used to perform morphological opening operation on the image to eliminate small isolated points (i.e., spots). Then, a 3*3 circular template structure is used to perform morphological closing operation on the image after the morphological opening operation to remove the influence of small holes on the integrity of the vehicle target. It is also necessary to perform connectivity analysis on the results and extract the desired moving targets according to certain rules. The rules can be formulated according to the size of the target, the aspect ratio, the area ratio of the target to the minimum enclosing rectangle, etc. For example, for vehicle targets, only those moving targets with a size (dimension) > 4 pixels and < 2000 pixels at the same time, the aspect ratio of the target ≤ 8, the ratio of the target area to the minimum enclosing rectangle area > 0.2, and the average pixel value of the target > 10, or < 250, will be considered as vehicle targets. After the above morphological analysis and connectivity analysis, the result image can be saved to obtain the current frame image marked with the final moving target extraction result.

[0065] In summary, the present invention is directed to the moving target detection technology based on super-phase data. On the basis of traditional inter-frame difference and optical flow method, it integrates the inter-frame registration of super-phase data, optical flow solution, three-frame difference, target-dependent environment extraction based on deep learning, and post-processing processes, so as to comprehensively utilize the speed attribute, time series attribute, spectral attribute and spatial position attribute of the target, that is, to perceive the multi-dimensional attributes of the target from different angles, and form a complete set of moving target detection methods for satellite super-phase data. Compared with traditional moving target detection methods such as inter-frame difference, optical flow method, background modeling method, etc., it is more systematic, with higher extraction accuracy, and thus can perform more accurate moving target detection, which is more practical in the application of satellite super-phase data. Thereby solving the problem of low detection accuracy and high false detection rate generated when the current moving target detection algorithm is applied to satellite video data detection, and realizing more accurate detection of moving targets. In summary, the present invention can be used for research on detection and tracking of moving targets (ground moving vehicle targets) in space-based satellite super-phase data. It has very good versatility and high extraction accuracy in the extraction of moving targets from satellite super-phase data. It is of great significance for conducting research on vehicle flow analysis based on video satellites, positioning and tracking of ground moving targets, etc., and can be widely used in intelligent transportation, smart cities, emergency rescue and other fields.

[0066] The above is only one embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A moving target extraction method based on multi-source information of satellite super-temporal data, comprising the following steps: a. Collecting a super-phase image of a region and preprocessing the image; b. Preliminarily extract the moving target in the image, and extract the road target on which the moving target in the image depends; c. Based on the initially extracted moving targets and road targets, overlay analysis is performed to jointly complete further extraction of moving targets in the image; d. Perform morphological processing on the target result image extracted in step c to obtain the final result; In the step b, the moving target in the image is initially extracted based on the speed attribute and the time series attribute of the moving target; When extracting moving targets based on their speed attributes, the optical flow method is used to perform dense optical flow calculations on the current frame and the previous frame to obtain the optical flow state of each pixel. If the speed and direction of the pixel remain unchanged, it is the background, otherwise it is the foreground target. When extracting moving targets based on their time series attributes, the three-frame difference method is used in combination with the moving characteristics of the moving targets in the time series to preliminarily divide the foreground targets and background targets. The D_LinkNet network based on deep learning is used to extract road targets that are dependent on moving targets in images; When extracting a moving target based on its speed attribute, the previous frame image and the current frame image for which optical flow calculation is required are input in sequence, and a pyramid is constructed for each image at a specified image ratio; Determine the number of pyramid levels, the size of the averaging window, the number of iterations of the algorithm at each level of the image pyramid, the number of neighboring pixels at each pixel for which the polynomial expansion is calculated, the Gaussian standard deviation used for smoothing derivatives, and the initial flow approximation; Convert the calculated optical flow from the Cartesian coordinate system to the polar coordinate system to obtain the speed and direction of each pixel; According to the speed and direction of the optical flow solution, the pixel whose speed and direction values ​​are both 0 is represented as a background target, otherwise, it is a foreground target; The morphological processing in step d is to use a 3*3 circular template structure to perform a morphological opening operation on the image extracted by the final target to eliminate spots in the image; A 3*3 circular template structure is used to perform morphological closing operation on the image after morphological opening operation to eliminate holes in the image; It also includes connectivity analysis of the morphologically processed image and extracting the final moving target according to the following rules: When the target size is between 4 pixels and 2000 pixels, the target aspect ratio is less than 8, the ratio of the target area to the minimum circumscribed rectangle area is greater than 0.2, and the target's average pixel value is between 10 and 250, it is determined to be a vehicle target.

2. The method according to claim 1, characterized in that In step a, the current frame image is captured and one frame image is captured each within a period of time before and after the current frame to form a three-frame sequence of a previous frame, a current frame and a subsequent frame; The time interval between the acquisition of three frames of images is determined by the speed and length of the moving target and the frame rate of the video. The movement amplitude of the moving target between the extracted adjacent frames is between 10m and 100m.

3. The method according to claim 2, characterized in that The preprocessing in step a is to perform inter-frame registration on the image frames extracted before and after the current frame based on the current frame; The inter-frame registration includes reading the current frame image and the previous or subsequent frame image into an array, and using SIFT, SURF, ORB or AKAZE algorithm to perform key point detection and feature description on the two frames of images respectively; The matcher is used to perform feature matching on the key points of the two frames. The matching method is to calculate the distance between the descriptors of each pair of key points and return the minimum distance among the k best matches to each key point. The homography matrix of the two images is calculated based on the matching point pairs, and the image is deformed for the image before or after the current frame. The RANSAC algorithm is used to remove abnormal point pairs during deformation.

4. The method according to claim 1, characterized in that When extracting the moving target based on the time series attribute of the moving target in step b, all three frames of the collected images are read, and the images are converted from RGB images to grayscale images; Perform inter-frame difference between the grayscale image of the current frame and the grayscale images of the previous and next frames to obtain two difference images; A threshold is set to binarize the two difference images respectively, and two binary images are obtained that distinguish the foreground and background objects; Perform an AND operation on two binary images and extract the moving target in the intersected image.

5. The method according to claim 1 or 4, characterized in that: When extracting the road target that the moving target depends on in step b, firstly construct a sample data set for network training and testing to generate a road extraction network; Select some remote sensing images from target satellites and mark the road targets and other targets in the images; The data set is divided into training set, validation set and test set according to a certain ratio. The training set is used to iteratively train the network parameters, and the validation set is used to verify whether the trained model can achieve the expected accuracy. The current frame image is used as a test data set and input into the trained road extraction network to obtain the final road segmentation result.

6. The method according to claim 5, characterized in that The road extraction network is a U-type network based on an encoder-bridge-decoder structure; The encoder part of the network is a ResNet34, and the bridge part is five convolutional blocks; The decoder part is the inverse operation of the encoder part, and uses upsampling and superimposed with the encoder part of the same level.

7. The method according to claim 6, characterized in that In the step c, the three types of results extracted in the step b are binary stored as 0 and 1, respectively, where 0 represents the background target and 1 represents the foreground; Targets whose values ​​are all 1 after binary storage are extracted as the accurate extraction results of moving targets.

Images