Video moving object detection method and system based on scan-gaze network

Abstract

The application discloses a video moving target detection method and system based on a scan-gaze network, is applied to the scan-gaze network, relates to the field of computer vision, and comprises the following steps: performing space-time modeling on a target video segment composed of a plurality of continuous satellite video images, performing feature extraction on a plurality of target probability response graphs obtained, and obtaining a plurality of dense feature graphs; calculating corresponding dynamic thresholds according to the mean and standard deviation of each graph in the target probability response graph, screening sampling points in the probability response graph according to the dynamic thresholds, and generating a sparse coordinate set according to the coordinates of obtained candidate sampling points; converting the dense feature graph into a sparse feature graph based on the sparse coordinate set, and enhancing the sparse feature graph; and detecting a moving target in the target video segment based on the enhanced sparse feature graph according to a preset sparse detection head, so as to obtain a detection result corresponding to the moving target. Therefore, the target detection precision can be improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and in particular to a video moving target detection method and system based on a scanning-staring network. Background Technology

[0002] Satellite video moving target detection is an important task in intelligent remote sensing interpretation and is widely used in scenarios such as urban traffic monitoring, maritime search and rescue, and disaster emergency response.

[0003] Existing methods for moving target detection in satellite video are mainly divided into spatial modeling-based methods and spatiotemporal modeling-based methods. However, current methods suffer from high computational redundancy, insufficient extraction of motion cues for weak targets, and difficulty in balancing detection accuracy and inference efficiency. Real moving targets in satellite video are highly sparse in space, but traditional methods still perform intensive calculations on all pixels or feature points across the entire image, wasting significant computational power on background areas, which is insufficient for real-time detection and on-board edge deployment requirements. Moving targets in satellite video typically exhibit continuous trajectories over short timeframes, while random noise and background clutter lack stable motion consistency. Without explicitly utilizing these short-term motion constraints, models are prone to misclassifying background textures as targets or missing low-contrast real targets. High-precision spatiotemporal models typically have a large number of parameters and slow inference speed; while lightweight models are faster, they lack the ability to identify weak targets in complex backgrounds. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a video moving target detection method and system based on a scan-staring network, which can reduce background redundancy calculations and enhance the features of weak moving targets, thereby improving detection accuracy. The specific solution is as follows: In a first aspect, this application discloses a video moving target detection method based on a scan-stare network, applied to a scan-stare network, wherein the scan-stare network is a pre-defined target detection network, comprising: The target video segment is composed of several consecutive satellite video images, and the target video segment is spatiotemporally modeled. Feature extraction is performed on several target probability response maps obtained by modeling to obtain several dense feature maps corresponding to the several target probability response maps. Calculate the mean and standard deviation of each graph in the plurality of target probability response graphs, and calculate the dynamic threshold corresponding to the plurality of target probability response graphs based on the mean and standard deviation. Then, filter the sampling points in the plurality of target probability response graphs based on the dynamic threshold, so as to generate a sparse coordinate set based on the coordinates of the candidate sampling points obtained after filtering. Based on the sparse coordinate set, the dense feature maps are converted into sparse feature maps, and the sparse feature maps are enhanced to obtain enhanced sparse feature maps. The preset sparse detection head detects moving targets in the target video segment based on the several enhanced sparse feature maps, so as to obtain the detection results corresponding to the moving targets.

[0005] Optionally, the step involves assembling a target video segment based on several acquired consecutive satellite video images, and performing spatiotemporal modeling on the target video segment to extract features from several target probability response maps obtained through modeling, thereby obtaining several dense feature maps corresponding to the several target probability response maps, including: Several consecutive satellite video images were acquired and normalized to generate a target video segment based on the obtained normalized images. The target video segment is input into a preset scanning component to perform spatiotemporal modeling on each frame of the target video segment, thereby obtaining several corresponding target probability response maps; the preset scanning component is a preset convolutional neural network. The preset scanning component performs coarse-grained feature extraction on the probability response maps of the targets to obtain a number of dense feature maps corresponding to the probability response maps of the targets.

[0006] Optionally, the step of calculating the mean and standard deviation of each graph in the plurality of target probability response graphs, calculating the dynamic threshold corresponding to the plurality of target probability response graphs based on the mean and standard deviation, and then filtering the sampling points in the plurality of target probability response graphs based on the dynamic threshold to generate a sparse coordinate set based on the coordinates of the candidate sampling points obtained after filtering includes: Calculate the mean and standard deviation of each graph in the plurality of target probability response graphs, and calculate the product of the standard deviation and the preset intensity coefficient to obtain the target product; The sum of the target product and the mean is used as the dynamic threshold corresponding to each graph in the plurality of target probability response graphs; The sampling points in the plurality of target probability response maps are filtered according to the dynamic threshold, so as to remove the sampling points whose probability values ​​in the plurality of target probability response maps are less than the dynamic threshold, and to obtain candidate sampling points. A sparse coordinate set is generated based on the coordinates of the candidate sampling points.

[0007] Optionally, the step of converting the dense feature maps into sparse feature maps based on the sparse coordinate set, and enhancing the sparse feature maps to obtain enhanced sparse feature maps, includes: Based on the coordinates in the sparse coordinate set, the dense feature maps are sampled to convert the dense feature maps into sparse feature maps. The sparse feature maps are convolved to obtain processed sparse feature maps, and the cosine similarity between the processed sparse feature maps and their adjacent processed sparse feature maps is calculated; the adjacent processed sparse feature maps are the feature maps that are adjacent to the processed sparse feature maps before and after the processed sparse feature maps. The similarity score between the processed sparse feature map and its adjacent processed sparse feature maps is calculated based on the cosine similarity. Calculate the product between the similarity scores to obtain the target product, and take the product with the largest value in the target product as the maximum response, and take the maximum response as the motion consistency score of each coordinate of the processed sparse feature map; The target sparse feature maps are enhanced based on the motion consistency score to obtain several enhanced sparse feature maps; the target sparse feature map is any one of the several sparse feature maps.

[0008] Optionally, the step of performing convolution processing on the plurality of sparse feature maps to obtain processed sparse feature maps, and calculating the cosine similarity between the processed sparse feature maps and their adjacent processed sparse feature maps, includes: Two-dimensional submanifold sparse convolution is performed on the target sparse feature map and the adjacent sparse feature map corresponding to the target sparse feature map to obtain the processed sparse feature map and the processed adjacent sparse feature map. Calculate the cosine similarity between the central feature corresponding to the processed sparse feature map and the central feature corresponding to the adjacent processed sparse feature maps.

[0009] Optionally, calculating the similarity score between the processed sparse feature map and its neighboring processed sparse feature maps based on the cosine similarity includes: The similarity score between the processed sparse feature map and the processed adjacent sparse feature map is calculated based on the cosine similarity and the positional features corresponding to each coordinate of the processed adjacent sparse feature map.

[0010] Optionally, the enhancement of each target sparse feature map based on the motion consistency score to obtain several enhanced sparse feature maps includes: Calculate the L1 norm of each sampling point in the target sparse feature map to obtain several norm values, and take the average of the several norm values ​​as the average intensity of the target sparse feature map. The gain factor of the target sparse feature map is calculated based on the average intensity and the L1 norm of each sampling point, and the target sparse feature map is enhanced based on the gain factor and the motion consistency score to obtain several enhanced sparse feature maps.

[0011] Optionally, the step of detecting moving targets in the target video segment based on the several enhanced sparse feature maps using a preset sparse detection head to obtain detection results corresponding to the moving targets includes: The preset sparse detection head detects moving targets in the target video segment based on the several enhanced sparse feature maps, and outputs the predicted target center point, target width and height, and center offset corresponding to the moving target. The detection result of the moving target is generated based on the predicted target center point, the target width and height, and the center offset.

[0012] Optionally, the video moving target detection method based on scan-stare networks further includes: The target loss generated by moving target detection on the target video segment is determined, and the scan-stare network is trained end-to-end based on the target loss and a preset joint loss function, so that the updated detection network is used as the current scan-stare network.

[0013] Secondly, this application discloses a video moving target detection system based on a scan-stare network, applied to a scan-stare network, wherein the scan-stare network is a preset target detection network, comprising: Management server, terminal; The terminal is configured to compose a target video segment based on several acquired continuous satellite video images, and perform spatiotemporal modeling on the target video segment to extract features from several target probability response maps obtained from the modeling, thereby obtaining several dense feature maps corresponding to the several target probability response maps; calculate the mean and standard deviation of each map in the several target probability response maps, and calculate the dynamic threshold corresponding to the several target probability response maps based on the mean and standard deviation, then filter the sampling points in the several target probability response maps based on the dynamic threshold, and generate a sparse coordinate set based on the coordinates of the candidate sampling points obtained after filtering; convert the several dense feature maps into several sparse feature maps based on the sparse coordinate set, and enhance the several sparse feature maps to obtain several enhanced sparse feature maps; and detect moving targets in the target video segment based on the several enhanced sparse feature maps using a preset sparse detection head to obtain detection results corresponding to the moving targets.

[0014] Thirdly, this application discloses a video moving target detection device based on a scan-stare network, applied to a scan-stare network, comprising: The feature extraction module is used to compose a target video segment based on several consecutive satellite video images, and to perform spatiotemporal modeling on the target video segment in order to extract features from several target probability response maps obtained by modeling, and to obtain several dense feature maps corresponding to the several target probability response maps. The coordinate filtering module is used to calculate the mean and standard deviation of each graph in the plurality of target probability response graphs, and calculate the dynamic threshold corresponding to the plurality of target probability response graphs based on the mean and standard deviation. Then, the sampling points in the plurality of target probability response graphs are filtered based on the dynamic threshold to generate a sparse coordinate set based on the coordinates of the candidate sampling points obtained after filtering. The feature enhancement module is used to convert the several dense feature maps into several sparse feature maps based on the sparse coordinate set, and to enhance the several sparse feature maps to obtain several enhanced sparse feature maps. The target detection module is used to detect moving targets in the target video segment based on the several enhanced sparse feature maps using a preset sparse detection head, so as to obtain the detection result corresponding to the moving target.

[0015] Fourthly, this application discloses an electronic device, comprising: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the video moving target detection method based on the scan-stare network as described above.

[0016] Fifthly, this application discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the video moving target detection method based on a scan-stare network as described above.

[0017] In this application, a target video segment can be composed of several consecutive satellite video images, and spatiotemporal modeling can be performed on the target video segment to extract features from several target probability response maps obtained from the modeling, resulting in several dense feature maps corresponding to the several target probability response maps; the mean and standard deviation of each map in the several target probability response maps are calculated, and a dynamic threshold corresponding to the several target probability response maps is calculated based on the mean and standard deviation; then, the sampling points in the several target probability response maps are filtered based on the dynamic threshold to generate a sparse coordinate set based on the coordinates of the candidate sampling points obtained after filtering; the several dense feature maps are converted into several sparse feature maps based on the sparse coordinate set, and the several sparse feature maps are enhanced to obtain several enhanced sparse feature maps; a preset sparse detection head is used to detect moving targets in the target video segment based on the several enhanced sparse feature maps to obtain detection results corresponding to the moving targets.

[0018] Therefore, the method of this application can perform spatiotemporal modeling on a target video segment composed of several consecutive satellite video images, extract features from the resulting target probability response maps, and obtain several dense feature maps corresponding to the target probability response maps. A corresponding dynamic threshold is calculated based on the mean and standard deviation of each map in the target probability response map, and sampling points in the probability response map are filtered according to the dynamic threshold to generate a sparse coordinate set based on the coordinates of the obtained candidate sampling points. The dense feature maps are then converted into sparse feature maps based on the sparse coordinate set, and the sparse feature maps are enhanced. Moving targets in the target video segment are detected based on the enhanced sparse feature maps using a preset sparse detection head, to obtain detection results corresponding to the moving targets. In this way, on the one hand, dense features can be converted into sparse candidate features during the scanning stage, ensuring that subsequent high-cost feature extraction only occurs in potential target areas, reducing computational overhead from the source; on the other hand, short-term motion consistency between consecutive frames can be utilized to adaptively enhance targets that meet motion constraints but have weak feature responses; furthermore, it can determine whether candidate points conform to the short-term trajectory patterns of real moving targets, thereby suppressing non-target areas that do not meet motion consistency. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] Figure 1(a) is a schematic diagram of the first part of the architecture of a scanning-staring network disclosed in this application; Figure 1(b) is a schematic diagram of the second part of the architecture of a scanning-staring network disclosed in this application; Figure 1(c) is a schematic diagram of the third part of the architecture of a scanning-staring network disclosed in this application; Figure 2 This is a schematic diagram of a peripheral scanning-foveal gaze mechanism disclosed in this application; Figure 3 This is a flowchart of a video moving target detection method based on a scan-staring network disclosed in this application; Figure 4(a) shows a comparison of the image enhancement before and after of the first dataset disclosed in this application; Figure 4(b) shows a comparison of the image enhancement before and after of the second dataset disclosed in this application; Figure 5 This is a visualization of the output of a scanning-gazing network at different stages as disclosed in this application; Figure 6 This is a visualization of the effect of a different method disclosed in this application; Figure 7(a) is a schematic diagram of the processing flow of the first existing spatiotemporal detection method disclosed in this application; Figure 7(b) is a schematic diagram of the processing flow of the second existing spatiotemporal detection method disclosed in this application; Figure 7(c) is a schematic diagram of the scanning-gazing method processing flow disclosed in this application; Figure 8 This is a schematic diagram of a video moving target detection device based on a scanning-staring network disclosed in this application; Figure 9 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Current satellite video moving target detection methods suffer from high computational redundancy, insufficient mining of motion cues for weak targets, and difficulty in balancing detection accuracy and inference efficiency.

[0023] To overcome the aforementioned technical problems, this application discloses a video moving target detection method and system based on a scanning-staring network, which can reduce background redundancy calculations and enhance the features of weak moving targets, thereby improving detection accuracy.

[0024] The video moving target detection method based on a scan-gaze network in this application applies a scan-gaze network, and the architecture of the scan-gaze network is shown in Figures 1(a), 1(b), and 1(c). Figure 1(a) shows SGNet (StepwiseGoal-Driven Networks for Trajectory Prediction), Figure 1(b) shows the ACS (Adaptive Coordinate Sampling) module, and Figure 1(c) shows the MFE (Mirror Feature Enhancement) module. It should be noted that the scan-gaze network consists of an input module, a scan module, an ACS module, a gaze module, an MFE module, a sparse detection head, and an output module. Furthermore, this method is inspired by the "peripheral scan-foveal gaze" collaborative mechanism in the human visual system, such as... Figure 2 As shown, the satellite video moving target detection process is divided into a coarse-grained scanning stage and a fine-grained staring stage.

[0025] The scanning phase corresponds to the rapid screening function, used to quickly locate potential target regions within a large field of view. This phase uses a scanning module to perform lightweight spatiotemporal feature extraction on consecutive input frames, obtaining a target probability response map for each frame. Furthermore, through an adaptive coordinate sampling mechanism, namely the ACS mechanism, positions with probability responses higher than the dynamic threshold are retained to form a sparse candidate coordinate set.

[0026] The staring phase corresponds to the fine-grained recognition function, used for high-resolution spatiotemporal feature extraction and fine discrimination of the sparse candidate regions retained in the scanning phase. This phase employs a staring module, performing sub-manifold sparse convolution and sparse 3D (Three Dimensional) convolution processing on sparse coordinates, and introducing a mirror feature enhancement module, i.e., the MFE module. The MFE module utilizes the short-term mirror correspondence between real moving targets in local windows of adjacent frames to calculate the motion consistency score of candidate positions, and combines an adaptive gain mechanism to enhance the response of weak targets. Finally, the enhanced sparse features are input into the sparse detection head, outputting the moving target bounding box for each frame.

[0027] It should be noted that the problem modeling for the video moving target detection method based on scanning-staring networks in this application is as follows: Let the input be a satellite video clip. Depend on Frames of continuous images consist of: .in, Indicates the first Frame image. For each frame... The target detection result set needs to be output: .in, Indicates the first The first frame A detection box for a moving target. Indicates the first The number of targets detected in a frame. Each detection box can be represented as: .in, Indicates the coordinates of the target center. and These represent the target width and height, respectively. This indicates the confidence level of the detection.

[0028] The core objective of this invention is to extract original dense video features while ensuring high recall and high precision. Convert to sparse features Only the coordinates that may contain the target are subsequently modeled using high-cost spatiotemporal methods. This process can be abstractly represented as: ; in, Indicates the first Dense features of frames Indicates the first Frame target probability response map This represents a dense-to-sparse transformation operation implemented by an adaptive coordinate sampling mechanism. This represents the sparse features obtained after the transformation.

[0029] See Figure 3 As shown, this embodiment of the invention discloses a video moving target detection method based on a scan-stare network, applied to a scan-stare network, including: Step S11: Based on the acquired continuous satellite video images, a target video segment is formed, and the target video segment is spatiotemporally modeled. Feature extraction is performed on the modeled target probability response maps to obtain a number of dense feature maps corresponding to the target probability response maps.

[0030] In this embodiment, a target video segment is first composed of several acquired continuous satellite video images. Then, spatiotemporal modeling is performed on these images to extract features from the resulting target probability response maps, yielding several dense feature maps corresponding to the target probability response maps. Specifically, the input module needs to normalize the acquired continuous satellite video images to generate the target video segment based on the normalized images. The input is a continuous... T A frame of satellite video image. Taking a real-world example, TA value of 10 is acceptable. If the actual scenario has higher speed requirements, a value of 3 or 5 can be used. If a longer temporal context is needed, a value of 15 can be used, but excessively long frame counts may be affected by satellite platform motion and background displacement, leading to weakened temporal consistency. The input image can be normalized, cropped, scaled, or segmented according to the network requirements.

[0031] Furthermore, the target video segment needs to be input into a preset scanning component. This component performs spatiotemporal modeling on each frame of the target video segment to obtain several corresponding target probability response maps. It should be noted that the preset scanning component is a scanning module used for coarse-grained target region selection. Its input is continuous... The output is a target probability response map for each frame of satellite video image. The scanning module can be implemented using a lightweight 3DU-Net architecture, including several layers of 3D convolutions, batch normalization, and ReLU (Rectified Linear Unit) activation operations to simultaneously extract spatial appearance features and short-term motion features. It can be represented as a spatiotemporal feature extraction network based on 3D convolutions. The basic unit of the scanning module can be represented as: ; in, Represents 3D convolution. Indicates batch normalization, This represents the activation function.

[0032] To adapt to the irregular shapes of small targets and the inconsistent length and width of vehicle or ship targets in satellite video, this invention uses an anisotropic Gaussian heatmap (AGH) as the supervision signal for the scanning module, instead of a simple binary mask. For the target center point... Construct the following heat map: ; in, Indicates position The monitoring response value at the location, Represents the center coordinates of the actual target bounding box. and These represent the standard deviations in the horizontal and vertical directions, respectively. Furthermore, they can be adaptively set according to the target bounding box's width and height. ; in, and These represent the width and height of the actual target bounding box, respectively. This setting follows the 3-σ rule, ensuring that the heatmap response range matches the actual shape of the target. This helps preserve the target's rough shape and neighborhood response, improving the coverage of small targets during the scanning phase.

[0033] Finally, coarse-grained feature extraction is performed on several target probability response maps using a preset scanning component to obtain several dense feature maps corresponding to the target probability response maps. Specifically, a lightweight 3D U-Net structure is used to extract coarse-grained spatiotemporal features to obtain the target probability response map for each frame. .

[0034] Step S12: Calculate the mean and standard deviation of each graph in the probability response graph of several targets, and calculate the dynamic threshold corresponding to the probability response graph of several targets based on the mean and standard deviation. Then, filter the sampling points in the probability response graph of several targets based on the dynamic threshold, so as to generate a sparse coordinate set based on the coordinates of the candidate sampling points obtained after filtering.

[0035] In this embodiment, firstly, the dynamic threshold of each graph in several target probability response maps needs to be calculated. Then, the sampling points in the target probability response maps are filtered according to the dynamic thresholds, ultimately generating a sparse coordinate set. Specifically, the mean and standard deviation of each graph in several target probability response maps need to be calculated, and the product of the standard deviation and a preset intensity coefficient is calculated to obtain the target product. Then, the sum of the target product and the mean is used as the dynamic threshold of each graph in several target probability response maps. Specifically, the ACS mechanism adaptively determines the sampling threshold based on the response distribution of each frame. Let... The mean is The standard deviation is The dynamic threshold is ,in, For the first Adaptive sampling threshold for frames, An intensity coefficient used to control sparsity. A larger one... A higher threshold allows for the removal of more background areas, resulting in sparser candidate points; a smaller threshold... It will retain more candidate regions, which is beneficial for improving recall, but it will increase the amount of computation.

[0036] Furthermore, it is necessary to filter the sampling points in several target probability response maps based on a dynamic threshold, removing sampling points whose probability values ​​are less than the dynamic threshold to obtain candidate sampling points. Then, a sparse coordinate set is generated based on the coordinates of the candidate sampling points. It should be noted that, based on the dynamic threshold, the map needs to retain... The location of the sampling points in the target probability response map is used to eliminate them, and a sparse coordinate set is created. Defined as: For example, in a specific situation, when β When the value is 3, the sampling rate can typically be reduced to below 1%. This allows large areas of background to be removed from subsequent calculations, retaining only a very small number of potential target areas.

[0037] Step S13: Based on the sparse coordinate set, convert the several dense feature maps into several sparse feature maps, and enhance the several sparse feature maps to obtain several enhanced sparse feature maps.

[0038] In this embodiment, it is necessary to convert dense feature maps into sparse feature maps and then enhance the sparse feature maps to obtain enhanced sparse feature maps. Specifically, firstly, it is necessary to sample several dense feature maps based on coordinates in the sparse coordinate set to convert several dense feature maps into several sparse feature maps. It should be noted that the output sparse features of the scanning phase can be written as... The dense features output by the scanning module need to be sorted according to a sparse coordinate set. The features are converted into sparse features and input into the gaze module. The gaze module is used for fine-grained spatiotemporal modeling of the sparse candidate features retained during the scanning phase, and an MFE module is inserted into the multi-layer features of the gaze module. Unlike the fast coarse screening of the scanning module, the gaze module has stronger feature representation capabilities, but its computation only occurs near the sparse candidate locations, thus avoiding the large amount of redundant computation on the background region of the entire image as in traditional dense networks. Furthermore, the gaze module preferably adopts a sparse 3D U-Net structure. Its convolutional operations include submanifold sparse convolution and sparse 3D convolution. Submanifold sparse convolution only updates features at existing active coordinate points and does not diffuse active points to blank background regions, thus maintaining the stability of the sparse structure. Sparse 3D convolution is used to aggregate motion context information within the spatiotemporal neighborhood. In this way, the gaze module can obtain strong local spatiotemporal representation capabilities with limited computational resources.

[0039] Furthermore, a two-dimensional submanifold sparse convolution process needs to be performed on the target sparse feature map and its corresponding adjacent sparse feature maps from the several sparse feature maps to obtain the processed sparse feature map and its adjacent sparse feature maps; the target sparse feature map is any one of the several sparse feature maps. Specifically, the two-dimensional submanifold sparse convolution process is mainly performed by the MFE module in the staring module. It should be noted that the MFE module is the core module of this invention for enhancing weak moving targets and suppressing false alarm regions. Its basic idea is that real moving targets usually have continuous trajectories within a short local window. If the target... Frame located Then in the adjacent front δ Frame and after δWithin a frame, the target's offset relative to its current position is typically approximately a mirror image over a short period. That is, if the target is in... Frame appears Then in Frames may appear Nearby. Random noise, static background textures, or occasional high-response regions typically do not satisfy this mirror motion consistency. It should be noted that the input to the gaze module is a sparse set of features from consecutive frames: Its output is sparse features after spatiotemporal modeling and feature enhancement. ,in, Indicates the first The sparse features after frame enhancement are used for subsequent object detection. T for t The maximum value.

[0040] set up , , They represent the first Frame, First Frame and the The sparse feature map of the frame. The MFE first performs sparse convolution on the features of adjacent frames using a shared two-dimensional submanifold, resulting in: ; ; ; For the Frame sparse coordinate set any position in Let its eigenvectors for: ; in, This represents the number of feature channels. Let the local window size be... The corresponding offset set is The offset vector is denoted as: In practice, center offset can be excluded during mirror aggregation. This is to avoid enhancement being dominated solely by the similarity of the current location itself.

[0041] The next step requires calculating the cosine similarity between the central feature of the processed sparse feature map and the central features of its neighboring processed sparse feature maps. Specifically, MFE calculates the... Frame center features Cosine similarity between the coordinates and the mirrored position features of adjacent frames. Defined as: ; in, To prevent division by zero stability constant.

[0042] Then, based on the cosine similarity and the positional features corresponding to each coordinate of the processed adjacent sparse feature maps, a similarity score needs to be calculated between the processed sparse feature maps and their adjacent counterparts. The product of these similarity scores is then calculated to obtain the target product, and the product with the largest value in the target product is taken as the maximum response. This maximum response is then used as the motion consistency score for each coordinate of the processed sparse feature map. Specifically, for the offset... Mirror position features of the previous frame Corresponding position features in the next frame They are respectively: ; ; in, This represents the interval between adjacent frames; here, it's set to 2 to ensure sufficient displacement of the target. For boundary frames, time-mirroring padding can be used to obtain missing adjacent frames, avoiding out-of-bounds sampling. The similarity of consecutive adjacent frames is as follows: ; ; Mirror response product definition for: ; Then, the position is obtained by taking the maximum response on the local offset set. Motion consistency score : ; If a candidate position corresponds to a real moving target, it is more likely to show a mirror-consistent response in the preceding and following frames. The magnitude is relatively large; if the candidate position is random noise or background clutter, it is difficult to form a stable mirror correspondence between its preceding and following frames. Smaller.

[0043] Finally, the sparse feature maps of each target are enhanced based on the motion consistency score to obtain several enhanced sparse feature maps; the target sparse feature map is any one of the several sparse feature maps. Specifically, the L1 norm of each sampling point in the target sparse feature map needs to be calculated to obtain several norm values, and the average of these norm values ​​is used as the average intensity of the target sparse feature map. Then, the gain factor of the target sparse feature map is calculated based on the average intensity and the L1 norm of each sampling point, and the target sparse feature map is enhanced based on the gain factor and the motion consistency score to obtain several enhanced sparse feature maps. An adaptive gain (AG) mechanism is needed to enhance weak targets with low response but satisfying motion consistency. Since the MFE module is usually located after ReLU activation and its eigenvalues ​​are non-negative, the L1 norm can be used to measure the feature intensity of candidate points. : ; No. Average intensity of all sampled points in the frame for: ; in, Indicates the first Number of sparse sampling points per frame.

[0044] Adaptive gain factor Defined as: ; in, For gain intensity control parameters, This is a stability constant. When the intensity of a real target feature is lower than the average intensity, The gain is larger, thus obtaining stronger compensation; when a target itself has a strong response, the gain is smaller, which can avoid over-amplifying the strong response region.

[0045] Finally, the enhanced feature vector Defined as: ; This formula exhibits residual enhancement properties. When the candidate point does not satisfy the mirror motion constraint... The signal strength is low, the enhancement term is small, and the network basically retains the original features; when candidate points satisfy motion consistency and their own response is weak, and The combined effect significantly enhances the real weak target.

[0046] After all sampling points are enhanced, the enhanced features are scattered back to their original sparse coordinate positions to obtain the enhanced sparse feature map. Then, by performing contextual semantic aggregation through dilated sparse 3D convolution, the final output of MFE is obtained: Then As the final output of the gaze module .

[0047] Step S14: Detect moving targets in the target video segment based on the several enhanced sparse feature maps using a preset sparse detection head, so as to obtain detection results corresponding to the moving targets.

[0048] In this embodiment, a preset sparse detection head is needed to detect moving targets in a target video segment based on several enhanced sparse feature maps to obtain detection results corresponding to the moving targets. Specifically, the preset sparse detection head is needed to detect moving targets in a target video segment based on several enhanced sparse feature maps to output the predicted target center point, target width and height, and center offset corresponding to the moving targets. Then, the detection results of the moving targets are generated based on the predicted target center point, target width and height, and center offset. It should be noted that the sparse features processed by the staring module and the MFE module need to be input into the sparse detection head. This detection head can be modified based on the CenterNet-style anchorless detection head, replacing the original dense convolution with sparse convolution, predicting target information only on sparse candidate coordinates. The detection head output includes a target center heatmap, target width and height, and center offset. ; in, This represents a predicted heatmap of the target center point. This indicates the predicted width and height of the target. This represents the center point sub-pixel offset prediction, also known as the center offset.

[0049] Furthermore, in this embodiment, the target loss generated by the moving target detection of the target video segment can be determined, and the scan-stare network can be trained end-to-end based on the target loss and a preset joint loss function, so that the updated detection network obtained is used as the current scan-stare network. The joint loss function is used during training. : ; in, This represents the coarse localization supervision loss of the scanning module. This represents the target detection loss of the sparse detection head. Indicates the number of input frames. A penalty based on anisotropic Gaussian heatmaps can be used to reduce focus loss, thereby encouraging the scanning module to cover potentially weak targets as much as possible. This can include center point heatmap loss, width and height regression loss, and offset regression loss: ; in, For the center point heatmap loss, For the target size regression loss, For center-shifted regression loss, and The weighting coefficients are used. Through end-to-end joint training, the scanning module can learn candidate regions that are more suitable for subsequent gaze detection, while the gaze and MFE modules can further enhance real moving targets based on sparse candidate points, achieving collaborative optimization between scanning and gaze. As shown in Figures 4(a) and 4(b), these are comparisons of the MFE module before and after enhancement on two different datasets, demonstrating the feature response comparison before and after the MFE module's operation. The figures may include examples such as weak targets being enhanced, non-linear moving targets still being enhanced, and non-target false alarm regions not being enhanced, to illustrate the selective enhancement capability of the MFE module.

[0050] Finally, the output module decodes the target bounding box for each frame based on the center point confidence, width, height, and offset. For multiple candidate boxes, post-processing methods such as confidence thresholding, local maxima filtering, or non-maximum suppression can be used to obtain the final moving target detection result.

[0051] In this embodiment, spatiotemporal modeling can be performed on a target video segment composed of several consecutive satellite video images to extract features from the resulting target probability response maps, obtaining several dense feature maps corresponding to the target probability response maps. A corresponding dynamic threshold is calculated based on the mean and standard deviation of each map in the target probability response map, and sampling points in the probability response maps are filtered according to the dynamic threshold to generate a sparse coordinate set based on the coordinates of the obtained candidate sampling points. The dense feature maps are then converted into sparse feature maps based on the sparse coordinate set, and the sparse feature maps are enhanced. A preset sparse detection head detects moving targets in the target video segment based on the enhanced sparse feature maps to obtain detection results corresponding to the moving targets. In this way, on the one hand, dense features can be converted into sparse candidate features during the scanning stage, ensuring that subsequent high-cost feature extraction only occurs in potential target areas, reducing computational overhead from the source; on the other hand, short-term motion consistency between consecutive frames can be utilized to adaptively enhance targets that meet motion constraints but have weak feature responses; furthermore, it can be determined whether candidate points conform to the short-term trajectory patterns of real moving targets, thereby suppressing non-target areas that do not meet motion consistency.

[0052] As a preferred embodiment, taking a specific implementation scenario as an example, this embodiment uses vehicle target detection in satellite video as an example. The input data is continuous. Frame satellite video images, preferred First, the input images are normalized and then assembled into video clips. The video clip is then input into the scanning module.

[0053] The scanning module employs a lightweight 3D U-Net architecture to perform coarse-grained spatiotemporal modeling on consecutive frames, outputting a target probability response map for each frame. When training the scanning module, an anisotropic Gaussian heatmap is constructed based on the real target bounding boxes. The Gaussian center is the center of the target bounding box, and the horizontal and vertical standard deviations are determined by the target width and height, respectively. Through this supervision method, the scanning module learns the coarse response distribution covering the potential small target region.

[0054] get Then, the ACS module calculates. mean and standard deviation and according to The threshold is obtained. Preferably, Take 3. Then keep all. The position forms a sparse coordinate set. Dense features in the scan output according to Convert to sparse features .

[0055] Sparse features are fed into the gaze module. The gaze module adopts a sparse 3D U-Net structure, performing convolution computations only in sparse coordinates and their neighborhoods to complete fine-grained spatiotemporal context modeling. To further enhance weak targets, MFE modules are inserted into the multi-layer features of the gaze module. Preferably, window sizes are used in the three feature layers respectively. , For the first Each sampling point of the frame The MFE module calculates the cosine similarity between the features of the current frame and the local mirror position features of the preceding and following frames, and obtains the motion consistency score through the maximum mirror response. Simultaneously, the MFE calculates the adaptive gain based on the feature intensity of the current sampling point and the average sampling point intensity across the entire frame. Ultimately, according to Enhanced features.

[0056] The enhanced sparse features are input into the sparse detection head. The detection head outputs a target center heatmap, target size, and center offset. The bounding box of the vehicle target in each frame is decoded based on the prediction results. A joint loss is used during the training phase. ,in Supervisory scan coarse scan results, Supervise the final test results. For example... Figure 5The diagram shows the output of the scan-stare network at different stages, illustrating the input image, the coarse scan result, and the final detection result. This diagram demonstrates that the scanning module can preserve potential target regions at a lower sampling rate, while stare and MFE further separate adherent targets and suppress false alarms.

[0057] like Figure 6 The figure shows a visualization of the performance of different methods on the VISO (VIdeo Satellite Objects) and SDM-Car (Small and Dim Moving Cars) datasets. In the figure, the green line represents the true target trajectory, green dots represent true positives (TP), red dots represent false negatives (FN), and blue dots represent false positives (FP). Table 1 shows the quantitative results of different methods on the VISO and SDM-Car datasets. Re, Pr, and F1 represent recall, precision, and F1 score, respectively, and FPS (Frames Per Second) represents the inference frame rate. The parameter unit is M (millions). In the table, the best results are shown in bold, and the second-best results are indicated by underline.

[0058] Table 1 Quantitative Results of Indicators ; Figure 7(a) shows the processing flowchart of the dual-stream spatiotemporal fusion method, Figure 7(b) shows the processing flowchart of the temporal cyclic propagation method, and Figure 7(c) shows the processing flowchart of the method of this application. Existing methods usually maintain dense features throughout the entire process, while this application first generates sparse features by scanning the backbone, and then performs fine spatiotemporal modeling by staring at the backbone, thereby reducing background redundant calculations.

[0059] In some embodiments, this application discloses a video moving target detection system based on a scan-staring network, comprising: a management server and a terminal; wherein the terminal is configured to compose a target video segment based on several acquired continuous satellite video images, and perform spatiotemporal modeling on the target video segment to extract features from several target probability response maps obtained by modeling, thereby obtaining several dense feature maps corresponding to the several target probability response maps; calculate the mean and standard deviation of each map in the several target probability response maps, and calculate the dynamic threshold corresponding to the several target probability response maps based on the mean and standard deviation, and then filter the sampling points in the several target probability response maps according to the dynamic threshold, so as to generate a sparse coordinate set based on the coordinates of the candidate sampling points obtained after filtering; convert the several dense feature maps into several sparse feature maps based on the sparse coordinate set, and enhance the several sparse feature maps to obtain several enhanced sparse feature maps; and detect moving targets in the target video segment based on the several enhanced sparse feature maps using a preset sparse detection head, so as to obtain detection results corresponding to the moving targets.

[0060] See Figure 8 As shown, this embodiment of the invention discloses a video moving target detection device based on a scan-stare network, applied to a scan-stare network, comprising: The feature extraction module 11 is used to compose a target video segment based on a number of acquired continuous satellite video images, and to perform spatiotemporal modeling on the target video segment in order to extract features from the modeled target probability response maps to obtain a number of dense feature maps corresponding to the target probability response maps. The coordinate filtering module 12 is used to calculate the mean and standard deviation of each graph in the plurality of target probability response graphs, and calculate the dynamic threshold corresponding to the plurality of target probability response graphs based on the mean and standard deviation, and then filter the sampling points in the plurality of target probability response graphs based on the dynamic threshold, so as to generate a sparse coordinate set based on the coordinates of the candidate sampling points obtained after filtering. The feature enhancement module 13 is used to convert the several dense feature maps into several sparse feature maps based on the sparse coordinate set, and to enhance the several sparse feature maps to obtain several enhanced sparse feature maps. The target detection module 14 is used to detect moving targets in the target video segment based on the several enhanced sparse feature maps using a preset sparse detection head, so as to obtain the detection result corresponding to the moving target.

[0061] In some embodiments, the feature extraction module 11 may specifically include: The video clip generation unit is used to normalize several consecutive satellite video images to generate target video clips based on the obtained normalized images. A spatiotemporal modeling unit is used to input the target video segment into a preset scanning component, so as to perform spatiotemporal modeling on each frame of video image in the target video segment through the preset scanning component, so as to obtain a number of corresponding target probability response maps; the preset scanning component is a preset convolutional neural network; The feature extraction unit is used to perform coarse-grained feature extraction on the plurality of target probability response maps through the preset scanning component, so as to obtain a plurality of dense feature maps corresponding to the plurality of target probability response maps.

[0062] In some embodiments, the coordinate filtering module 12 may specifically include: The parameter calculation unit is used to calculate the mean and standard deviation of each graph in the plurality of target probability response graphs, and to calculate the product of the standard deviation and the preset intensity coefficient to obtain the target product; A dynamic threshold determination unit is used to use the sum of the target product and the mean as the dynamic threshold corresponding to each graph in the plurality of target probability response graphs; The sampling point elimination unit is used to filter the sampling points in the plurality of target probability response maps according to the dynamic threshold, so as to eliminate the sampling points whose probability values ​​in the plurality of target probability response maps are less than the dynamic threshold, and obtain candidate sampling points. The coordinate set generation unit is used to generate a sparse coordinate set based on the coordinates of the candidate sampling points.

[0063] In some embodiments, the feature enhancement module 13 may specifically include: The feature transformation submodule is used to sample the several dense feature maps based on the coordinates in the sparse coordinate set, so as to transform the several dense feature maps into several sparse feature maps. The cosine similarity calculation submodule is used to perform convolution processing on the several sparse feature maps to obtain processed sparse feature maps, and to calculate the cosine similarity between the processed sparse feature maps and the processed adjacent sparse feature maps; the processed adjacent sparse feature maps are the feature maps that are adjacent to the processed sparse feature maps before and after the processed sparse feature maps. The first score calculation submodule is used to calculate the similarity score between the processed sparse feature map and the processed adjacent sparse feature maps based on the cosine similarity. The second score calculation submodule is used to calculate the product between the similarity scores to obtain the target product, and the product with the largest value in the target product is taken as the maximum response, and the maximum response is taken as the motion consistency score of each coordinate of the processed sparse feature map. The feature enhancement submodule is used to enhance each target sparse feature map based on the motion consistency score to obtain several enhanced sparse feature maps; the target sparse feature map is any one of the several sparse feature maps.

[0064] In some embodiments, the cosine similarity calculation submodule may specifically include: The convolution processing unit is used to perform two-dimensional submanifold sparse convolution processing on the target sparse feature map and the adjacent sparse feature map corresponding to the target sparse feature map in the plurality of sparse feature maps, so as to obtain the processed sparse feature map and the processed adjacent sparse feature map. The cosine similarity calculation unit is used to calculate the cosine similarity between the central feature corresponding to the processed sparse feature map and the central feature corresponding to the adjacent sparse feature maps after processing. In some embodiments, the first score calculation submodule may specifically include: The similarity score calculation unit is used to calculate the similarity score between the processed sparse feature map and the processed adjacent sparse feature map based on the cosine similarity and the positional features corresponding to each coordinate of the processed adjacent sparse feature map.

[0065] In some embodiments, the feature enhancement submodule may specifically include: The average intensity determination unit is used to calculate the L1 norm of each sampling point in the target sparse feature map to obtain several norm values, and to use the average value of the several norm values ​​as the average intensity of the target sparse feature map. The feature enhancement unit is used to calculate the gain factor of the target sparse feature map based on the average intensity and the L1 norm of each sampling point, and enhance each of the target sparse feature maps based on the gain factor and the motion consistency score to obtain several enhanced sparse feature maps.

[0066] In some embodiments, the target detection module 14 may specifically include: The detection parameter determination unit is used to detect moving targets in the target video segment based on the several enhanced sparse feature maps according to the preset sparse detection head, so as to output the predicted target center point, target width and height and center offset corresponding to the moving target; The target detection unit is used to generate the detection result of the moving target based on the predicted target center point, the target width and height, and the center offset.

[0067] In some embodiments, the video moving target detection system based on a scan-stare network may further include: The network training unit is used to determine the target loss generated by moving target detection of the target video segment, and to perform end-to-end training of the scan-stare network based on the target loss and a preset joint loss function, so as to use the updated detection network as the current scan-stare network.

[0068] Furthermore, embodiments of this application also disclose an electronic device, Figure 9 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0069] Figure 9 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the video moving target detection method based on a scan-staring network disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be a computer.

[0070] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0071] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0072] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the video moving target detection method based on a scanning-staring network executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0073] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned video moving target detection method based on a scan-stare network. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0074] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0075] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0076] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0077] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0078] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A video moving target detection method based on a scan-staring network, characterized in that, Applied to a scan-gaze network, the scan-gaze network is a pre-defined target detection network, comprising: The target video segment is composed of several consecutive satellite video images, and the target video segment is spatiotemporally modeled. Feature extraction is performed on several target probability response maps obtained by modeling to obtain several dense feature maps corresponding to the several target probability response maps. Calculate the mean and standard deviation of each graph in the plurality of target probability response graphs, and calculate the dynamic threshold corresponding to the plurality of target probability response graphs based on the mean and standard deviation. Then, filter the sampling points in the plurality of target probability response graphs based on the dynamic threshold, so as to generate a sparse coordinate set based on the coordinates of the candidate sampling points obtained after filtering. Based on the sparse coordinate set, the dense feature maps are converted into sparse feature maps, and the sparse feature maps are enhanced to obtain enhanced sparse feature maps. The preset sparse detection head detects moving targets in the target video segment based on the several enhanced sparse feature maps, so as to obtain the detection results corresponding to the moving targets.

2. The video moving target detection method based on scan-staring network according to claim 1, characterized in that, The process involves assembling a target video segment from several acquired consecutive satellite video images, performing spatiotemporal modeling on the target video segment, and extracting features from several target probability response maps obtained through modeling to obtain several dense feature maps corresponding to the several target probability response maps, including: Several consecutive satellite video images were acquired and normalized to generate a target video segment based on the obtained normalized images. The target video segment is input into a preset scanning component to perform spatiotemporal modeling on each frame of the target video segment, thereby obtaining several corresponding target probability response maps; the preset scanning component is a preset convolutional neural network. The preset scanning component performs coarse-grained feature extraction on the probability response maps of the targets to obtain a number of dense feature maps corresponding to the probability response maps of the targets.

3. The video moving target detection method based on scan-staring network according to claim 1, characterized in that, The process involves calculating the mean and standard deviation of each graph in the plurality of target probability response graphs, calculating a dynamic threshold corresponding to each graph based on the mean and standard deviation, and then filtering the sampling points in the plurality of target probability response graphs based on the dynamic threshold to generate a sparse coordinate set based on the coordinates of the candidate sampling points obtained after filtering, including: Calculate the mean and standard deviation of each graph in the plurality of target probability response graphs, and calculate the product of the standard deviation and the preset intensity coefficient to obtain the target product; The sum of the target product and the mean is used as the dynamic threshold corresponding to each graph in the plurality of target probability response graphs; The sampling points in the plurality of target probability response maps are filtered according to the dynamic threshold, so as to remove the sampling points whose probability values ​​in the plurality of target probability response maps are less than the dynamic threshold, and to obtain candidate sampling points. A sparse coordinate set is generated based on the coordinates of the candidate sampling points.

4. The video moving target detection method based on scan-staring network according to claim 1, characterized in that, The process of converting the dense feature maps into sparse feature maps based on the sparse coordinate set, and then enhancing the sparse feature maps to obtain enhanced sparse feature maps, includes: Based on the coordinates in the sparse coordinate set, the dense feature maps are sampled to convert the dense feature maps into sparse feature maps. The sparse feature maps are convolved to obtain processed sparse feature maps, and the cosine similarity between the processed sparse feature maps and their adjacent processed sparse feature maps is calculated; the adjacent processed sparse feature maps are the feature maps that are adjacent to the processed sparse feature maps before and after the processed sparse feature maps. The similarity score between the processed sparse feature map and its adjacent processed sparse feature maps is calculated based on the cosine similarity. Calculate the product between the similarity scores to obtain the target product, and take the product with the largest value in the target product as the maximum response, and take the maximum response as the motion consistency score of each coordinate of the processed sparse feature map; The target sparse feature maps are enhanced based on the motion consistency score to obtain several enhanced sparse feature maps; the target sparse feature map is any one of the several sparse feature maps.

5. The video moving target detection method based on scan-staring network according to claim 4, characterized in that, The step of performing convolution processing on the plurality of sparse feature maps to obtain processed sparse feature maps, and calculating the cosine similarity between the processed sparse feature maps and their adjacent processed sparse feature maps, includes: Two-dimensional submanifold sparse convolution is performed on the target sparse feature map and the adjacent sparse feature map corresponding to the target sparse feature map to obtain the processed sparse feature map and the processed adjacent sparse feature map. Calculate the cosine similarity between the central feature corresponding to the processed sparse feature map and the central feature corresponding to the adjacent processed sparse feature maps.

6. The video moving target detection method based on scan-staring network according to claim 4, characterized in that, The calculation of the similarity score between the processed sparse feature map and its adjacent processed sparse feature maps based on the cosine similarity includes: The similarity score between the processed sparse feature map and the processed adjacent sparse feature map is calculated based on the cosine similarity and the positional features corresponding to each coordinate of the processed adjacent sparse feature map.

7. The video moving target detection method based on scan-staring network according to claim 4, characterized in that, The enhancement of each target sparse feature map based on the motion consistency score to obtain several enhanced sparse feature maps includes: Calculate the L1 norm of each sampling point in the target sparse feature map to obtain several norm values, and take the average of the several norm values ​​as the average intensity of the target sparse feature map. The gain factor of the target sparse feature map is calculated based on the average intensity and the L1 norm of each sampling point, and the target sparse feature map is enhanced based on the gain factor and the motion consistency score to obtain several enhanced sparse feature maps.

8. The video moving target detection method based on scan-staring network according to claim 1, characterized in that, The step of detecting moving targets in the target video segment based on the several enhanced sparse feature maps using a preset sparse detection head to obtain detection results corresponding to the moving targets includes: The preset sparse detection head detects moving targets in the target video segment based on the several enhanced sparse feature maps, and outputs the predicted target center point, target width and height, and center offset corresponding to the moving target. The detection result of the moving target is generated based on the predicted target center point, the target width and height, and the center offset.

9. The video moving target detection method based on a scan-staring network according to any one of claims 1 to 8, characterized in that, Also includes: The target loss generated by moving target detection on the target video segment is determined, and the scan-stare network is trained end-to-end based on the target loss and a preset joint loss function, so that the updated detection network is used as the current scan-stare network.

10. A video moving target detection system based on a scan-staring network, characterized in that, Applied to a scan-gaze network, the scan-gaze network is a pre-defined target detection network, comprising: Management server, terminal; The terminal is configured to compose a target video segment based on several acquired continuous satellite video images, and perform spatiotemporal modeling on the target video segment to extract features from several target probability response maps obtained from the modeling, thereby obtaining several dense feature maps corresponding to the several target probability response maps; calculate the mean and standard deviation of each map in the several target probability response maps, and calculate the dynamic threshold corresponding to the several target probability response maps based on the mean and standard deviation, then filter the sampling points in the several target probability response maps based on the dynamic threshold, and generate a sparse coordinate set based on the coordinates of the candidate sampling points obtained after filtering; convert the several dense feature maps into several sparse feature maps based on the sparse coordinate set, and enhance the several sparse feature maps to obtain several enhanced sparse feature maps; and detect moving targets in the target video segment based on the several enhanced sparse feature maps using a preset sparse detection head to obtain detection results corresponding to the moving targets.

Images