A video sar moving target shadow detection method based on timing appearance feature aggregation

Abstract

The application belongs to the technical field of image processing, and discloses a video SAR moving target shadow detection method based on time sequence appearance feature aggregation. First, a SAR video is acquired, and the SAR video is disassembled into frame sequence images. Then, adjacent two SAR images are input into a trained target detection network model based on time sequence appearance feature aggregation. When processing, the adjacent two SAR images are sequentially processed through a module such as an encoder in the network model, a reliable association clue generation module of environmental information compensation, a feature enhancement module guided by the association clue, a detection head and the like, so as to acquire a moving target shadow detection result of a next frame, i.e., a current frame SAR image. Finally, the previous step is repeated, and all adjacent two SAR image pairs are processed in time sequence, so as to obtain a video SAR moving target shadow detection result. The method disclosed by the application can solve the missed detection and false detection problems caused by the appearance instability of the moving target shadow and the appearance similarity between instance targets, and realize high-precision ViSAR moving target shadow detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image processing, and particularly relates to a video SAR moving target shadow detection method based on time sequence appearance feature aggregation. BACKGROUND

[0002] Video Synthetic Aperture Radar (ViSAR) has the ability of all-weather and all-day observation and motion target monitoring, and has great application value in disaster relief, traffic management and military reconnaissance. Due to the high frame rate of ViSAR imaging, the Doppler modulation of moving target echoes is very sensitive to the motion, which causes the shift and defocus of the backscattering body of the moving target after imaging, and forms a shadow on the real position. Therefore, detecting the moving target shadow can realize the continuous monitoring of the moving target. In addition, compared with the static target shadow of the adjacent strong scatterer caused by occlusion, the moving target shadow with interval from the scatterer has distinguishable visual features, so the deep learning method based on data driving has good applicability and feasibility in moving target shadow detection.

[0003] However, the moving target shadow of ViSAR has the following visual characteristics, which makes the target detection deep learning method currently applied to optical images cannot be directly applied to ViSAR data: (1) Due to the Doppler effect between the ViSAR load platform and the moving target, and the coherent accumulation method for video imaging, the moving target shadow has appearance changes such as occlusion and imaging blur during the movement, which causes the instability of the target visual features and the missing detection problem of the shadow target; (2) There is appearance similarity between shadow instances, which makes the motion detection method depending on optical flow estimation deep learning model produce false motion clues, causing the missing detection and false detection problem of the shadow target. Therefore, the deep learning method for the task of ViSAR moving target shadow detection needs to be designed and developed. SUMMARY

[0004] In view of the above problems in the background art, the present application provides a video SAR moving target shadow detection method based on time sequence appearance feature aggregation, which realizes high-precision ViSAR moving target shadow detection.

[0005] The present application adopts the following technical solutions to solve the technical problems:

[0006] A video SAR moving target shadow detection method based on time sequence appearance feature aggregation, comprising the following steps:

[0007] Step 1: acquiring a SAR video, and disassembling the SAR video into a frame sequence image;

[0008] Step 2: input the adjacent two frames of SAR images into the trained target detection network model based on the time sequence appearance feature aggregation, and process them through the encoder, the reliable association clue generation module of environmental information compensation, the feature enhancement module guided by the association clue, the detection head and other modules in the network model in sequence to obtain the moving target shadow detection result of the current frame of SAR images;

[0009] The weight-shared backbone network encoder is used to extract the deep features of the original image, input the original image, and output the deep features of the image.

[0010] The reliable association clue generation module of environmental information compensation is used to extract the similarity matrix of each pixel of the previous frame of deep feature maps and each pixel of the next frame of deep feature maps, input the deep features of the previous frame and the next frame of images, and output a four-dimensional pixel-by-pixel similarity matrix.

[0011] The feature enhancement module guided by the association clue is used to sample and add the same-named pixel points of the previous frame of feature maps and the next frame of feature maps by weighting, input the previous frame of feature maps and the four-dimensional pixel-by-pixel similarity matrix, and output the weighted addition matrix of the same-named pixel points of the previous frame of feature maps and the next frame of feature maps.

[0012] The detection head is used to output the feature maps related to the detection task, input the weighted addition matrix of the previous module, and output the position, size and classification feature maps which are the same size as the original image.

[0013] Step 3: repeat step 2 to process all adjacent two frames of SAR images in time sequence to obtain the video SAR moving target shadow detection result.

[0014] Moreover, the structure of the target detection network model based on the time sequence appearance feature aggregation in step 2 is composed of the weight-shared backbone network encoder (such as ResNet, MobileNet or DLA-34), the reliable association clue generation module of environmental information compensation, the feature enhancement module guided by the association clue, and the detection head.

[0015] Moreover, the reliable association clue generation module of environmental information compensation in step 2 is composed of an environmental information compensation module and a cost volume. The environmental information compensation module includes three dilated convolution layers with different expansion rates, a 1x1 convolution layer and a ReLU activation function, two layers of 3x3 convolution normalization layers ConvBN and a layer of 3x3 convolution layer. The cost volume is a pixel-by-pixel similarity matrix of the deep features of the previous and next two frames, which is obtained by processing through the reliable association clue generation module of environmental information compensation and the backbone network encoder. The feature enhancement module guided by the association clue is composed of an offset matrix, a regular convolution layer, a deformable convolution layer, a matrix multiplication operation and a Hadamard product operation.

[0016] Further, the step 2 environment information compensation module processing step includes the following 2 steps:

[0017] Step 2.1.1: input the high-dimensional feature map (high, wide and channel number are h F , w F and 64 respectively) from the encoder, denoted as E target , first extract features by three 3x3 dilated convolution layers DConv with dilation rates d of 1, 2 and 3 respectively, then splice the three obtained feature maps to obtain spliced feature E' env , process the spliced feature by 1x1 convolution layer and ReLU activation function to obtain environment information feature map E env :

[0018]

[0019]

[0020] Wherein, the subscript 3x3 is the convolution kernel size, and d represents the dilation rate;

[0021] Step 2.1.2: matrix multiplication of environment information feature maps E env and E target to obtain environment information compensation feature map , and sequentially pass through two layers of 3x3 convolution normalization layer ConvBN and one layer of 3x3 convolution layer Conv to obtain environment information compensation embedding vector E target :

[0022]

[0023]

[0024] Further, the cost volume construction method in step 2 is:

[0025] Step 2.2.1: input the previous frame I B and the current frame I C , sequentially pass through the weight-shared encoder and the environment information compensation module to obtain respective embedding feature maps and , and matrix multiplication is performed to obtain the cost volume V BC :

[0026]

[0027] The cost volume V BC stores the response value of each pixel (i, j) of the current frame feature map and each pixel (k, l) of the previous frame feature map.

[0028] Moreover, the feature enhancement module processing step of the step 2 correlation index guiding includes the following 5 steps:

[0029] Step 2.3.1: input cost volume V BC , previous frame detection result R B (that is, the detection head output of the previous frame, including position, size, classification feature map), previous frame feature map , and current frame feature map First, set the detection target intersection over union threshold value as T IOU , and calculate the Gaussian kernel radius r B of the corresponding target (x, y) using the width w and height h of each target in R g , draw a Gaussian circle at the corresponding position, and finally generate the target heat distribution map H B of the previous frame detection result:

[0030] r g = min{r g1 , r g2 , r g3} (6)

[0031]

[0032] When i = 1, a = 4*T IOU , b = 2*T IOU *(h+w), c = (T IOU -1)*(h*w);

[0033] When i = 2, a = 4, b = -2*(h+w), c = (1-T IOU )*(h*w);

[0034] When i = 3, a = 1, b = -(h+w),

[0035] Step 2.3.2: use the target heat distribution map H B of the previous frame detection result to enhance the previous frame feature map to obtain , which represents the enhancement of the target region of the previous frame feature map:

[0036]

[0037] wherein, represents the Hadamard product;

[0038] Step 2.3.3: in the cost volume V BC , take the maximum response value index (k max, l max ), to obtain the maximum response matrix O Bc :

[0039]

[0040]

[0041]

[0042]

[0043]

[0044] wherein the vertical coordinate of the previous frame high-dimensional feature map is k and the horizontal coordinate is l, and the vertical coordinate and the horizontal coordinate of the current frame are i and j, respectively;

[0045] Step 2.3.4: performing convolution operation on the maximum response matrix O BC to extract the feature O BC , inputting O BC as an offset matrix into a deformable convolution layer DCN (the hyperparameter K is 3, representing the size of the deformable convolution kernel), and performing feature transformation on to obtain

[0046]

[0047]

[0048] Step 2.3.5: using the feature transformation result to enhance the current frame feature map to obtain the current frame enhanced feature map

[0049]

[0050] Moreover, the processing step of the detection head in the step 2 is:

[0051] Step 2.4.1: performing feature transformation on the current frame enhanced feature map to obtain the target position output feature map H C , the size output feature map WH C and the classification output feature map CLS C (the number of classifications is num_class) of the current frame through the feature extractor CRC, respectively:

[0052] CRC 3×3 (x) = Conv 3×3 (ReLU(Conv 3×3 (x))) (17)

[0053]

[0054]

[0055]

[0056] Where x is the independent variable, representing the input; 3×3 represents the size of the convolution kernel; and the subscripts 1, 2, and 3 of 3×3 represent three CRC feature extractors with different parameters; based on H... C It can output a set of target information corresponding to the location, including coordinates, size, and target category.

[0057] Furthermore, in step 2, during training, after step 2.4.1, the loss function L of the video SAR shadow multi-target detection network model based on temporal appearance feature aggregation should also be calculated, including the cost volume loss. and detection loss Used for backpropagation of network model parameters:

[0058]

[0059] Loss of cost body In the example, taking pixel (i, j) of the current frame feature map, its corresponding cost body is: This represents the similarity between pixel (i, j) in the current frame feature map and all pixels in the previous frame feature map. Two max-pooling kernels, MaxPool (with kernel sizes h...), are used. F ×1 and 1×w F ) respectively act on The cost vectors in both directions can be obtained by calculating using the softmax function σ. and Let Y represent the cost vector (a horizontal vector) formed by the maximum values ​​of each column vector in the matrix, and Y represent the cost vector (a column vector) formed by the maximum values ​​of each horizontal vector in the matrix; ijkl As the training supervision label for the cost body, if the target in the current frame (i, j) appears at the position (k, l) in the previous frame, then Y... ijkl =1, otherwise Y ijkl =0, then the cost body loss This can be expressed as the sum of cost vector losses in two directions, and can be represented using the FocalLoss function (with hyperparameter β = 2):

[0060]

[0061]

[0062]

[0063] In detecting loss The data includes the target position loss for the current frame. Size loss and classification loss CLS C Among them, target location loss The target size loss is expressed in the form of the FocalLoss function (with a hyperparameter β of 2). and classification loss CLS C Represented by the L1Loss function:

[0064]

[0065]

[0066]

[0067]

[0068] In the formula, and These are the training supervision labels for the size and classification output feature maps, respectively. ij The training supervision label is the target location. If the pixel (i, j) in the current frame is the target, then Y... ij =1, otherwise Y ijkl =0.

[0069] Compared with the prior art, the present invention has the following advantages:

[0070] (1) By generating inter-frame pixel offset features, the temporal appearance features are aggregated and propagated to the current frame for feature enhancement, thereby improving the detection accuracy of moving target shadows under appearance changes such as occlusion and image blurring.

[0071] (2) By sampling the pixels around the target point, environmental information is compensated into the target point, reducing inter-frame mismatch in the cost volume, generating more reliable association clues for current frame feature enhancement, and reducing the problem of missed detection and false detection of moving target shadows in cases where the instance appearance is similar. Attached Figure Description

[0072] Figure 1 This is a target detection network model based on temporal appearance feature aggregation, as described in an embodiment of the present invention.

[0073] Figure 2 This is the environmental information compensation module in an embodiment of the present invention. Detailed Implementation

[0074] The application provides a video SAR moving target shadow detection method based on time sequence appearance feature aggregation.

[0075] The flow of the embodiment of the application comprises the following steps:

[0076] Step 1: Obtain a SAR video, and disassemble the SAR video into a frame sequence image;

[0077] Step 2: input the adjacent two frames of SAR images into the trained target detection network model based on time sequence appearance feature aggregation, and sequentially pass through the modules in the network model, such as an encoder, an environment information compensation reliable association clue generation module, an association clue guided feature enhancement module and a detection head, for processing (as shown in the figure), to obtain the moving target shadow detection result of the next frame, i.e., the current frame of SAR image; Figure 1

[0078] The weight-shared backbone network encoder is used to extract the depth features of the original image, input the original image, and output the depth features of the image.

[0079] The weight-shared backbone network encoder includes but is not limited to ResNet, MobileNet and DLA-34.

[0080] The environment information compensation reliable association clue generation module is used to extract a similarity matrix of each pixel of the depth feature map of the previous frame and each pixel of the depth feature map of the next frame, input the depth features of the previous frame and the next frame of image, and output a four-dimensional pixel-wise similarity matrix.

[0081] The environment information compensation reliable association clue generation module is composed of an environment information compensation module and a cost volume, the environment information compensation module includes three dilated convolution layers with different expansion rates, a 1x1 convolution layer and a ReLU activation function, two layers of 3x3 convolution normalization layers ConvBN and a layer of 3x3 convolution layer, the cost volume is a pixel-wise similarity matrix of the depth features of the previous and next frames, and is obtained through the environment information compensation reliable association clue generation module and the backbone network encoder.

[0082] The association clue guided feature enhancement module is used to sample and add the same-named pixel points of the previous frame of feature map and the next frame of feature map by weighting, input the previous frame of feature map and the four-dimensional pixel-wise similarity matrix, and output the weighted addition matrix of the same-named pixel points of the previous frame of feature map and the next frame of feature map.

[0083] The association clue guided feature enhancement module is composed of an offset matrix, a regular convolution layer, a deformable convolution layer, a matrix multiplication operation and a Hadamard product operation.

[0084] ​The detection head is used to output feature maps related to the detection task. It takes the weighted summation matrix from the previous module as input and outputs pixel-by-pixel position, size, and classification feature maps of the same size as the original image.

[0085] Step 2, the environmental information compensation module processing steps include the following two steps:

[0086] Step 2.1.1: In the environmental information compensation module (e.g.) Figure 2 The input (shown) is a high-dimensional feature map from the encoder (h, h ... F w F And 64) is denoted as E target First, features are extracted from the three 3×3 dilated convolutional layers DConv with dilation rates d of 1, 2, and 3. Then, the three feature maps are concatenated to obtain the concatenated feature E′. env The concatenated features are processed using a 1×1 convolutional layer and the ReLU activation function to obtain the environmental information feature map E. env :

[0087]

[0088]

[0089] Where the subscript 3×3 is the kernel size and d represents the dilation rate;

[0090] Step 2.1.2: Transfer the environmental information feature map E env and E target Matrix multiplication is used to obtain the feature map of environmental information compensation. The embedding vector E′ for environmental information compensation is obtained by sequentially passing it through two 3×3 convolutional normalization layers (ConvBN) and one 3×3 convolutional layer (Conv). target :

[0091]

[0092]

[0093] The method for constructing the cost body in step 2 is as follows:

[0094] Step 2.2.1: Input the previous frame I into the cost body B and current frame I C The embedded feature maps are obtained sequentially through a weight-sharing encoder and an environmental information compensation module. and Performing matrix multiplication on it yields the cost volume V. BC :

[0095]

[0096] Cost volume V BC Store the response value of each pixel (i, j) of the current frame feature map and the pixel (k, l) of the previous frame feature map.

[0097] The feature enhancement module processing step guided by the association line index in step 2 includes the following 5 steps:

[0098] Step 2.3.1: Input the cost volume V BC , the previous frame detection result R B , the previous frame feature map and the current frame feature map First, set the detection target intersection over union threshold value as T IOU , and calculate the Gaussian kernel radius r B of the corresponding target (x, y) using the width w and height h of each target in R g , draw a Gaussian circle on the corresponding position, and finally generate the target heat distribution map H B of the previous frame detection result:

[0099] r g = min{r g1 , r g2 , r g3} (6)

[0100]

[0101] When i = 1, a = 4 * T IOU , b = 2 * T IOU *(h + w), c = (T IOU - 1) * (h * w);

[0102] When i = 2, a = 4, b = -2 * (h + w), c = (1 - T IOU ) * (h * w);

[0103] When i = 3, a = 1, b = -(h + w),

[0104] Step 2.3.2: Use the target heat distribution map H B of the previous frame detection result to enhance the previous frame feature map to obtain , which represents the enhancement of the target region of the previous frame feature map:

[0105]

[0106] Step 2.3.3: In the cost volume V BCIn the middle, the maximum response value index (k of the previous frame high-dimensional feature map of the current frame feature map each point pixel (i, j) is taken max , max ), get the maximum response matrix O BC :

[0107]

[0108]

[0109]

[0110]

[0111]

[0112] Step 2.3.4: Convolution operation is performed on the maximum response matrix O BC Extract the feature O BC , input O BC to the deformable convolution layer DCN (the hyperparameter K is 3, representing the size of the deformable convolution kernel), and perform feature transformation on the current time to get

[0113]

[0114]

[0115] Step 2.3.5: Use the feature transformation result Enhance the current frame feature map Get the current frame enhanced feature map

[0116]

[0117] The processing steps of the detection head in step 2 are as follows:

[0118] Step 2.4.1: In the detection head, the current frame enhanced feature map Respectively through the feature extractor CRC, get the target position output feature map H C , size output feature map WH C And classification output feature map CLS C (Classification number is num_class):

[0119] CRC 3×3 (x)=Conv 3×3 (ReLU(Conv 3×3 (x))) (17)

[0120] ​

[0121]

[0122]

[0123] where x is the independent variable, representing the input, 3x3 represents the size of the convolution kernel, and the subscripts 1, 2, and 3 of 3x3 represent three CRC feature extractors with different parameters; according to H C The target information set including the coordinate position, size, and target category of the corresponding position can be output.

[0124] In addition, after step 2.4.1 in the training, the loss function L of the video SAR shadow multi-target detection network model based on the time sequence appearance feature aggregation should also be calculated, including the cost volume loss and the detection loss Back propagation for network model parameters:

[0125]

[0126] In the cost volume loss , taking the current frame feature image pixel point (i, j) as an example, the corresponding cost volume is , which represents the similarity between the current frame feature image pixel point (i, j) and all pixel points of the previous frame feature image. Two kinds of max pooling kernels MaxPool (pooling kernel sizes are h F x1 and 1xw F ) are respectively applied to and the softmax function σ is calculated to obtain two direction cost vectors and respectively, which represent the cost vector formed by the maximum value of each column vector in the matrix (a horizontal vector) and the cost vector formed by the maximum value of each horizontal vector in the matrix (a column vector); let Y ijkl be the training supervision label of the cost volume, if the target of the current frame (i, j) appears on the position (k, l) of the previous frame, then Y ijkl = 1, otherwise Y ijkl = 0, then the cost volume loss can be represented as the sum of the loss of the two direction cost vectors, which can be represented in the form of FocalLoss function (the hyperparameter β is 2):

[0127]

[0128]

[0129]

[0130] The detection loss target location loss size loss and classification loss CLS C wherein the target location loss is expressed in the form of FocalLoss function (hyper-parameter β is 2), the target size loss and the classification loss CLS C is expressed in L1Loss function:

[0131]

[0132]

[0133]

[0134]

[0135] wherein, and are the training supervision labels of size and classification output feature maps respectively. Y ij is the training supervision label of target location, if the current frame pixel point (i, j) is the target, Y ij = 1, otherwise Y ijkl = 0.

[0136] Step 3: repeat step 2, process all adjacent two-frame SAR image pairs in time sequence to obtain video SAR moving target shadow detection results.

[0137] In specific implementation, the above flow can be realized by computer software technology to realize automatic running flow.

[0138] The specific embodiments described herein are merely illustrative of the spirit of the present application. Those skilled in the art of the present application can make various modifications or supplements to the described specific embodiments or replace them with similar ways, but will not deviate from the spirit of the present application or exceed the scope defined by the appended claims.

Claims

1. A video SAR moving target shadow detection method based on timing appearance feature aggregation, characterized in that, The method comprises the following steps: Step 1, obtaining a SAR video, and disassembling the SAR video into a frame sequence image; Step 2, inputting adjacent two frames of SAR images into a target detection network model based on time sequence appearance feature aggregation which has been trained, and processing the adjacent two frames of SAR images by sequentially passing through a weight-shared backbone network encoder, an environment information compensation reliable association clue generation module, an association clue guided feature enhancement module and a detection head in the network model to obtain a moving target shadow detection result of a current frame of SAR image; The weight-shared backbone network encoder is used for extracting deep features of an original image, and inputs the original image to output the deep features of the image; The reliable association cue generation module of the environment information compensation is used for extracting a similarity matrix of each pixel of a depth feature map of a previous frame and each pixel of a depth feature map of a next frame, inputting depth features of the previous frame and the next frame, and outputting a four-dimensional pixel-by-pixel similarity matrix; the reliable association cue generation module of the environment information compensation is composed of an environment information compensation module and a cost volume, the environment information compensation module includes three dilated convolution layers with different dilated rates, a 1×1 convolution layer, an activation function, two 3×3 convolution normalization layers, and a 3×3 convolution layer; the cost volume is a pixel-by-pixel similarity matrix of depth features of the previous frame and the next frame, and is obtained through the reliable association cue generation module of the environment information compensation and a backbone network encoder. The reliable association cue generation module of the environment information compensation is used for extracting a similarity matrix of each pixel of a depth feature map of a previous frame and each pixel of a depth feature map of a next frame, inputting depth features of the previous frame and the next frame, and outputting a four-dimensional pixel-by-pixel similarity matrix; the reliable association cue generation module of the environment information compensation is composed of an environment information compensation module and a cost volume, the environment information compensation module includes three dilated convolution layers with different dilated rates, a 1×1 convolution layer, an activation function, two 3×3 convolution normalization layers, and a 3×3 convolution layer; the cost volume is a pixel-by-pixel similarity matrix of depth features of the previous frame and the next frame, and is obtained through the reliable association cue generation module of the environment information compensation and a backbone network encoder. The environment information compensation module processing step comprises the following two steps: Step 2.1.1, input the high-dimensional feature map from the encoder, denoted as... The height, width, and number of channels are respectively , And 64, firstly through the expansion rate Three types of 3×3 hollow convolutional layers, namely 1, 2, and 3. Features are extracted from them, and then the three feature maps are concatenated using a 1×1 convolutional layer. Activation function calculation yields environmental information feature map : （1） （2） Wherein, the subscript 3x3 is the convolution kernel size, and d represents the dilation rate; Step 2.1.2, the environmental information feature map is obtained by matrix multiplication and environmental information compensation feature map , and sequentially passes through two 3x3 convolution normalization layers and a 3x3 convolution layer to obtain the environmental information compensation embedding vector : （3） （4） The association clue guided feature enhancement module is used for sampling and weightingly adding same-named pixel points of a previous frame feature map and a next frame feature map, inputs the previous frame feature map and a four-dimensional pixel-by-pixel similarity matrix, and outputs a weighting addition matrix of the same-named pixel points of the previous frame feature map and the next frame feature map; The detection head is used for outputting a feature map related to a detection task, inputs the weighting addition matrix of the previous module, and outputs a position, size and classification feature map which is the same as the original image in size and has pixels; Step 3, repeating step 2, processing all adjacent two frame SAR image pairs in time sequence to obtain a video SAR moving target shadow detection result.

2. The video SAR moving target shadow detection method based on the time-series appearance feature aggregation of claim 1, wherein: The weight-shared backbone network encoder in step 2 comprises ResNet, MobileNet and DLA-34.

3. The video SAR moving target shadow detection method based on the time-series appearance feature aggregation of claim 1, wherein: The construction method of the cost volume is: input previous frame and current frame , sequentially through the weight-shared backbone network encoder and the environmental information compensation module to obtain the respective embedded feature map and , and do matrix multiplication to obtain the cost volume : ， （5） cost volume stores the response value of the pixel of each point of the current frame feature map and the pixel of each point of the previous frame feature map ​ 4. The video SAR moving target shadow detection method based on the time-series appearance feature aggregation of claim 1, wherein: The association clue guided feature enhancement module is composed of an offset matrix, a regular convolution layer, a deformable convolution layer, a matrix multiplication operation and a Hadamard product operation.

5. The video SAR moving target shadow detection method based on the time-series appearance feature aggregation of claim 3, wherein: The processing step of the association clue guided feature enhancement module comprises the following five steps: Step 2.3.1, Input Cost Body Previous frame detection results Previous frame feature map and current frame feature map First, let the cross-union ratio (CUI) threshold for the detected targets be... ,use Width of each target and high The corresponding target was calculated. Gaussian kernel radius A Gaussian circle is drawn at the corresponding position, and finally the target thermal distribution map of the previous frame detection result is generated. : （6） （7） When time, , , ; When time, , , ; When time, , , ; Step 2.3.2, target heat map using previous frame detection result to the previous frame feature map Hadamard product to obtain , indicating that the target region of the previous frame feature map is enhanced: （8） wherein denotes the Hadamard product; Step 2.3.3, in the cost volume , take the previous frame high-dimensional feature map maximum response value index of the current frame feature map each point pixel , get the maximum response matrix : （9） （10） （11） （12） （13） Wherein, the vertical coordinate of the previous frame high-dimensional feature map is k and the horizontal coordinate is l, and the vertical coordinate and the horizontal coordinate of the current frame are i and j respectively; Step 2.3.4, on the maximum response matrix extracting features by convolution operation , the inputting to the deformable convolution layer as offset matrix , is a hyperparameter, representing the size of the deformable convolution kernel, and transforming the features to the current time to obtain : （14） （15） Step 2.3.5, using the feature transformation result enhancing the current frame feature map obtaining the current frame enhanced feature map : （16）。 6. The video SAR moving target shadow detection method based on timing appearance feature aggregation of claim 5, wherein: The processing step of the detection head is as follows: Step 2.4.1, enhancing feature maps for the current frame through the feature extractor to obtain target location output feature maps , size output feature maps and classification output feature maps with a classification number of : （17） （18） （19） （20） wherein, x is an independent variable, representing input, 3x3 represents the size of the convolution kernel, and the subscripts 1, 2, and 3 of 3x3 represent three CRC feature extractors with different parameters; according to The target information set can output target information including coordinate positions, size, and target categories corresponding to the positions.

7. The video SAR moving target shadow detection method based on the time-series appearance feature aggregation of claim 6, wherein: Loss function for training a video SAR shadow multi-target detection network model based on time-series appearance feature aggregation including cost volume loss and detection loss for back propagation of network model parameters: （21） Loss of cost body In the context of the current frame feature map pixels... The corresponding cost body is , representing the pixel points of the feature map of the current frame. Similarity to all pixels in the feature map of the previous frame; applying two max-pooling kernels. Acting on respectively The pooling kernel sizes are respectively and and after function The calculation can yield cost vectors in both directions. and Let and represent the cost vector formed by the maximum values ​​of each column vector in the matrix and the cost vector formed by the maximum values ​​of each horizontal quantity in the matrix, respectively; Let As the training supervision label for the cost body, if the current frame The target appears in the previous frame position The above ,otherwise Then the cost body loss This can be represented as the sum of the cost vector losses in two directions, using... The function can be represented in the following form: （22） （23） （24） In the detection loss , a target position loss , a size loss and a classification loss containing the current frame are included; wherein the target position loss is expressed in the form of a function, is a hyperparameter, and the target size loss and the classification loss are expressed by functions. （25） （26） （27） （28） wherein, and are training supervision labels for size and classification output feature maps, respectively, is a training supervision label for target location, if the current frame pixel point is a target then , otherwise .

Images