An anti-interference method and device based on spatiotemporal progressive learning

Abstract

The present application relates to the field of precision guidance, in particular to an anti-interference method based on space-time progressive learning, an anti-interference device based on space-time progressive learning, an electronic device and a computer readable storage medium. The method comprises: taking a continuous frame of a target to be detected, wherein the continuous frame comprises multiple frames of images continuously on a time axis; generating an initial proposal box; predicting the position of an adjacent proposal box based on the current proposal box position to form a track pipeline; obtaining a classification probability based on the regional feature space-time relationship through a global branch of a double-branch network, and determining a regression box based on global information and local information through a local branch; replacing the proposal box with the regression box with the largest classification probability to update the track pipeline. The present application starts from a coarse-scale proposal box, adopts progressive learning, uses a spatial refinement module to complete detection and cascade tasks, and combines a time sequence expansion module to extract more time sequence information, thereby improving recognition accuracy and having excellent anti-interference performance.

Description

Technical Field

[0001] This invention relates to the field of precision guidance, and in particular to an anti-interference method based on spatiotemporal progressive learning, an anti-interference device based on spatiotemporal progressive learning, an electronic device, and a computer-readable storage medium. Background Technology

[0002] In precision-guided technology, infrared imaging guidance utilizes the different thermal radiation distributions of the target and background to achieve precise strikes, possessing the ability to autonomously search, track, and identify targets in complex tactical environments. Infrared guidance boasts advantages such as high spatial resolution, high sensitivity, and the ability to operate day and night, making it the mainstream development direction for precision-guided technologies worldwide. However, with the widespread use of infrared-guided weapons, corresponding infrared jamming technologies are also constantly evolving. This leads to decreased guidance accuracy, an increased probability of misidentification of jammed targets, and insufficient anti-jamming capabilities of existing guidance technologies. Summary of the Invention

[0003] To address the aforementioned problems, this invention provides an anti-interference method based on spatiotemporal progressive learning, an anti-interference device based on spatiotemporal progressive learning, an electronic device, and a computer-readable storage medium.

[0004] In a first aspect, the present invention provides an anti-interference method based on spatiotemporal progressive learning, comprising: acquiring continuous frames of a target to be detected, wherein the continuous frames include multiple consecutive frames of images on a time axis; generating initial proposal boxes; predicting the positions of adjacent proposal boxes based on the current proposal box position to form a trajectory pipeline; obtaining classification probabilities based on the spatiotemporal relationship of regional features through a dual-branch network, and determining regression boxes based on global and local information through a local branch; replacing the proposal boxes with the regression boxes that have the highest classification probabilities, and updating the trajectory pipeline.

[0005] In some embodiments, generating the initial suggestion box includes: generating the initial suggestion box based on a preset scale and overlap.

[0006] In some embodiments, predicting the positions of adjacent suggestion boxes based on the current suggestion box position to form a trajectory pipeline includes: performing a stepwise time extension on the suggestion boxes based on time series, wherein each step s satisfies 1 <s<S max Furthermore, the trajectory pipeline B containing K frames of images s Expanded to in Indicate the connection; determine the regression function as L. s = f(x), where f is the input and x is the feature, and the formula for predicting the location is: Among them, f -1 and f +1The regression value is used; based on the formula for the predicted position, adjacent trajectory pipelines are determined. and

[0007] In some embodiments, the step of obtaining classification probabilities based on the spatiotemporal relationship of regional features through a dual-branch network, and determining regression boxes based on global and local information through a local branch, includes: inputting the regional features generated by mapping the trajectory pipeline to a feature map into the dual-branch network; performing spatiotemporal modeling on the regional features based on the global branch to obtain the classification probabilities; and fusing global and local information based on the local branch, and combining the position of the proposed boxes to obtain updated regression boxes.

[0008] In some embodiments, replacing the proposal box with the regression box that has the highest classification probability and updating the trajectory pipeline includes updating the trajectory pipeline based on the following formula:

[0009]

[0010] Where 'c' represents the category, This represents the probability distribution of the i-th suggestion box across c categories and the background class. This represents the bounding box regression parameters for each class and each frame. This represents decoding the coordinates.

[0011] In some embodiments, the dual-branch network includes an attention mechanism module.

[0012] In some embodiments, the attention mechanism module includes a spatiotemporal attention module, a channel attention module, and a motion attention module, wherein the spatiotemporal attention module highlights the feature information of the target; the channel attention module enhances the temporal correlation of the network convolutional channels; and the motion attention module highlights the motion information of the target.

[0013] Secondly, the present invention also provides an anti-interference device based on spatiotemporal progressive learning, comprising: an acquisition unit for acquiring continuous frames of a target to be detected, wherein the continuous frames include multiple consecutive frames of images on a time axis; a proposal box generation unit for generating initial proposal boxes; a spatiotemporal expansion unit for predicting the positions of adjacent proposal boxes based on the current proposal box position to form a trajectory pipeline; a spatial refinement unit for obtaining classification probabilities based on the spatiotemporal relationship of regional features through a dual-branch network, and determining regression boxes based on global and local information through a local branch; and a trajectory update unit for replacing the proposal box with the regression box with the highest classification probability to update the trajectory pipeline.

[0014] Thirdly, the present invention also provides an electronic device, comprising: a memory for storing instructions; and a processor for calling the instructions stored in the memory to execute the anti-interference method based on spatiotemporal progressive learning as described in the first aspect.

[0015] Fourthly, the present invention also provides a computer-readable storage medium storing instructions that, when executed by a processor, perform the anti-interference method based on spatiotemporal progressive learning as described in the first aspect.

[0016] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: The present invention proposes an end-to-end anti-interference recognition method based on spatiotemporal progressive learning. Starting from coarse-scale proposal boxes, it adopts the idea of ​​progressive learning, uses a spatial refinement module to complete detection and cascading tasks, and combines a temporal extension module to extract more temporal information, thereby improving recognition accuracy and possessing excellent anti-interference performance.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0018] Figure 1 A schematic diagram of the anti-interference method based on spatiotemporal progressive learning in some embodiments is shown;

[0019] Figure 2 The diagram shows schematic representations of I3D network structures in some embodiments;

[0020] Figure 3 Schematic diagrams of some embodiments of the Inc. substructure are shown;

[0021] Figure 4 A schematic diagram of a progressive learning framework for some embodiments is shown;

[0022] Figure 5 The diagram shows network structure schematics for some embodiments;

[0023] Figure 6 A schematic diagram of the initial suggestion box for some embodiments is shown;

[0024] Figure 7 The convergence plots of the loss function of the progressive learning network in some embodiments are shown in the first and third steps;

[0025] Figure 8 The diagram shows the output results of some embodiments;

[0026] Figure 9 A schematic diagram comparing the initial suggestion box and output results of some embodiments is shown;

[0027] Figure 10 Schematic diagrams of detection results from some embodiments are shown;

[0028] Figure 11 Schematic diagrams of target recognition device modules from some embodiments are shown;

[0029] Figure 12 A schematic diagram of an electronic device is shown. Detailed Implementation

[0030] The invention will now be discussed with reference to several exemplary embodiments. It should be understood that these embodiments are described merely to enable those skilled in the art to better understand and thus implement the invention, and are not intended to imply any limitation on the scope of the invention.

[0031] As used herein, the term "comprising" and its variations are to be interpreted as open-ended terms meaning "including but not limited to". The term "based on" is to be interpreted as "at least partially based on". The terms "one embodiment" and "an embodiment" are to be interpreted as "at least one embodiment". The term "another embodiment" is to be interpreted as "at least one other embodiment".

[0032] Currently, deep learning technology has become a hot research area in the field of image processing, but research on its application in anti-interference tasks is scarce. In some techniques, the performance of single-frame candidate target extraction and inter-frame concatenation in detection cascade networks severely restricts their anti-interference capabilities. This makes the algorithm's performance highly dependent on the accuracy of the detection cascade network. If the detection cascade network performs poorly, lacks sufficient computational power, or has inadequate training results, it cannot complete subsequent classification tasks.

[0033] Inspired by I3D (Inflated 3D) networks and the RPN (Region Proposal Network) in Faster RCNN (Fast Region Convolutional Neural Network), some related techniques extract a short clip around keyframes (labeled frames). First, this clip, containing multiple frames, is passed to the Mixed_4f layer through the I3Dhead structure to obtain the clip's feature map. Then, the feature map is segmented temporally to obtain the feature map representation of the keyframe, which is input into the RPN to extract bounding boxes for objects in the image. Since the regions detected by the RPN only correspond to keyframes, the bounding boxes are copied temporally to form the corresponding action channels. Next, RoIPooling (Region of Interest Pooling) is used to map the bounding boxes of each frame onto the feature map, and these are concatenated temporally to obtain the region feature map for each action pipeline. Finally, the extracted region feature maps are passed to the Mixed_5c layer using the I3D tail module, and the classification result is output through softmax.

[0034] However, a core idea of ​​the aforementioned network is to use time-delayed copying of proposal boxes from keyframes to generate action pipelines (trajectory pipelines). This involves a necessary condition: the object cannot undergo significant spatial displacement within a clip containing multiple frames. If the object's spatial displacement is large, the time-delayed copy of the action pipeline will have a significant error relative to the actual action pipeline, leading to large spatial object positioning deviations, inaccurate extraction of temporal information features, and ultimately, poor classification results.

[0035] In the anti-interference task studied in this invention, the target and interference exhibit strong spatial motion in time sequence, failing to meet the aforementioned necessary conditions. Reducing the number of images within a clip can mitigate the spatial displacement problem to some extent, but long-term event dependence is crucial for accurate action classification. Therefore, selecting too few images is insufficient to describe the temporal characteristics of the object.

[0036] To address the aforementioned issues, this invention employs an anti-interference network based on spatial progressive learning, using progressive learning to solve the problem. The progressive learning employed in this invention gradually corrects the trajectory pipeline under conditions of large input errors, bringing it closer and closer to the true target location, thus completing the spatial object position regression problem. Simultaneously, it extends the temporal dimension, extracting as many temporal dimensions of object trajectory pipelines as possible while maintaining positional accuracy, completing the temporal classification task, rather than directly detecting all targets and interference at once. Furthermore, the progressive learning of this invention does not rely on a specific object detection framework; it can directly output spatial location and classification results in one go.

[0037] To address the aforementioned issues, and considering the severe limitations imposed on anti-interference capabilities by single-frame candidate target extraction and inter-frame concatenation performance in detection cascaded networks, this invention proposes an end-to-end spatiotemporally progressive anti-interference algorithm for target detection. The method starts with coarse-scale proposal boxes, employs a progressive learning approach, utilizes a spatial refinement module to complete detection and concatenation tasks, and combines this with a temporal extension module to extract more temporal information, thereby improving classification accuracy. Furthermore, this invention can employ an attention mechanism to highlight the spatiotemporal and motion features of targets and interference, further enhancing anti-interference performance. Experimental results show that the target recognition method of this invention achieves a classification accuracy of 96.3% and possesses excellent anti-interference performance.

[0038] like Figure 1 As shown, this embodiment of the invention provides an anti-interference method based on spatiotemporal progressive learning, which may include steps S11-S15, as detailed below.

[0039] Step S11: Acquire consecutive frames of the target to be detected, wherein the consecutive frames include multiple consecutive frames of images on the time axis. In this invention, images of the target can be acquired using infrared or other imaging devices. Single-frame images are prone to misidentification; therefore, multiple consecutive frames are used to determine the target from the consecutive images, and identification is performed based on motion information and image information respectively.

[0040] In some embodiments, the anti-interference data may include different interference release methods, such as four-shot bursts or dual-shot bursts; it may also include different target movement methods, such as turning, acceleration, and other escape maneuvers. The anti-interference test data may consist of 50 sets, each set of data may be continuous frames or video, including 160 frames of images, and the size of each frame of image may be normalized, such as 400×400 pixels.

[0041] Step S12: Generate an initial suggestion box. In some embodiments, generating an initial suggestion box includes: generating an initial suggestion box based on a preset scale and overlap.

[0042] Step S13: Based on the current suggestion box position, predict the positions of adjacent suggestion boxes to form a trajectory pipeline.

[0043] Step S14: Using a dual-branch network, the classification probability is obtained based on the spatiotemporal relationship of regional features through the global branch, and the regression box is determined based on the global and local information through the local branch.

[0044] Step S15: Replace the proposal box with the regression box that has the highest classification probability and update the trajectory pipeline.

[0045] The spatiotemporal progressive learning network provided by this invention uses an I3D head as the feature extraction network. The I3D network is an improvement on the C3D (3D convolutional) network. Compared to the C3D network, a BN (Batch Normalization) layer is added after the convolutional and fully connected layers, accelerating the training process and effectively solving the gradient vanishing phenomenon. The I3D head network structure can be as follows: Figure 2 As shown, the I3D network's Conv3d_1a convolutional layer to the Mixed_4f layer is extracted, containing three Conv3d convolutional layers, three Max-Pool pooling layers, and seven Inc. substructures. The Inc. substructures are as follows: Figure 3 As shown, this design increases the depth and width of the network. In image tasks, when performing pooling operations, the network typically sets the convolution kernel and stride to be the same size, indicating that the two spatial dimensions are treated equally. However, in temporal tasks, the time factor needs to be considered. If the time dimension grows too quickly relative to the spatial dimension, it may destroy the features of the object, such as merging the edge information of two objects. If it grows too slowly, it is not conducive to capturing dynamic scenes. Therefore, in the I3D head, the time dimension can be set to 1 in the first two Max-Pool layers, and the spatial scale can be 2×2.

[0046] Let S be the largest asymptotic step in the network. max A clip I t Containing K-frame images, first processed by the I3Dhead network... Feature extraction is performed on several clips. Initially, M suggestion boxes are generated, and an initial trajectory pipeline is generated by copying them along the timeline.

[0047] This invention provides a framework for progressive learning, such as... Figure 4 As shown, S represents the spatial refinement module, T represents the temporal extension module, L represents location regression, P represents the classification probability, and the following numbers represent the number of steps. For example, L0 represents the action pipeline generated by copying the initial proposal box along time. Figure 4As shown, the image sequence first passes through the backbone network to obtain feature maps F, and then enters the progressive learning step. Taking the first step as an example, L0 is the action trajectory at a larger scale, which, together with the feature map F, is input into the RoI Pool to obtain region feature maps. In S1, a multi-task learning task is performed, including action classification and localization regression, outputting the classification probability and trajectory pipeline position under the first step. However, the classification and regression results under the first step are still not accurate enough. Therefore, the trajectory pipeline L1 generated by regression is input into T1. T1 performs a temporal extension task, extending from the current trajectory pipeline position to adjacent clip segments as new proposed trajectory pipelines, which are then input into the second step. In summary, the network continuously corrects the regression position through spatial refinement S to make the regression results more accurate through progressive learning, and includes as many time dimensions as possible through temporal extension T to make the temporal classification results more accurate.

[0048] In each progressive step, the specific structure of the network is as follows: Figure 5 As shown, after the feature map and trajectory pipeline are input into RoIPool to obtain the region feature map, they enter a two-branch structure (two-branch network). The global branch, shown as GlobalBranch in the figure, performs spatiotemporal modeling on the region features to generate global features, which are used for action classification. The local branch, shown as Local Branch in the figure, connects the global features with the corresponding region features in each frame to form local features, which are used for regression output.

[0049] In this network, the entire loss function can include temporal classification loss, regression loss of the current clip, and prediction regression loss of adjacent clips.

[0050] The loss function can be represented by the following formula:

[0051]

[0052]

[0053]

[0054] Among them, P s and N s U represents the set of positive samples and the set of negative samples selected in the s-th sampling step, respectively. i and u′ i v represents the true class label of the i-th sample in different pipelines. i and v′ irepresents the target bounding box position of the i-th sample in different pipelines, l is the weight for adjusting the classification and current regression losses, and b is the weight for the temporal extension prediction regression loss. The classification loss can be the cross-entropy loss function, and the current regression loss is the weight of the ratio between the predicted bounding box and the ground truth position in each frame. 1，smooth The loss function for prediction regression loss has the same form as the current regression loss, and is the distance between the predicted bounding box and the ground truth location in adjacent frames. 1，smooth loss.

[0055] In this embodiment of the invention, a series of scales and overlaps can be set to generate initial suggestion boxes. Specifically, the following methods can be used:

[0056] One type of initial suggestion box is generated, and there can be a total of 9 suggestion boxes, such as... Figure 6 As shown, in each sub-image, the larger bounding box represents the initial proposal box, and the smaller bounding box represents the ground truth location of the target. During network training, the initial proposal boxes need to be filtered, selecting positive and negative samples, and the loss function is calculated.

[0057] It is necessary to balance the number of positive and negative samples and select the most difficult negative samples to improve training performance. The average IoU (Intersection-over-Union) of the bounding boxes in K frames of the trajectory pipeline is calculated, and the overlap between two trajectory pipelines is calculated using the average IoU. This increases the probability of selecting negative samples with higher classification scores. In some embodiments, in step s, for a given set of sample boxes, the sample box with the largest IoU with the ground truth box can be selected first as the positive sample, ensuring that each ground truth box has at least one positive sample. Then, from the remaining samples, samples with an IoU greater than a threshold t are selected. s Samples that meet the criteria are added to the positive sample pool, and the remaining samples are added to the negative sample pool. Sampling is then performed from both the positive and negative sample pools based on their classification scores; higher classification scores result in a greater sampling probability. In the first sampling step, the IoU (Interval of Units) is used as the classification criterion for positive and negative samples.

[0058] The number of initial bounding boxes is also important for sampling positive and negative samples. When an action pipeline contains a large number of targets and chaff, i.e., a large number of real bounding boxes, the number of positive samples should at least match the number of real bounding boxes. The ratio of positive to negative samples can be set to 1:3.

[0059] In this embodiment of the invention, 34 initial proposal boxes containing different scales can be selected for the selection of positive and negative samples in the first step.

[0060] In some embodiments, step S13, predicting the positions of adjacent suggestion boxes based on the current suggestion box position to form a trajectory pipeline, includes: performing progressive time extension on the suggestion boxes based on time series, wherein each step s satisfies 1 <s<S maxFurthermore, the trajectory pipeline B containing K frames of images s Expanded to in Indicate the connection; determine the regression function as L. s = f(x), where f is the input and x is the feature, and the formula for predicting the location is: Among them, f -1 and f +1 For regression parameters; based on the formula for predicted location, determine adjacent trajectory pipelines. and

[0061] In this embodiment of the invention, the suggestion box is expanded step by step based on time sequence, thereby ensuring the reliability of the expanded trajectory pipeline and avoiding the spatial displacement problem of the target and interference. The time sequence expansion process is as follows:

[0062] The trajectory pipeline of the object has been extended. However, temporal extension cannot simply replicate the proposal box over time. Considering the spatial displacement of the target and interference in the anti-interference task, temporal extension should be performed progressively and adaptively. For each step s, satisfy 1 <s<S max The trajectory pipeline B containing K frames of images s Expanded to in Indicates a connection.

[0063] This invention employs position prediction to achieve adaptive temporal extension by training an additional regression branch that predicts the positions of adjacent channels based on the current clip's trajectory channel position. This regression branch primarily learns the residual between two adjacent clip channels. Let the regression function be L at step s. s = f(x), where x is a feature of the regression network input f, then the predicted location can be represented by the formula:

[0064]

[0065] Among them, f -1 and f +1 As a regressor, it can be decoded from this. and

[0066] In some embodiments, step S14, using a dual-branch network, obtains classification probabilities based on the spatiotemporal relationship of regional features using the global branch, and determines regression boxes based on global and local information using the local branch. This includes: inputting the regional features generated by mapping the trajectory pipeline to the feature map into the dual-branch network; performing spatiotemporal modeling on the regional features based on the global branch to obtain classification probabilities; and fusing global and local information based on the local branch, and combining the position of the proposal box to obtain the updated regression box.

[0067] In this embodiment of the invention, the suggestion boxes can be updated using a dual-branch network to obtain more accurate updated regression boxes. Specifically, the method is as follows:

[0068] The input is the action pipeline of the current step, which is mapped to the feature map by the RoI Pool to generate the region feature Pool-Feat.

[0069] The Global Branch performs spatiotemporal modeling of regional features and is responsible for the temporal classification task. Specifically, the Pool-Feat module is input into the temporal modeling module. Figure 5 The Temporal modeling structure in the code consists of I3D's mixed_5b, mixed_5c, mixed_5d, and Conv3d convolutional layers, which generate a global feature, Global-Feat. The Global-Feat is then straightened to encode the contextual information throughout the action pipeline. This encoded information is then fed into a classification convolutional layer via a dropout layer, and the output is the classification probability.

[0070] The Local Branch merges global and local information and is responsible for regressing object detection bounding boxes. Specifically, it concatenates the Pool-Feat region feature and the Global-Feat global feature to generate a Local-Feat local feature in each frame. This Local-Feat is then input into the regression module, which consists of several cascaded bottleneck layers connected to a downsampling layer. Finally, it outputs the bias value of the regressed bounding box, and combines this bias with the position of the proposed bounding box to decode the updated proposed bounding box.

[0071] In some embodiments, step S15, replacing the proposal box with the regression box that has the highest classification probability and updating the trajectory pipeline, includes updating the trajectory pipeline based on the following formula:

[0072]

[0073] Where 'c' represents the category, This represents the probability distribution of the i-th suggestion box across c categories and the background class. This represents the bounding box regression parameters for each class and each frame. This represents decoding the coordinates.

[0074] In some embodiments, the dual-branch network includes an attention mechanism module, thereby further improving network performance.

[0075] In some embodiments, the attention mechanism module includes a spatiotemporal attention module, a channel attention module, and a motion attention module. The spatiotemporal attention module highlights the feature information of the target; the channel attention module enhances the temporal correlation of the network convolutional channels; and the motion attention module highlights the motion information of the target.

[0076] In this embodiment of the invention, an attention mechanism can be incorporated into the network, specifically in the following ways:

[0077] In anti-interference tasks, the feature map size obtained after the input image sequence information is processed by a 3D convolutional network is I(N,T,C,H,W), where N represents the batch size, T represents the number of feature groups in the sequence image, C represents the number of segments, and H and W represent the height and width of the feature map, respectively. Although 3D convolutional networks have certain spatiotemporal modeling capabilities, the attention mechanism allows the network to focus more on more important information, such as spatial displacement information between different segments and spatiotemporal information of the target.

[0078] The basic structural idea of ​​the Spatiotemporal Attention Module (STE) is similar to that of traditional spatiotemporal attention modules. It can be understood as directing the neural network to focus on specific areas to highlight the features of an object. Borrowing from residual networks, this module generates a spatiotemporal information mask on a single channel. This mask is multiplied by the original input to obtain the activated features, which are then added back to the original input as the module's output. Features in the Spatiotemporal Attention Module The importance of cross-channel averaging, the input X of the module, and the fact that each channel can obtain spatiotemporal information from the mask M.

[0079] The spatiotemporal attention module takes input features X(N,T,C,H,W) and performs global average pooling on all channels to obtain global spatiotemporal features F(N,T,C,H,W). The dimension of feature F(N,T,1,H,W) is adjusted to F*(N,1,T,H,W), and then input into a 3D convolutional layer K to obtain the features. Adjustment Dimension F o (N,T,1,H,W), input Sigmoid activation function, output weight coefficient M=d(F0), spatiotemporal attention module output result: Y=X+X⊙M, output attention-added feature Y(N,T,C,H,W).

[0080] The structure of the Channel Attention Module (CE) can be similar to the channel attention mechanism in SE-Net (Squeeze-and-Excitation Networks). Since each layer of a convolutional network has a large number of convolutional kernels, resulting in a large number of channels in the feature map, the channel attention mechanism mainly involves the reallocation of resources among the various convolutional channels. The channel attention module can employ the concept of residuals, highlighting channel features through a mask and then adding them back to the original input as the output.

[0081] Channel attention can be achieved by compressing and restoring channels using two 1×1 2D convolutions respectively, with a 1D convolution in between to enhance the temporal relevance of channels. Specifically, input features X(N,T,C,H,W) are subjected to average pooling in the spatial dimension to compress the dimension of the feature map and obtain the global information F of the input features.

[0082]

[0083] Input a 2D convolutional layer K1, compress the number of channels, and obtain the features. F h =K1*F. Adjustment The dimension is Inputting a 1D convolutional layer K2 yields features

[0084] Adjustment Dimensions Input the number of channels expanded by K3 in the 2D convolutional layer to obtain F. o :F o =K3*F tmp .

[0085] Input the Sigmoid activation function and output the weight coefficient M = d(F0); the channel attention module outputs: Y = X + X ⊙ M, and outputs the attention-added features Y(N, T, C, H, W).

[0086] The structure of the motion attention module is primarily designed for temporal motion problems. For I(N,T,C,H,W), T represents the number of channels fused from several images after 3D convolution in the temporal dimension. By splitting the T dimension, the network can learn the differences between different features. Motion attention focuses on the residuals between adjacent segment features, highlighting motion information. Input features X(N,T,C,H,W) are fed into a 2D convolutional layer K1, compressing the number of channels to obtain the feature... Divide the T-dimensional space into T segments: F respectively ht The input is fed into the 2D convolutional layer K2, and then into the F layer of the previous layer.h(t-1) Subtracting them, we get the feature F. m :F m =K2*F h(t+1) -F ht .

[0087] All adjacent F m Connect them together to get F M Input spatial pooling layer to obtain features Inputting into a 2D convolutional layer increases the dimensionality of the channels, resulting in feature F. o (N, T, C, 1, 1); Input the Sigmoid activation function and output the weight coefficient M = d(F0); The output of the motion attention module is: Y = X + X⊙M, outputting the attention-added feature Y(N, T, C, H, W).

[0088] The experimental environment for this invention is as follows: operating system is Ubuntu 16.04, CPU is Intel(R) Core(TM) i7-6850k, memory is 64G, GPU is NVIDIA GTX 2080Ti, PyTorch and Keras deep learning frameworks are used, and Python 3.6 programming language is used.

[0089] The maximum progressive learning steps are set to 3, with classification thresholds of 0.2, 0.35, and 0.5 for each step. The batch size is set to 2, the training epochs to 16, the initial learning rate for the backbone I3D head to be 5 × 10⁻⁶, and the initial learning rates for the global and local branch networks to be 5 × 10⁻⁵, decreasing to half after the 6th epoch until training ends. Data augmentation techniques such as random cropping and erasure are employed, with temporal expansion used only in the third step. The classification weight in the loss function is set to 1, the regression loss weight for the current clip to be 5, and the regression loss weight for predicting the clip to be 1.

[0090] like Figure 7The graph shows the convergence of the loss function in the first and third steps of the progressive learning network. The curve marked with a triangle represents the convergence curve of the loss function in the first step, and the curve marked with a rectangle represents the convergence curve of the loss function in the third step. As can be seen from the graph, the curve marked with a triangle has a larger loss during training and is more oscillating, while the curve marked with a rectangle is relatively smoother. The reason for this is that in the first step, the input network proposal box is large in scale, differing significantly from the actual target box, resulting in a large loss. Through progressive learning, the network gradually refines the proposal box, reducing the regression loss. In the early stages of network training, the loss in the third step is greater than that in the first step because the third step includes regression loss for location prediction due to temporal expansion, while the first step does not perform temporal expansion. This demonstrates that the progressive learning network used in this invention effectively learns to gradually transform from a coarse-scale proposal box to a fine-scale one.

[0091] To illustrate the role of progressive learning, this invention visualizes the output results of the target in three steps by magnifying a specific area, such as... Figure 8 As shown. The first to third columns represent the detection results from step one to step three, respectively. Each row contains three frames from an image sequence. The topmost check box in each frame represents the target, and the remaining boxes represent decoys. From... Figure 8 As shown in the first row of images, after three progressive learning iterations, the target's detection box becomes increasingly closer to its true location, and its classification confidence level also increases: 0.53 in the first step, 0.78 in the second, and 0.98 in the third. This demonstrates that the network progressively refines the spatial location of the object through progressive learning, thereby improving classification confidence. In the second row of images, interference appeared in the target detection in the first step: a red detection box identified it as a target with a confidence level of 0.65, while a yellow detection box identified it as a decoy with a confidence level of 0.54. However, with the network's progressive learning, the incorrect classification results were removed in the second step, indicating that the network can effectively improve the accuracy of anti-interference classification through progressive learning.

[0092] like Figure 9 As shown, the large box represents the initial proposal box input to the network in the first step, and the small box represents the final output result of the network. This shows that although the network starts learning from proposal boxes with a relatively large scale, it still has a good effect on small-scale targets and chaff in anti-jamming tasks.

[0093] like Figure 10The image shows the network's detection results across three image sequences. In each frame, the top right bounding box represents the target, and the remaining boxes represent decoys. The time axis runs from left to right. In the first row of images, the target is turning; in the second row, it is accelerating; and in the third row, it is escaping. The image demonstrates that the network exhibits good detection and classification performance under all three target maneuvers.

[0094] This invention conducted a comparative experiment on the attention mechanism, using classification accuracy as the evaluation index. The experimental results are shown in Table 1. As can be seen from Table 1, after adding the attention mechanism, the classification accuracy increased from 94.2% to 96.3%, proving that the attention mechanism module can highlight the characteristics of the target and interference, and further improve network performance.

[0095] Table 1:

[0096] Anti-interference algorithm Classification accuracy STP 94.2％ STP+MA 96.3％

[0097] This invention is compared with a method that first obtains the object's motion pipeline through a detection cascade network, and then uses a dual-branch network of 1DCNN and CLSTM to complete the anti-interference task. The experimental results of this method and this invention are compared in Table 2.

[0098] Table 2:

[0099] Anti-interference algorithm Classification accuracy DC-1DCNN-CLSTM 93.3％ STP+MA 96.3％

[0100] As shown in Table 2, the classification accuracy of the algorithm designed in this invention is higher than that of the method using a dual-branch network of 1DCNN and CLSTM to complete the anti-interference task. In the DC-1DCNN-CLSTM method, the anti-interference task is divided into two stages. First, the trajectory pipeline is extracted using a detection cascade network, and then the classification task is performed on the trajectory pipeline. In the second stage of the classification task, the extraction of temporal information mainly relies on the CNN+LSTM structure. This invention uses a 3D convolutional network and incorporates an attention mechanism, which has a better ability to represent spatiotemporal information than CNN+LSTM. The algorithm of this invention can achieve end-to-end training, and the position of the object can be regressed from the initially generated proposal box without relying on the detector and cascade.

[0101] In summary, the anti-interference algorithm based on spatiotemporal progressive learning proposed in this invention completes object detection and cascading within the network, independent of a specific detection cascading framework. The algorithm starts with several pre-defined, large-scale randomly generated proposal boxes, refines the detection boxes at each step using a spatial refinement module, and acquires more temporal information using a temporal extension module, thus improving the anti-interference classification accuracy. Furthermore, the integration of an attention mechanism highlights the features of both the target and interference, further enhancing the performance of the anti-interference algorithm.

[0102] 1. This invention proposes an anti-interference method based on spatiotemporal progressive learning, which locates the position information of objects in space and classifies the category information of objects in time. In view of the problem that the performance of detection cascaded networks severely restricts the anti-interference ability, action classification networks are applied to the anti-interference task.

[0103] 2. This invention does not rely on a detection cascade framework. Instead, it progressively regresses the object's position from several initially generated coarse-scale proposal boxes, using the object detection boxes regressed in the previous step as object proposal boxes for the next step, gradually achieving spatial refinement. At the same time, in the third step, it expands the temporal information to include more time information, making the classification results more accurate.

[0104] 3. This invention employs an attention module that combines spatiotemporal attention, channel attention, and motion attention, highlighting the spatiotemporal and motion characteristics of objects. By inserting this module into the I3Dhead network structure, the network performance is further improved.

[0105] Based on the same inventive concept, such as Figure 11 As shown, the present invention also provides an anti-interference device 100 based on spatiotemporal progressive learning, comprising: an acquisition unit 110 for acquiring continuous frames of the target to be detected, wherein the continuous frames include multiple consecutive frames of images on the time axis; a proposal box generation unit 120 for generating initial proposal boxes; a spatiotemporal expansion unit 130 for predicting the positions of adjacent proposal boxes based on the current proposal box position to form a trajectory pipeline; a spatial refinement unit 140 for obtaining classification probabilities based on the spatiotemporal relationship of regional features through a dual-branch network, and determining regression boxes based on global and local information through a local branch; and a trajectory update unit 150 for replacing the proposal box with the regression box with the highest classification probability to update the trajectory pipeline.

[0106] In some embodiments, the suggestion box generation unit 120 generates an initial suggestion box in the following manner: based on a preset scale and overlap, an initial suggestion box is generated.

[0107] In some embodiments, the spatiotemporal extension unit 130 forms the trajectory pipeline by progressively extending the suggestion box based on time sequence, wherein each step s satisfies 1 <s<S max Furthermore, the trajectory pipeline B containing K frames of images s Expanded to in Indicate the connection; determine the regression function as L. s = f(x), where f is the input and x is the feature, and the formula for predicting the location is: Among them, f -1 and f +1For regression parameters; based on the formula for predicted location, determine adjacent trajectory pipelines. and

[0108] In some embodiments, the spatial refinement unit 140 uses a dual-branch network to obtain classification probabilities based on the spatiotemporal relationship of regional features through the global branch, and determines regression boxes based on global and local information through the local branch: the regional features generated by mapping the trajectory pipeline to the feature map are input into the dual-branch network; spatiotemporal modeling of regional features is performed based on the global branch to obtain classification probabilities; and global and local information are fused based on the local branch, and the updated regression boxes are obtained by combining the position of the proposal boxes.

[0109] In some embodiments, the trajectory update unit 150 updates the trajectory pipeline by replacing the proposal box with the regression box that has the highest classification probability, based on the following formula:

[0110]

[0111] Where 'c' represents the category, This represents the probability distribution of the i-th suggestion box across c categories and the background class. This represents the bounding box regression parameters for each class and each frame. This represents decoding the coordinates.

[0112] In some embodiments, the dual-branch network includes an attention mechanism module.

[0113] In some embodiments, the attention mechanism module includes a spatiotemporal attention module, a channel attention module, and a motion attention module. The spatiotemporal attention module highlights the feature information of the target; the channel attention module enhances the temporal correlation of the network convolutional channels; and the motion attention module highlights the motion information of the target.

[0114] Regarding the anti-interference device 100 based on spatiotemporal progressive learning in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments of the anti-interference method based on spatiotemporal progressive learning, and will not be elaborated here.

[0115] like Figure 12As shown, one embodiment of the present invention provides an electronic device 400. The electronic device 400 includes a memory 401, a processor 402, and an input / output (I / O) interface 403. The memory 401 is used to store instructions. The processor 402 is used to execute the target recognition method of the present invention by calling the instructions stored in the memory 401. The processor 402 is connected to both the memory 401 and the I / O interface 403, for example, via a bus system and / or other forms of connection mechanism (not shown). The memory 401 can be used to store programs and data, including the program for the target recognition method involved in the embodiments of the present invention. The processor 402 executes various functional applications and data processing of the electronic device 400 by running the program stored in the memory 401.

[0116] In this embodiment of the invention, the processor 402 can be implemented using at least one of the following hardware forms: digital signal processor (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 402 can be one or a combination of several of the following: central processing unit (CPU) or other processing units with data processing capability and / or instruction execution capability.

[0117] The memory 401 in this embodiment of the invention may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD).

[0118] In this embodiment of the invention, the I / O interface 403 can be used to receive input instructions (such as numerical or character information, and to generate key signal inputs related to user settings and function control of the electronic device 400), and can also output various information (such as images or sounds) to the outside. In this embodiment of the invention, the I / O interface 403 may include one or more of the following: a physical keyboard, function keys (such as volume control keys, power buttons, etc.), a mouse, a joystick, a trackball, a microphone, a speaker, and a touch panel.

[0119] It is understood that although operations are described in a specific order in the accompanying drawings in the embodiments of the present invention, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0120] The methods and apparatuses involved in the embodiments of the present invention can be implemented using standard programming techniques, and various method steps can be implemented using rule-based logic or other logic. It should also be noted that the terms "apparatus" and "module" as used herein and in the claims are intended to include implementations using one or more lines of software code and / or hardware implementations and / or devices for receiving input.

[0121] Any step, operation, or procedure described herein may be performed or implemented using one or more hardware or software modules, either alone or in combination with other devices. In one embodiment, the software module is implemented using a computer program product comprising a computer-readable medium containing computer program code, which is executable by a computer processor to perform any or all of the described steps, operations, or procedures.

[0122] The foregoing description of embodiments of the invention has been provided for purposes of illustration and description. The foregoing description is not exhaustive and is not intended to limit the invention to the exact forms described; various modifications and variations may be made in accordance with the foregoing teachings, or may be derived from the practice of the invention. These embodiments were chosen and described to illustrate the principles of the invention and its practical application, enabling those skilled in the art to utilize the invention in various embodiments and with various modifications to suit the particular purpose of the concept.

Claims

1. An anti-interference method based on spatiotemporal progressive learning, characterized in that, The anti-interference method based on spatiotemporal progressive learning includes: Acquire consecutive frames of the target to be detected, wherein the consecutive frames include multiple consecutive frames of images on the time axis; Generate the initial suggestion box; Based on the current suggestion box position, predict the positions of adjacent suggestion boxes to form a trajectory pipeline, including: performing stepwise temporal expansion of the suggestion boxes based on time series, wherein each step s satisfies 1 <s<S max Furthermore, the trajectory pipeline B containing K frames of images s Expanded to in Indicate the connection; determine the regression function as L. s = f(x), where f is the input and x is the feature, and the formula for predicting the location is: Among them, f -1 and f +1 The regression value is used; based on the formula for the predicted position, adjacent trajectory pipelines are determined. and The dual-branch network obtains classification probabilities based on the spatiotemporal relationships of regional features using the global branch, and determines regression boxes based on global and local information using the local branch. This includes: inputting the regional features generated by mapping the trajectory pipeline to a feature map into the dual-branch network; performing spatiotemporal modeling on the regional features based on the global branch to obtain the classification probabilities; and fusing global and local information based on the local branch, combined with the position of the proposed bounding boxes, to obtain updated regression boxes. Replace the suggestion box with the regression box that has the highest classification probability, and update the trajectory pipeline, including updating the trajectory pipeline based on the following formula: Where 'c' represents the category, This represents the probability distribution of the i-th suggestion box across c categories and the background class. This represents the bounding box regression parameters for each class and each frame. This represents decoding the coordinates.

2. The anti-interference method based on spatiotemporal progressive learning according to claim 1, characterized in that, The generation of the initial suggestion box includes: generating the initial suggestion box based on a preset scale and overlap.

3. The anti-interference method based on spatiotemporal progressive learning according to claim 1, characterized in that, The dual-branch network includes an attention mechanism module.

4. The anti-interference method based on spatiotemporal progressive learning according to claim 3, characterized in that, The attention mechanism module includes a spatiotemporal attention module, a channel attention module, and a motion attention module, wherein... Based on the spatiotemporal attention module, the feature information of the target is highlighted; Based on the aforementioned channel attention module, the temporal relevance of network convolutional channels is enhanced; Based on the motion attention module, the motion information of the target is highlighted.

5. An anti-interference device based on spatiotemporal progressive learning, characterized in that, For implementing the anti-interference method based on spatiotemporal progressive learning as described in any one of claims 1-4, the anti-interference device based on spatiotemporal progressive learning comprises: An acquisition unit is used to acquire consecutive frames of the target to be detected, wherein the consecutive frames include multiple consecutive frames of images on the time axis; The suggestion box generation unit is used to generate the initial suggestion box; The spatiotemporal extension unit is used to predict the positions of adjacent suggestion boxes based on the current suggestion box position, forming a trajectory pipeline; The spatial refinement unit is used to obtain the classification probability based on the spatiotemporal relationship of regional features through a dual-branch network, and to determine the regression box based on global and local information through the local branch. Update the trajectory unit to replace the proposal box with the regression box with the highest classification probability, and update the trajectory pipeline.

6. An electronic device, wherein, The electronic device includes: Memory, used to store instructions; and A processor is configured to invoke instructions stored in the memory to execute the anti-interference method based on spatiotemporal progressive learning as described in any one of claims 1-4.

7. A computer-readable storage medium storing instructions that, when executed by a processor, perform the anti-interference method based on spatiotemporal progressive learning as described in any one of claims 1-4.

Images