Anti-interference method based on detection cascade, motion analysis, timing analysis network

Abstract

The application relates to the field of precise guidance, in particular to an anti-interference method based on a detection cascade, motion analysis and time sequence analysis network, an anti-interference device based on the detection cascade, motion analysis and time sequence analysis network and electronic equipment. The method comprises the following steps: acquiring continuous frames of a target to be detected, wherein the continuous frames comprise multiple frames of images continuously arranged on a time axis; confirming a track pipeline of the target to be detected in the continuous frames, wherein the track pipeline comprises a detection frame of the target to be detected in each image; determining a motion classification result corresponding to the track pipeline based on a motion feature of the target to be detected; determining an appearance classification result corresponding to the track pipeline based on an image of the target to be detected; and determining a detection result of the target to be detected based on the motion classification result and the appearance classification result. The application judges and analyzes the motion feature of the target and the feature of the image respectively, and obtains a final target detection result by comprehensively combining the classification results of the two, so that the anti-interference ability of single-frame target recognition is avoided.

Description

Technical Field

[0001] This invention relates to the field of precision guidance, and in particular to an anti-interference method, an anti-interference device, and an electronic device based on a detection cascade, motion analysis, and timing analysis network. Background Technology

[0002] In precision-guided technology, infrared imaging guidance utilizes the different thermal radiation distributions of the target and background to achieve precise strikes, possessing the ability to autonomously search, track, and identify targets in complex tactical environments. Infrared guidance boasts advantages such as high spatial resolution, high sensitivity, and the ability to operate day and night, making it the mainstream development direction for precision-guided technologies worldwide. However, with the widespread use of infrared-guided weapons, corresponding infrared jamming technologies are also constantly evolving. This leads to decreased guidance accuracy, an increased probability of misidentification of jammed targets, and insufficient anti-jamming capabilities of existing guidance technologies. Summary of the Invention

[0003] To address the aforementioned problems, this invention provides an anti-interference method based on detection cascade, motion analysis, and time series analysis networks, an anti-interference device based on detection cascade, motion analysis, and time series analysis networks, an electronic device, and a computer-readable storage medium.

[0004] In a first aspect, the present invention provides an anti-interference method based on a detection cascade, motion analysis, and temporal analysis network, comprising: acquiring consecutive frames of a target to be detected, wherein the consecutive frames include multiple consecutive frames of images on a time axis; identifying a trajectory pipeline of the target to be detected in the consecutive frames, wherein the trajectory pipeline includes a detection box of the target to be detected in each image; determining a motion classification result corresponding to the trajectory pipeline based on the motion features of the target to be detected; determining an appearance classification result corresponding to the trajectory pipeline based on the image of the target to be detected; and determining a detection result of the target to be detected based on the motion classification result and the appearance classification result.

[0005] In some embodiments, confirming the trajectory pipeline of the target to be detected in the consecutive frames includes: determining the detection box of the target to be detected in each image based on the consecutive frames; and associating the detection boxes to obtain the trajectory pipeline.

[0006] In some embodiments, associating the detection boxes to obtain the trajectory pipeline includes: obtaining an initial trajectory based on the detection boxes; predicting the predicted box of the current frame based on the initial trajectory; matching the predicted box of the current frame with the detection box of the current frame; if the match is successful, confirming the association and updating the trajectory pipeline.

[0007] In some embodiments, determining the motion classification result corresponding to the trajectory pipeline based on the motion features of the target to be detected includes: determining a temporal feature vector based on the motion features, wherein the motion features include one or more of the following features of the target to be detected: velocity, acceleration, aspect ratio, curvature; extracting features from the temporal feature vector to obtain temporal features, and obtaining the motion classification result based on the features.

[0008] In some embodiments, determining the appearance classification result corresponding to the trajectory pipeline based on the image of the target to be detected includes: extracting image features based on the detection boxes of the target to be detected to obtain appearance information corresponding to each detection box; and determining the appearance classification result based on the appearance information corresponding to each detection box of the trajectory pipeline.

[0009] In some embodiments, the step of extracting image features based on the detection boxes of the target to be detected to obtain appearance information corresponding to each detection box includes: normalizing each detection box; and extracting image features through a convolutional neural network to obtain appearance information corresponding to each detection box.

[0010] In some embodiments, determining the appearance classification result based on the appearance information corresponding to each detection box of the trajectory pipeline includes: classifying the appearance information corresponding to each detection box of the trajectory pipeline based on time sequence through a long short-term memory network, and obtaining the appearance classification result.

[0011] In some embodiments, determining the detection result of the target to be detected based on the motion classification result and the appearance classification result includes: determining a motion classification factor based on the motion classification result and the motion classification weight; determining an appearance classification factor based on the appearance classification result and the appearance classification weight; and obtaining the detection result based on the motion classification factor and the appearance classification factor.

[0012] Secondly, the present invention also provides an anti-interference device based on a detection cascade, motion analysis, and temporal analysis network, comprising: an acquisition unit for acquiring continuous frames of a target to be detected, wherein the continuous frames include multiple consecutive frames of images on a time axis; a confirmation unit for confirming the trajectory pipeline of the target to be detected in the continuous frames, wherein the trajectory pipeline includes a detection box of the target to be detected in each image; a first analysis unit for determining a motion classification result corresponding to the trajectory pipeline based on the motion features of the target to be detected; a second analysis unit for determining an appearance classification result corresponding to the trajectory pipeline based on the image of the target to be detected; and a detection unit for determining a detection result of the target to be detected based on the motion classification result and the appearance classification result.

[0013] Thirdly, the present invention also provides an electronic device, comprising: a memory for storing instructions; and a processor for calling the instructions stored in the memory to execute the anti-interference method based on detection cascade, motion analysis, and timing analysis network as described in the first aspect.

[0014] Fourthly, the present invention also provides a computer-readable storage medium storing instructions that, when executed by a processor, perform the anti-interference method based on a detection cascade, motion analysis, and timing analysis network as described in the first aspect.

[0015] The technical solution provided by the embodiments of the present invention may include the following beneficial effects: by judging and analyzing the motion features of the target and the features of the image respectively, and combining the classification results of the two to obtain the final target detection result, the poor anti-interference ability of single-frame target recognition is avoided.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0017] Figure 1 A schematic flowchart of an anti-interference method based on detection cascade, motion analysis, and time series analysis network is shown in some embodiments.

[0018] Figure 2 Schematic diagrams of anti-interference methods based on detection cascade, motion analysis, and time series analysis networks are shown in some other embodiments;

[0019] Figure 3 Schematic diagrams of anti-interference methods based on detection cascade, motion analysis, and time series analysis networks are shown in some other embodiments;

[0020] Figure 4 The diagram shows two sets of detection results from some embodiments;

[0021] Figure 5 Schematic diagrams of cascaded test results for some embodiments are shown;

[0022] Figure 6 Schematic diagrams of 1DCNN network structures from some embodiments are shown;

[0023] Figure 7 The diagram illustrates the training process of a 1DCNN network in some embodiments;

[0024] Figure 8 A schematic diagram of the CLSTM network structure of some embodiments is shown;

[0025] Figure 9 The diagram illustrates the training process of a CLSTM network in some embodiments;

[0026] Figure 10 Schematic diagrams of fusion test results for some embodiments are shown;

[0027] Figure 11 Schematic diagrams of target recognition device modules from some embodiments are shown;

[0028] Figure 12 A schematic diagram of an electronic device is shown. Detailed Implementation

[0029] The invention will now be discussed with reference to several exemplary embodiments. It should be understood that these embodiments are described merely to enable those skilled in the art to better understand and thus implement the invention, and are not intended to imply any limitation on the scope of the invention.

[0030] As used herein, the term "comprising" and its variations are to be interpreted as open-ended terms meaning "including but not limited to". The term "based on" is to be interpreted as "at least partially based on". The terms "one embodiment" and "an embodiment" are to be interpreted as "at least one embodiment". The term "another embodiment" is to be interpreted as "at least one other embodiment".

[0031] Currently, deep learning technology has become a hot research area in the field of image processing, but research on its application in anti-interference tasks is scarce. In some techniques, the performance of single-frame candidate target extraction and inter-frame concatenation in detection cascade networks severely restricts their anti-interference capabilities. This makes the algorithm's performance highly dependent on the accuracy of the detection cascade network. If the detection cascade network performs poorly, lacks sufficient computational power, or has inadequate training results, it cannot complete subsequent classification tasks.

[0032] To solve the above problems, such as Figure 1 As shown, this embodiment of the invention provides an anti-interference method based on detection cascade, motion analysis, and temporal analysis networks for target detection, which may include steps S11-S15, as detailed below.

[0033] Step S11: Acquire consecutive frames of the target to be detected, wherein the consecutive frames include multiple consecutive frames of images on the time axis. In this invention, images of the target can be acquired using infrared or other imaging devices. Single-frame images are prone to misidentification; therefore, multiple consecutive frames are used to determine the target from the consecutive images, and identification is performed based on motion information and image information respectively.

[0034] In some embodiments, the anti-interference data may include different interference release methods, such as four-shot bursts or dual-shot bursts; it may also include different target movement methods, such as turning, acceleration, and other escape maneuvers. The anti-interference test data may consist of 50 sets, each set of data may be continuous frames or video, including 160 frames of images, and the size of each frame of image may be normalized, such as 400×400 pixels.

[0035] Step S12: Confirm the trajectory pipeline of the target to be detected in consecutive frames, wherein the trajectory pipeline includes the detection box of the target to be detected in each image. This invention can construct a detection cascade network. In some embodiments, Faster R-CNN (Fast Region Convolutional Neural Network) can be used as the detector to detect the position information of the target to be detected in each frame image, and Deep SORT (Deep Simple Online and Realtime Tracking) can be used as the trajectory cascader. After passing through the detection cascade network, the trajectory pipeline of each object (target to be detected) on the time axis is obtained.

[0036] Step S13: Based on the motion features of the target to be detected, determine the motion classification result corresponding to the trajectory pipeline. This invention extracts and classifies features based on the motion features of the target. Since the motion trajectories of the target and interfering targets differ, the motion of the objects can assist in classification. In some embodiments, a 1DCNN (One Dimensional Convolutional Neural Networks) can be constructed to calculate the motion information of each object in each frame, extract features, and perform classification.

[0037] In some embodiments, step S13, determining the motion classification result corresponding to the trajectory pipeline based on the motion features of the target to be detected, may include: determining a temporal feature vector based on the motion features, wherein the motion features include one or more of the following features of the target to be detected: velocity, acceleration, aspect ratio, and curvature; extracting features from the temporal feature vector to obtain temporal features, and obtaining the motion classification result based on the features. In this embodiment, the velocity, acceleration, aspect ratio, and curvature of each object in each frame can be calculated using a constructed 1DCNN network to construct a trajectory feature matrix M, obtaining the temporal feature vector of the trajectory pipeline. The temporal features of the trajectory pipeline are then input into the 1DCNN to classify the trajectory pipeline, and the classification probability is output.

[0038] Step S14: Based on the image of the target to be detected, determine the appearance classification result corresponding to the trajectory pipeline. This invention can also judge and classify the target to be detected based on the target's appearance features in the image.

[0039] In some embodiments, such as Figure 2 As shown, step S14, determining the appearance classification result corresponding to the trajectory pipeline based on the image of the target to be detected, may include: step S141, extracting image features based on the detection boxes of the target to be detected to obtain appearance information corresponding to each detection box; step S142, determining the appearance classification result based on the appearance information corresponding to each detection box of the trajectory pipeline. In this embodiment, a CLSTM (Contextual Long Short-Term Memory) network can be built, consisting of a feature extraction network and a temporal classification network. The trajectory pipeline obtained by the detection cascade network is used as input, and each frame is input into the feature extraction network once to extract spatial features of the object's information in each frame to describe the object's appearance information in each frame. Then, the features extracted in each frame are input into the LSTM (Long Short-Term Memory) network unit to learn the temporal changes in appearance features. The LSTM unit at each time step outputs a prediction probability, and the output of the last frame is used as the classification result of the entire trajectory pipeline.

[0040] Step S15: Based on the motion classification result and the appearance classification result, determine the detection result of the target to be detected. In this embodiment of the invention, the motion classification result and the appearance classification result can be comprehensively judged to obtain a more accurate and interference-resistant detection result. In some embodiments, the classification probabilities of the target and interference obtained from the 1DCNN network and the CLSTM network can be weighted and fused to obtain the final classification result.

[0041] Existing anti-interference algorithms only utilize the dynamic trajectory information of the target and interference, without deeply mining the motion patterns. This invention provides an anti-interference algorithm based on a detection cascade-motion feature analysis-appearance temporal analysis (DC-1DCNN-CLSTM) network. The front end first uses a detection cascade network (DC) composed of a Faster-RCNN detection network and a Deep-SORT cascade algorithm to detect the object's position and associate the object's detection boxes along the time axis to obtain the object's trajectory pipeline. Through analysis of the trajectory pipeline, two network branches based on different information are designed: Network branch one, the 1DCNN branch, utilizes the temporal positional change information of the trajectory pipeline, using changes in motion speed, acceleration, curvature, and aspect ratio as factors to construct a temporal feature vector, which is input into the 1DCNN network to classify the trajectory pipeline; Network branch two, the CLSTM network branch, utilizes the temporal appearance changes of the trajectory pipeline to classify the trajectory pipeline. The CLSTM network first inputs each frame of the object's trajectory pipeline into a convolutional neural network to extract features, and then inputs the features into an LSTM to achieve classification. By comprehensively considering both features and combining the data fusion module to weightedly fuse the classification probabilities of the two networks, the final classification result is obtained, which has good anti-interference ability.

[0042] In some embodiments, such as Figure 3 As shown, step S12, confirming the trajectory pipeline of the target to be detected in consecutive frames, may include: step S121, determining the detection box of the target to be detected in each image based on consecutive frames; step S122, associating the detection boxes to obtain the trajectory pipeline. In this embodiment of the invention, the trajectory pipeline of the same target in consecutive frames can be obtained by associating the detection boxes of each image, which facilitates subsequent analysis of the target's motion and image features.

[0043] In some embodiments, step S122, associating the detection boxes to obtain a trajectory pipeline, may include: obtaining an initial trajectory based on the detection boxes; predicting the predicted box of the current frame based on the initial trajectory; matching the predicted box of the current frame with the detection box of the current frame; if the match is successful, confirming the association and updating the trajectory pipeline.

[0044] In this embodiment of the invention, step S12 can specifically determine the trajectory pipeline in the following manner.

[0045] First, a detection cascade network can be constructed. In practice, the network can be pre-built. Faster R-CNN can be used as the detector in the detection cascade network, and Deep SORT can be used to associate the trajectories.

[0046] Then, the recognized image can be input into the Faster R-CNN network. The feature map of the image is extracted using a combination of convolutional layers (conv+relu+pooling), which is used in subsequent Region Proposal Network (RPN) and RoIPooling (Region of Interest Pooling) layers. Next, the RPN layer generates initial candidate boxes. By calculating the IoU (Intersection-over-Union) with the ground truth location of the object, the initial candidate boxes are divided into foreground and background. Then, bounding box regression is used to refine the candidate boxes. The RoI pooling layer maps the detection boxes onto the feature map to obtain feature vectors, which are then connected to a fully connected layer for classification. Bounding box regression obtains the precise location of the detection boxes.

[0047] Next, the Deep SORT multi-target tracking algorithm can be used to associate detection boxes across frames. The Deep SORT algorithm uses an 8-dimensional state space to describe the state at a given moment: (u,v,g,h,x*,y*,g*,h*), where (u,v) represents the center coordinates of the target box, g represents the aspect ratio, h represents the height, and (x*,y*,g*,h*) represents the velocity corresponding to each variable in the current frame's image coordinate system. For target state prediction, a Kalman filter based on a constant velocity model and a linear observation model is used, and the result is represented by (u,v,g,h). For each trajectory, the time from the last successful match to the current moment is recorded. When this value is greater than or equal to a set threshold A... max If a match is not found, the trajectory is terminated. Unmatched trajectories are marked as tentative, and their success in the subsequent N consecutive frames is observed (N can be 3 or other values). If a match is found, a new trajectory is considered to have appeared, and the state changes to confirmed; otherwise, the trajectory is considered a false trajectory, and the state changes to deleted.

[0048] Next, trajectories with states of "confirmed" and "tentative" need to be matched with the detection boxes in the current frame. For motion matching accuracy, Mahalanobis local distance can be used to characterize the matching degree between the trajectory and the detection box, and its formula is:

[0049] d (1) (i, j) = (d j -y i )T S i -1 (d j -y i )

[0050] In the formula, d j y represents the position of the j-th detection box. i S represents the predicted position of the i-th trajectory. i This represents the covariance matrix between the two.

[0051] After that, it can be done through formulas The calculated Mahalanobis distance is used to filter the detection boxes. If the Mahalanobis distance of a given association is less than a specified threshold t... (1) =9.4877. The associated state is as shown in the formula:

[0052]

[0053] Considering the randomness of motion, using only a Kalman filter to evaluate motion states is not very accurate. Therefore, a second metric is introduced: for each detection box, a vector r describing its features is calculated using a convolutional neural network. j (||r j ||=1), store the most recent L k =20 trajectory descriptors, i.e. Calculate the i-th trajectory R k With the current j-th detection box r j The minimum cosine distance. The formula is:

[0054]

[0055] If d (2) (i, j) is less than the specified threshold t (2) If the association is successful, the association is successful; otherwise, it fails. The association status is shown in the formula:

[0056]

[0057] The two indicators are combined using a weighted average, and the formula is as follows:

[0058] c i，j =ld (1) (i,j)+(1-l)d (2) (i, j)

[0059] When d (1) (i, j), d (2) The association is considered successful only when (i, j) simultaneously meet the threshold requirements.

[0060] In this embodiment of the invention, when two trajectories are competing for the matching right of the same detection box, the trajectory whose position information has not been updated for a long time has a greater randomness in its tracking position and a smaller Mahalanobis distance. The detection box is more likely to be associated with the trajectory whose position has not been updated for a long time. Cascaded matching is used to prioritize matching the frequently occurring target.

[0061] During model training, the original anti-interference data can be divided into training and test sets in groups of 16 frames each. The training epochs can be set to 20, the batch size to 2, the initial learning rate to 0.001, and the loss function to be the cross-entropy loss function. mAP (mean Average Precision) can be used as the evaluation metric for the detection algorithm.

[0062] like Figure 4 The image shown is a schematic diagram of the detection results for two sets of images. Figure 4 (a) is a schematic diagram of the target detection without any maneuvering. Figure 4 Figure (b) is a schematic diagram illustrating the detection of target pose changes. As can be seen from the figure, the Faster-RCNN network can detect almost all targets and disturbances in the environment of this invention. Figure 4 The detection results in (b) demonstrate that the target can still be detected even when its attitude changes. From... Figure 4 The third image in (b) shows a small detection box marked on the lower right of the target aircraft. This is the residue of the chaff, which was detected because its appearance is similar to that of the chaff. Figure 4 The second image in (a) and Figure 4 In the second image of (b), the recently released interference was detected, and there was partial overlap with the detection bounding box of the target aircraft, but... Figure 4 In Figure (b) 3, the newly released interference was not detected. The reason for this is that the interference in the former has essentially been released, and its appearance characteristics are largely formed, while the latter is still in the release phase, and its appearance characteristics are not yet fully manifested. Interference will gradually reveal its characteristics over time, so this does not affect subsequent work.

[0063] mAP was used to evaluate the detection performance. Experiments showed that Faster R-CNN achieved a detection rate of 98% in the anti-interference task, which strongly supported subsequent recognition and other tasks.

[0064] like Figure 5 The results of the cascade test are shown below. Figure 5 (a) shows the normal association case. Figure 5 (b) shows the sequence where skipped numbers occur. For example... Figure 5As shown in (a), each target and interference cascades well and is assigned consecutive ID numbers. Figure 5 As shown in (b), there is no trajectory with ID=3 in the third image. This is because the newly appearing interference was not associated when it was released. In the cascading algorithm, if three consecutive frames are not associated, it is considered not to constitute a trajectory, but a number will be assigned. When the detection box is regarded as a new trajectory, it is assigned the number ID=4. This situation occasionally occurs in anti-interference tasks when the target releases interference, but it is relatively rare and has little impact on subsequent work.

[0065] In this embodiment of the invention, a temporal feature vector T can be used to characterize the changes in the motion characteristics of an object and input into a classification model. The temporal feature vector refers to a vector composed of attributes that change over time for the target or interference. In the temporal imaging analysis of both targets and interference, the temporal differences between the two types of objects lie in their motion velocity, acceleration, and curvature, which are determined by the motion properties of the target and the chaff. Analysis of the detection frames for the target and the chaff reveals their different aspect ratios. Motion velocity, acceleration, and curvature are used to characterize the motion characteristics of the object, while the aspect ratio is used to characterize its appearance.

[0066] Motion speed indicates the displacement of an object per unit time. Here, displacement refers to the displacement of the image position between frames, as shown in the formula below:

[0067]

[0068] Where, x i+1 x i Let t represent the center coordinates of the detection box of the object in frame i and frame (i+1), respectively. i+1 , t i represents the imaging time of the object in the i-th frame and the (i+1)-th frame, respectively, and dis represents the Euclidean distance between the two points.

[0069] Acceleration represents how quickly an object's velocity changes. Similarly, the acceleration here corresponds to the position in the image, as shown in the formula below:

[0070]

[0071] The aspect ratio represents the geometric properties of an object, and the formula is as follows:

[0072]

[0073] Where w i h i These represent the length and width of the object detection box, respectively.

[0074] Curvature represents the degree of bending of the trajectory of an object during motion, and its formula is as follows:

[0075]

[0076] In this embodiment of the invention, a 1DCNN network can be built, such as... Figure 6 As shown, the velocity, acceleration, aspect ratio, and curvature of each object in each frame are calculated to construct a trajectory feature matrix M, obtaining the temporal feature vector of the trajectory pipeline. This temporal feature vector is then input into a 1DCNN network to classify the trajectory pipeline and output the classification probability. Specific implementation methods may include:

[0077] First, the velocity, acceleration, aspect ratio, and curvature of each object in each frame can be calculated to construct the trajectory feature matrix M, expressed by the following formula:

[0078]

[0079] Then, the matrix M can be expanded row-wise into a 1-dimensional vector to form the time series feature vector T, as expressed by the following formula:

[0080] T = [v1, v2, ..., v n a1, a2, ..., a n h1, h2, ..., h n s1, s2, ..., s n ]

[0081] Complete the construction of the temporal feature vector of the trajectory pipeline.

[0082] The 1DCNN network structure can then consist of three one-dimensional convolutional layers (Conv1d), one dropout layer, and one fully connected layer. The three one-dimensional convolutional layers can use ReLU (Linear Rectification Function) as the activation function, and the softmax function can be used after the fully connected layer to calculate the classification probability of the target and the interference.

[0083] Next, three one-dimensional convolutional layers can be used to extract features from the temporal feature vector. After the one-dimensional convolutional layer (c), a dropout layer is connected. Finally, the network is fully connected, which combines the aforementioned local features and outputs the classification probability.

[0084] The 1DCNN network structure consists of three one-dimensional convolutional layers followed by dropout layers, and finally a fully connected layer that outputs classification probabilities via softmax. The class with the highest probability is taken as the classification result.

[0085] In this embodiment of the invention, the convolutional kernel size can be 3, the training epochs can be 14, the dropout ratio can be set to 0.5, classification accuracy can be used as the evaluation metric, and the loss function can be the cross-entropy loss function. The training process is as follows: Figure 7 As shown. Figure 7 (a) is the loss curve during the training process. Figure 7 (b) is the classification accuracy curve during the training process. As can be seen from the figure, as the network is trained, the loss gradually converges and the classification accuracy gradually stabilizes.

[0086] In some embodiments, step S141, which involves extracting image features based on the detection boxes of the target to be detected to obtain appearance information corresponding to each detection box, may include: normalizing each detection box; and extracting image features through a convolutional neural network to obtain appearance information corresponding to each detection box.

[0087] In some specific embodiments of the present invention, a CLSTM network can be built, such as... Figure 8 As shown, the system consists of a feature extraction network and a temporal classification network. The trajectory pipeline obtained from the detection cascade network is used as input, and each frame is sequentially input into the feature extraction network. Spatial features are extracted from the information of the object in each frame to describe the object's appearance information in each frame. The specific process is as follows:

[0088] First, a CLSTM network can be built or pre-built, which can be mainly divided into feature extraction network and temporal classification network.

[0089] Then, a ResNet14 (Residual Network) feature extraction network can be constructed or pre-constructed as a CNN feature extraction network. It consists of 14 parametric layers, including 13 convolutional layers and 1 fully connected layer. The first two layers of the network are a convolutional layer and a pooling layer, followed by three layer layers. Each layer layer includes two BasicBlock structures. The BasicBlock can be connected by shortcuts using two convolutional layers.

[0090] Next, each frame in each trajectory pipeline of the object output by the detection cascade network is resized, which is also a form of image normalization, to ensure that the input size of the CNN remains constant.

[0091] Then, each frame of the size-processed trajectory pipeline can be input into the feature extraction network. The CNN extracts spatial features from the information of the object in each frame to describe the appearance information of the object in each frame.

[0092] In some embodiments, step S142, determining the appearance classification result based on the appearance information corresponding to each detection box of the trajectory pipeline, includes: classifying the appearance information corresponding to each detection box of the trajectory pipeline based on time sequence through a long short-term memory network, and obtaining the appearance classification result.

[0093] In a specific embodiment of this invention, the features extracted from each frame are input into an LSTM unit to learn the temporal changes in appearance features. The LSTM unit outputs a predicted probability at each time step, and the output of the last frame is used as the classification result for the entire trajectory pipeline. The specific process is as follows:

[0094] First, a temporal classification network can be built or pre-built. An LSTM network can be used to extract and fuse features from each frame of the trajectory pipeline in the CNN network structure for classification. Key variables in the LSTM include: the hidden layer h at time t-1. t-1 and the eigenvector x at time t t Output the hidden layer h at time t. t And the cellular t-1 time-series characteristics C transmitted in time sequence t-1 Features C at time t t The input h t-1 and x t Interaction, as the input part; output h t As the hidden layer input for the next time step, h t This can then be connected to a softmax function as the output for the current time step; this part becomes the output section. t-1 and C t It is passed sequentially and updated through the forget gate, input gate, and output gate.

[0095] The forget gate selectively retains features input from the previous time step, and can be expressed by the following formula:

[0096] f t =s(W f ×[h t-1 x t ]+b f )

[0097] Among them, W f b represents the weight. f Indicates bias.

[0098] The input gate represents the selective storage of new information at the current moment within the cell, including information i that needs to be updated. t And new candidate information C t It can be expressed by the following formula:

[0099] i t =s(Wi ×[h t-1 x t ]+b i )

[0100] C t =tanh(W c ×[h t-1 x t ]+b c )

[0101] Among them, W i W c b represents the weight. i b c Indicates bias.

[0102] The state of the LSTM cell units is then updated, which can be represented by the following formula:

[0103] C t =f t ×C t-1 +i t ×C t

[0104] It can be seen that the state update takes into account both the filtered information from the previous moment and the filtered information from the current moment.

[0105] The input gate represents the selection of partial information to be output for the current cell state, and can be expressed by the following formula:

[0106] o t =s(W o ×[h t-1 x t ]+b o )

[0107] h t =o t ×tanh[C t ]

[0108] Among them, W o b represents the weight. o Indicates bias.

[0109] Next, the parameters of the LSTM network can be specified, including the dimension of the hidden layers, the number of layers in the network, whether to use the dropout strategy, etc., and the h output of the last unit will be calculated. t After connecting to the softmax layer, the classification probabilities of the target and interference can be obtained.

[0110] For the CLSTM branch network, the input is the trajectory pipeline extracted by the detection cascade network. Sixteen frames are randomly extracted sequentially from each trajectory pipeline. Considering the inconsistent and relatively small size of the target and interference in the trajectory pipelines, the original image can be expanded outward by 1.2 times and then resized before being input into the feature extraction network. The final input image size can be 64×64 pixels. The feature extraction network can adopt a ResNet14 network structure. By changing the output dimension of the last fully connected layer, the feature map is used as the input to the LSTM network. That is, each trajectory pipeline inputs 16 images, and the feature vector dimension of each image is 128.

[0111] The relevant parameters of LSTM can be set based on experience or needs. For example, the LSTM hidden layer data can be set to 16, the number of LSTM network layers can be set to 1, and the LSTM can be set as a unidirectional network. Network training parameters can be set, including the number of training epochs (30), batch size (2), the loss function (cross-entropy loss), the learning rate (0.0005), and the classification standard error as the evaluation metric. The network training process can be as follows: Figure 9 As shown. From Figure 9 As can be seen, the network loss gradually decreases in the first 10 epochs, and after 10 epochs, the network loss tends to remain unchanged and no longer decreases significantly, indicating that the network has been trained.

[0112] In some embodiments, step S15, determining the detection result of the target to be detected based on the motion classification result and the appearance classification result, may include: determining a motion classification factor based on the motion classification result and the motion classification weight; determining an appearance classification factor based on the appearance classification result and the appearance classification weight; and obtaining the detection result based on the motion classification factor and the appearance classification factor.

[0113] In this embodiment of the invention, the classification probabilities of the target and interference obtained from the 1DCNN network and the CLSTM network are weighted and fused to obtain the final classification result. The specific method is as follows:

[0114] For the same trajectory pipeline, the motion information extracted by the 1DCNN network branches changes over time, and the output classification probability after softmax is P. o ={P o1 ,P o2 The corresponding weight is W. o ={W o1 W o2 The CLSTM network branches extract the changes in object appearance information over time, and after softmax, the output classification probability is P. t ={P t1 ,Pt2 The corresponding weight is W. t ={W t1 W t2}, where 1 represents the target and 2 represents interference. The final classification probability P after fusion can be formulated as follows:

[0115] P = W o ×P o +W t ×P t

[0116] Weight W o and W t The weights of similar objects are kept at 1. The weights can be adjusted by setting the weight step size d, and the weights that result in the highest classification accuracy are selected. The weights can be confirmed as follows:

[0117] Initialize object weights W o ={0, 0}, W t ={1, 1}.

[0118] Calculate the fusion probability P = W o ×P o +W t ×P t .

[0119] Calculate the fusion probability classification result label = max(P).

[0120] Calculate the classification accuracy under the current weights: n represents the number of samples in label = groundtruch, i.e., the number of correctly classified samples; N represents the total number of samples.

[0121] Update weight W o1 =W o1 +d, W o2 =W o2 +d, W t1 =W t1 -d, W t2 =W t2 -d, where d is the weight adjustment step size.

[0122] Determine if the weights are out of bounds.

[0123] Output the weight W corresponding to the highest probability of each object class. o and W t The final classification probabilities after fusion are obtained.

[0124] In this embodiment of the invention, a fusion experiment was designed with identical trajectory pipelines. For each trajectory pipeline, 16 images were randomly selected sequentially, and the corresponding temporal feature vectors were calculated. These vectors were then fed into a CLSTM network and a 1DCNN network to calculate the classification probability. The initial weights of the 1DCNN network were set to W. o = {0, 0}, the initial weights W of the CLSTM network t ={1, 1}, and based on experience or actual needs, the weight adjustment step size can be set to d = 0.2. Calculate the probabilities after fusion, take the category corresponding to the highest probability as the classification result, and statistically analyze the classification accuracy. Experimental results can be shown as follows: Figure 10 As shown in the figure. The vertical axis represents classification accuracy, and the horizontal axis represents W. o , represents the weights of the 1DCNN network, the circled line represents the target classification accuracy, and the triangular line represents the interference classification accuracy.

[0125] from Figure 10 As can be seen, the sensitivity of the target and interference branches of the network differs. For the target, as W... o1 That is, as the weights of a 1DCNN network increase, the classification accuracy initially rises and then falls, at W... o1 The highest classification accuracy of 94.6% was achieved at W = 0.2. Regarding interference, as W... o1 With the increase in W, classification accuracy first rose and then fell. o1 The highest classification accuracy of 91.6% was achieved at a value of 0.8. Analysis of the experimental results suggests that the motion characteristics of interference are relatively fixed due to the constraints of the trajectory equation, while the motion characteristics of the target lack obvious regularity due to maneuvering. Therefore, the 1DCNN branch, which uses motion characteristics for classification, achieves higher accuracy in classifying interference. The target undergoes rich changes during maneuvering; therefore, the CLSTM network, which utilizes changes in appearance features, achieves higher classification accuracy for the target. However, 1DCNN and CLSTM can achieve even higher classification results through feature complementarity. Therefore, in this embodiment of the invention, a weight of W can be selected. o ={0.2, 0.8}, W t ={0.8, 0.2}.

[0126] The results of the 1DCNN branch network, CLSTM branch network and fusion experiment are used as a comparison experiment, and the results are shown in Table 1.

[0127] Table 1. Comparison of different experimental methods:

[0128]

[0129]

[0130] As shown in Table 1, using only the 1DCNN network achieves a classification accuracy of 90.6%, while using only the CLSTM network achieves 92.1%. Overall, the CLSTM network outperforms the 1DCNN, primarily because 1DCNN extracts motion features, and the temporal feature vectors extracted from two-dimensional images have limitations in characterizing three-dimensional motion. CLSTM, on the other hand, considers not only the appearance features of each frame but also the temporal changes in those features. However, by combining the 1DCNN and CLSTM network structures and adjusting the weights, the advantages of both feature types can be fully utilized, ultimately achieving a classification accuracy of 93.3%. Experiments demonstrate that the data fusion module of this invention, by comprehensively considering the temporal changes in dynamic and appearance features, significantly improves classification accuracy and possesses good anti-interference capabilities.

[0131] Based on the same inventive concept, the present invention also provides an anti-interference device 100 based on a detection cascade, motion analysis, and timing analysis network, such as... Figure 11 As shown, the anti-interference device 100 based on a detection cascade, motion analysis, and temporal analysis network may include: an acquisition unit 110 for acquiring continuous frames of the target to be detected, wherein the continuous frames include multiple consecutive frames of images on the time axis; a confirmation unit 120 for confirming the trajectory pipeline of the target to be detected in the continuous frames, wherein the trajectory pipeline includes the detection box of the target to be detected in each image; a first analysis unit 130 for determining the motion classification result corresponding to the trajectory pipeline based on the motion features of the target to be detected; a second analysis unit 140 for determining the appearance classification result corresponding to the trajectory pipeline based on the image of the target to be detected; and a detection unit 150 for determining the detection result of the target to be detected based on the motion classification result and the appearance classification result.

[0132] In some embodiments, the verification unit 120 may verify the trajectory pipeline of the target to be detected in consecutive frames in the following manner: based on consecutive frames, determine the detection box of the target to be detected in each image; associate the detection boxes to obtain the trajectory pipeline.

[0133] In some embodiments, the confirmation unit 120 may associate the detection boxes to obtain a trajectory pipeline in the following manner: obtain an initial trajectory based on the detection boxes; make a prediction based on the initial trajectory to obtain the predicted box of the current frame; match the predicted box of the current frame with the detection box of the current frame, and if the match is successful, confirm the association and update the trajectory pipeline.

[0134] In some embodiments, the first analysis unit 130 may determine the motion classification result corresponding to the trajectory pipeline based on the motion characteristics of the target to be detected in the following manner: based on the motion characteristics, determine the temporal feature vector, wherein the motion characteristics include one or more of the following features of the target to be detected: velocity, acceleration, aspect ratio, curvature; perform feature extraction on the temporal feature vector to obtain temporal features, and obtain the motion classification result based on the features.

[0135] In some embodiments, the second analysis unit 140 may determine the appearance classification result corresponding to the trajectory pipeline based on the image of the target to be detected in the following manner: extracting image features based on the detection box of the target to be detected to obtain the appearance information corresponding to each detection box; and determining the appearance classification result based on the appearance information corresponding to each detection box of the trajectory pipeline.

[0136] In some embodiments, the second analysis unit 140 may extract image features based on the detection boxes of the target to be detected to obtain appearance information corresponding to each detection box in the following manner: normalize each detection box; extract image features through a convolutional neural network to obtain appearance information corresponding to each detection box.

[0137] In some embodiments, the second analysis unit 140 may determine the appearance classification result based on the appearance information corresponding to each detection box of the trajectory pipeline in the following manner: based on the time series, the appearance information corresponding to each detection box of the trajectory pipeline is classified through a long short-term memory network, and the appearance classification result is obtained.

[0138] In some embodiments, the detection unit 150 may determine the detection result of the target to be detected based on the motion classification result and the appearance classification result in the following manner: determining the motion classification factor based on the motion classification result and the motion classification weight; determining the appearance classification factor based on the appearance classification result and the appearance classification weight; and obtaining the detection result based on the motion classification factor and the appearance classification factor.

[0139] Regarding the anti-interference device 100 based on detection cascade, motion analysis, and timing analysis network in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments of the anti-interference method based on detection cascade, motion analysis, and timing analysis network, and will not be elaborated here.

[0140] like Figure 12As shown, one embodiment of the present invention provides an electronic device 400. The electronic device 400 includes a memory 401, a processor 402, and an input / output (I / O) interface 403. The memory 401 is used to store instructions. The processor 402 is used to execute the target recognition method of the present invention by calling the instructions stored in the memory 401. The processor 402 is connected to both the memory 401 and the I / O interface 403, for example, via a bus system and / or other forms of connection mechanism (not shown). The memory 401 can be used to store programs and data, including the program for the target recognition method involved in the embodiments of the present invention. The processor 402 executes various functional applications and data processing of the electronic device 400 by running the program stored in the memory 401.

[0141] In this embodiment of the invention, the processor 402 can be implemented using at least one of the following hardware forms: digital signal processor (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 402 can be one or a combination of several of the following: central processing unit (CPU) or other processing units with data processing capability and / or instruction execution capability.

[0142] The memory 401 in this embodiment of the invention may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD).

[0143] In this embodiment of the invention, the I / O interface 403 can be used to receive input instructions (such as numerical or character information, and to generate key signal inputs related to user settings and function control of the electronic device 400), and can also output various information (such as images or sounds) to the outside. In this embodiment of the invention, the I / O interface 403 may include one or more of the following: a physical keyboard, function keys (such as volume control keys, power buttons, etc.), a mouse, a joystick, a trackball, a microphone, a speaker, and a touch panel.

[0144] It is understood that although operations are described in a specific order in the accompanying drawings in the embodiments of the present invention, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0145] The methods and apparatuses involved in the embodiments of the present invention can be implemented using standard programming techniques, and various method steps can be implemented using rule-based logic or other logic. It should also be noted that the terms "apparatus" and "module" as used herein and in the claims are intended to include implementations using one or more lines of software code and / or hardware implementations and / or devices for receiving input.

[0146] Any step, operation, or procedure described herein may be performed or implemented using one or more hardware or software modules, either alone or in combination with other devices. In one embodiment, the software module is implemented using a computer program product comprising a computer-readable medium containing computer program code, which is executable by a computer processor to perform any or all of the described steps, operations, or procedures.

[0147] The foregoing description of embodiments of the invention has been provided for purposes of illustration and description. The foregoing description is not exhaustive and is not intended to limit the invention to the exact forms described; various modifications and variations may be made in accordance with the foregoing teachings, or may be derived from the practice of the invention. These embodiments were chosen and described to illustrate the principles of the invention and its practical application, enabling those skilled in the art to utilize the invention in various embodiments and with various modifications to suit the particular purpose of the concept.

Claims

1. An anti-interference method based on detection cascade, motion analysis, and time series analysis networks, characterized in that, For target detection, the anti-interference method based on detection cascade, motion analysis, and temporal analysis networks includes: Acquire consecutive frames of the target to be detected, wherein the consecutive frames include multiple consecutive frames of images on the time axis; Confirm the trajectory pipeline of the target to be detected in the consecutive frames, wherein the trajectory pipeline includes the detection box of the target to be detected in each of the images; Based on the motion characteristics of the target to be detected, the motion classification result corresponding to the trajectory pipeline is determined, including: determining a temporal feature vector based on the motion characteristics, wherein the motion characteristics include one or more of the following features of the target to be detected: velocity, acceleration, aspect ratio, curvature; performing feature extraction on the temporal feature vector to obtain temporal features, and obtaining the motion classification result based on the features; Based on the image of the target to be detected, the appearance classification result corresponding to the trajectory pipeline is determined, including: normalizing each detection box; extracting image features through a convolutional neural network to obtain the appearance information corresponding to each detection box; and classifying the appearance information corresponding to each detection box of the trajectory pipeline through a long short-term memory network based on time sequence to obtain the appearance classification result. Based on the motion classification result and the appearance classification result, the detection result of the target to be detected is determined, including: determining a motion classification factor based on the motion classification result and the motion classification weight; determining an appearance classification factor based on the appearance classification result and the appearance classification weight; and obtaining the detection result based on the motion classification factor and the appearance classification factor.

2. The anti-interference method based on detection cascade, motion analysis, and time series analysis network according to claim 1, characterized in that, The process of confirming the trajectory pipeline of the target to be detected in the consecutive frames includes: Based on the consecutive frames, the detection box of the target to be detected in each image is determined; The detection boxes are associated to obtain the trajectory pipeline.

3. The anti-interference method based on detection cascade, motion analysis, and time series analysis network according to claim 2, characterized in that, The step of associating the detection boxes to obtain the trajectory pipeline includes: Based on the detection box, the initial trajectory is obtained; Based on the initial trajectory, a prediction box for the current frame is obtained; The predicted bounding box of the current frame is matched with the detection bounding box of the current frame. If the match is successful, the association is confirmed and the trajectory pipeline is updated.

4. An anti-interference device based on a detection cascade, motion analysis, and time series analysis network, characterized in that, The apparatus for implementing the anti-interference method based on detection cascade, motion analysis, and timing analysis network as described in any one of claims 1-3, comprises: An acquisition unit is used to acquire consecutive frames of the target to be detected, wherein the consecutive frames include multiple consecutive frames of images on the time axis; A confirmation unit is used to confirm the trajectory pipeline of the target to be detected in the consecutive frames, wherein the trajectory pipeline includes a detection box of the target to be detected in each image; The first analysis unit is used to determine the motion classification result corresponding to the trajectory pipeline based on the motion characteristics of the target to be detected. The second analysis unit is used to determine the appearance classification result corresponding to the trajectory pipeline based on the image of the target to be detected. The detection unit is used to determine the detection result of the target to be detected based on the motion classification result and the appearance classification result.

5. An electronic device, wherein, The electronic device includes: Memory, used to store instructions; and The processor is configured to invoke instructions stored in the memory to execute the anti-interference method based on detection cascade, motion analysis, and timing analysis network as described in any one of claims 1-3.

Images