A method for detecting extremely small targets in sky and space background based on video sequence feature matching

Abstract

The present invention belongs to the field of image processing technology, and specifically relates to a method for detecting extremely small targets in an air-sky background based on video feature matching. The method comprises the following steps: constructing a feature extraction network, taking a video sequence as input, extracting the position coordinates of key points contained in continuous frame images of the video sequence and the feature description vectors of the key points; constructing a feature matching network, extracting the obtained key point coordinates and key point feature description vectors, inputting them into the feature matching network for associative matching of continuous frames, and outputting key point pairs for continuous frame matching; constructing a secondary classifier based on the pixel value features and motion features of the target, further screening and classifying the successfully matched key point pairs, and obtaining extremely small targets in the video sequence. The present invention realizes the detection of extremely small targets with an effective pixel number of less than 12*12 in an air-sky background, effectively solving the problem that it is difficult for neural networks to characterize extremely small targets, and improving the precision and recall rate of extremely small target detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method for detecting extremely small targets in an air-sky background based on video feature matching. Background Art

[0002] Modern warfare has fully entered the information age. As countries around the world vigorously develop intelligent and digital weapons and equipment, new and diverse equipment has put forward higher and higher requirements for various defense countermeasures and reconnaissance detection. A large number of manned aircraft, unmanned aerial vehicles, missiles and other aerial firepower have been put into use in actual battlefields. In particular, small and medium-sized unmanned aerial vehicles and missiles can use low-altitude flight and stealth technology to avoid detection by traditional military sensors such as radar, which poses increasingly severe challenges and threats to traditional air defense detection systems. Therefore, it is urgent to adopt other more efficient and accurate detection methods to quickly and accurately detect, warn and strike flying targets in complex battlefield environments.

[0003] With the increasing maturity of video image target detection technology and its wide application in military and civilian fields, passive detection methods such as optoelectronic systems have become a more efficient and accurate means of detecting flying targets in the air and space background. However, in the complex and ever-changing battlefield space environment, due to the long distance between the imaging system and the target, the flying targets often appear in the image as extremely small target forms such as short strips and point clusters. The target has very few effective pixels (less than 12*12 pixels) and lacks effective identification characteristics. This has brought huge difficulties and challenges to the existing video image target detection.

[0004] At present, the industry is conducting relevant research on small target detection, including traditional algorithms and deep learning algorithms. In terms of traditional algorithms, small target detection is mainly achieved by using feature pyramids, contextual information, local spatial domain, frequency domain features and other methods. The Chinese patent document with publication number CN107705319A discloses a real-time detection and tracking method for small targets based on air and space background. The image is first segmented into multi-scale sub-domains, and small targets are detected by using local grayscale feature changes. Then, the target is weighted and tracked by the centroid to make up for the shortcomings of a single local grayscale feature. However, this method is insensitive to smaller targets and is easily affected by noise. Small target detection based on deep learning mainly improves detection performance through methods such as data enhancement and multi-scale feature fusion. The Chinese patent document with publication number CN113378905A discloses a small target detection method based on distribution distance, which uses the normalized Gaussian distance to measure the positional relationship between the prediction box and the annotation box, designs a small target detection network based on distribution distance to improve the quality of positive and negative sample allocation, designs a non-maximum suppression method based on distribution distance to improve the quality of non-maximum suppression, and designs a loss function based on distribution distance to guide the network to better regress, and the above items together improve the detection effect. However, this method is mainly applied to smaller targets (less than 32*32 pixels). For extremely small targets in the sky and space background, the ability and accuracy of target detection technology based on general neural networks are greatly affected, or even fail.

[0005] In general, the detection of extremely small targets in the sky and space background has the difficulties of few effective pixels, simple spatial structure, low target recognizability, and susceptibility to noise interference. Although the small target detection based on traditional algorithms has good timeliness, it has the disadvantages of low detection accuracy and recall rate due to its weak feature expression ability; the target detection technology based on deep learning has been widely used and relatively mature, but small target detection is still a difficulty and pain point in the field, especially for extremely small targets with a small number of effective pixels (less than 12*12 pixels), the target detection method based on deep learning is often helpless. This is because as the neural network downsamples, the deeper the feature layer, the larger the receptive field, and the spatial scale of the feature layer decreases accordingly. As a result, the extremely small targets with a small number of pixels will fail to express their features as the network downsamples. Summary of the invention

[0006] The purpose of the present invention is to provide a method for detecting extremely small targets in the air and space background based on video sequence feature matching, which utilizes the characteristics of large gradients of pixels between the target and the background, the correlation of targets in adjacent frames, the pixel value characteristics of the target and the motion characteristics of the target. By constructing a feature extraction network, a feature matching network, and a feature classifier, the target detection problem is converted into a feature matching and classification problem, thereby realizing the detection of extremely small targets in the air and space background.

[0007] To achieve the above objectives, the present invention adopts the following technical solutions:

[0008] A method for detecting extremely small targets in sky and space background based on video sequence feature matching comprises the following steps:

[0009] S1. Construct a feature extraction network, take the video sequence as input, and extract key point information of continuous frame images of the video sequence, wherein the key point information includes the position coordinates of the key point and the feature description vector of the key point;

[0010] S2, construct a feature matching network, input the key point coordinates and key point feature description vectors extracted in step S1 into the feature matching network for association matching of continuous frames, and output key point pairs matched by continuous frames;

[0011] S3. According to the pixel value characteristics and motion characteristics of the target, a secondary classifier is constructed to further screen and classify the successfully matched key point pairs. The final key points obtained are the extremely small targets in the video sequence.

[0012] In step S1, the feature extraction network constructed by the present invention includes an input layer, a feature encoding layer, a feature extraction layer, and an output layer connected in sequence; the feature extraction layer includes a key point feature extraction branch and a feature description extraction branch;

[0013] The feature coding layer uniformly encodes the continuous frame images of the video sequence input to the network to obtain a feature map containing key point feature information corresponding to each frame of the image; the key point feature extraction branch extracts the key point features of each feature map, and the feature description extraction branch extracts the key point feature description vector of each feature map to obtain the position coordinates and feature description vector of the key points contained in each frame of the image; the key points are image points that can be detected under different sources, different imaging modes, different viewing angles, different lighting, and different time conditions.

[0014] The output of the key point feature extraction branch is a key point probability map with the same size as the input, and the probability map is decoded to obtain the key point position coordinates; based on the idea of ​​non-maximum suppression, the present invention uses maximum pooling and maximum mask to decode the key point probability map.

[0015] The key point feature description extraction branch outputs a feature description vector of uniform length for each pixel on each feature map, and uses the decoded key point position coordinate index to obtain the feature description vector corresponding to the key point. Specifically, the key point feature description extraction branch first performs bicubic polynomial interpolation processing on each feature map to obtain the feature description vector of each pixel on each feature map, then performs L2 normalization processing on the feature description vector of each pixel on each feature map, and finally outputs a feature description vector of uniform length corresponding to each pixel on each feature map.

[0016] In step S2, the feature matching network constructed by the present invention includes a graph neural network based on an attention mechanism and a matching layer;

[0017] The attention graph neural network encodes the position coordinates and feature description of each key point in each frame of the image into an initial feature expression vector, and uses the attention mechanism to enhance the feature matching performance of the initial feature expression vector to obtain the matching description vector of each key point in each frame of the image.

[0018] In the attention graph neural network structure, the position coordinates of the key points are first encoded using a multi-layer perceptron and upgraded to a size that matches the key point feature description vector. Then, the key point feature description vector is coupled with the encoded key point position coordinates to obtain the initial feature expression vector.

[0019] The specific calculation method is:

[0020] ,

[0021] in, represents the position coordinates of the key point k, ENC() represents the position feature encoding through a multi-layer perceptron, The feature description vector representing the key point, Represents the initial feature expression vector.

[0022] After completing the feature encoding, alternating self-attention and cross-attention calculations are used to enhance the feature matching performance of the initial feature expression vector. Finally, the matching layer is used to calculate the inner product of each matching description vector to obtain the matching score matrix, and the score matrix is ​​optimized. According to the preset value of the score matrix, the key point pairs with successful matching are screened.

[0023] In step S3, the constructed secondary classifier includes a pixel value feature classifier and a motion feature classifier;

[0024] The pixel value feature classifier first sets a pixel value change threshold condition according to the characteristics of the pixel value change of the same target in adjacent frames in the video sequence, filters the key point pairs of adjacent frames successfully matched in step S2, and removes the key points whose pixel values ​​are greater than the threshold; then sets a pixel gradient threshold condition according to the characteristics of the gradient of pixels between the target and the background, and removes the key points whose values ​​are less than the gradient threshold;

[0025] The motion feature classifier further classifies the key point pairs filtered by the pixel value feature classifier according to the motion trajectory characteristics of the same target in the video sequence to obtain the final key point, which is the extremely small target in the video sequence.

[0026] The beneficial effects of the present invention are:

[0027] The present invention utilizes the characteristics of large gradients between pixels of the target and the background, the correlation of the target in the adjacent frames, the pixel value characteristics of the target and the motion characteristics of the target, and transforms the target detection problem into a feature matching and classification problem by constructing a feature extraction network, a feature matching network and a feature classifier, thereby realizing the detection of extremely small targets in the sky and space background. The present invention focuses on the detection of extremely small targets with the number of effective pixels of the target less than 12*12 in the sky and space background, effectively solving the problem that it is difficult for neural networks to characterize extremely small targets, and improving the precision and recall rate of extremely small target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 The present invention is a flowchart of a method for detecting extremely small targets in sky and space background based on video sequence feature matching.

[0030] Figure 2 It is a schematic diagram of the specific structure of the constructed feature extraction network.

[0031] Figure 3 is the original video sequence image.

[0032] Figure 4 It is a measured effect diagram obtained by detecting extremely small targets in original video sequence images using the method of the present invention. DETAILED DESCRIPTION

[0033] The following describes the embodiments of the present invention through specific embodiments, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0034] The extremely small targets in the sky-sky background video sequence have the following characteristics:

[0035] 1. When the relative position of the imaging system and the target is far away, the target angle is much smaller than the field of view of the imaging system, and the number of pixels in the imaging plane is less than 12*12, or even the number of effective pixels of the target is in the single digit. At the same time, because the target pixels are too few, it appears as a point target or a pixel cluster, and the spatial structure of the target is single, with little effective information.

[0036] 2. The target and the surrounding background pixels are unrelated. The target appears as isolated pixel blocks or point clusters, and there is a difference with the background pixels, with a large gradient, which is the high-frequency part of the image frequency domain. In general, the background pixel changes are relatively slow. Even for brighter clouds, the pixels inside the clouds are continuously correlated, mainly occupying the low-frequency components of the image frequency domain. Similarly, noise and targets have similar characteristics.

[0037] 3. In a video sequence, the same target pixel in adjacent frames should conform to certain rules and be similar without significant changes, that is, the target pixels in adjacent frames are correlated.

[0038] 4. In the video sequence, the target position changes continuously, and the changes should conform to certain rules (including static state) and remain within a certain range.

[0039] [Reference Figure 1 According to the above characteristics of the existence of extremely small targets in the sky-sky background video sequence, a method for detecting extremely small targets in the sky-sky background based on video sequence feature matching is designed, which includes the following steps:

[0040] S1. Construct a feature extraction network, take the video sequence as input, and extract the key point information contained in the continuous frame images of the video sequence. The key point information includes the position coordinates of the key point and the feature description vector of the key point.

[0041] Taking advantage of the fact that extremely small target pixels appear as point targets or pixel clusters, and that there is a large gradient between the pixels of the target and the background, a feature extraction network is constructed to extract key points in video sequences. Key points include some salient points, edge points, pixels with large gradients, etc. in the image. Key points represent potential extremely small targets in the image.

[0042] Specifically, the feature extraction network described in the present invention should have the following characteristics:

[0043] 1) The network has a strong ability to extract key points, and can extract key points with large local gradients such as corner points and edge points in the image and their feature description vectors;

[0044] 2) The key points extracted by the network have strong robustness and generalization ability. The key points are image points that can be detected under different sources, different imaging modes, different viewing angles, different lighting conditions, and different time conditions;

[0045] 3) The network should reduce downsampling operations such as pooling and convolution with a step size greater than 1 as much as possible, or adopt other effective measures to reduce or prevent the problem of feature expression failure caused by network downsampling;

[0046] 4) The purpose of the feature extraction network is to extract as many potential key points as possible where there may be very small objects in the image, which requires the network to have a high feature detection recall rate.

[0047] S2. Construct a feature matching network, input the key point coordinates and key point feature description vectors extracted in step S1 into the feature matching network for associative matching of consecutive frames, and output key point pairs for consecutive frame matching.

[0048] The feature matching network is constructed by taking advantage of the fact that objects in adjacent frames are correlated (the features represented by the same object in adjacent frames of a video sequence are similar). Specifically, the feature matching network should have the ability to match features and filter out outliers, and be able to perform inference calculations on the coordinates of each key point on the image obtained by the feature extraction network for consecutive frame images A and B of the video sequence, and the feature description vector of each key point, to obtain the matching relationship between each other's features, and output aggregated paired key point pairs.

[0049] S3. According to the pixel value characteristics and motion characteristics of the target, a secondary classifier is constructed to further screen and classify the successfully matched key point pairs. The final key points obtained are the extremely small targets in the video sequence.

[0050] Specifically, considering the complexity of the actual scene space and the influence of noise, which interferes with the screening and detection of extremely small targets, a secondary feature classifier is constructed to further classify and screen feature matching point pairs.

[0051] In step S1, refer to Figure 2 ,The constructed feature extraction network includes an input layer, a feature encoding layer, a feature extraction layer, and an output layer connected in sequence; the feature extraction layer includes a key point feature extraction branch and a feature description extraction branch;

[0052] The feature encoding layer uniformly encodes the features of continuous frame images in the video sequence input to the network to obtain a feature map containing key point feature information corresponding to each frame image; the key point feature extraction branch extracts the key point features of each feature map, and the feature description extraction branch extracts the key point feature description vector of each feature map to obtain the position coordinates and feature description vector of the key points contained in each frame image.

[0053] The output of the key point feature extraction branch is a key point probability map with the same size as the input. Based on the idea of ​​non-maximum suppression, the key point probability map is decoded using maximum pooling and maximum masking to obtain the key point position coordinates. The key point feature description extraction branch first performs bicubic polynomial interpolation on each feature map to obtain the feature description vector of each pixel on each feature map, and then performs L2 normalization on the feature description vector of each pixel on each feature map. Finally, a feature description vector with a uniform length corresponding to each pixel on each feature map is output.

[0054] The training of the feature extraction network can be divided into three steps:

[0055] The first step is to create a data set using the video image data containing the target and to label the target in the video image;

[0056] The second step is to train the network based on the above data set. The trained network can extract effective feature points and descriptors from video images.

[0057] The third step is to perform geometric transformation (random rotation and scaling) on ​​the image data in the dataset to obtain new data, and then obtain image pairs with known pose relationships. The image pairs are input into the network to improve the network's generalization ability in extracting feature points and descriptors.

[0058] The loss function of the feature extraction network consists of two items, one is the key point detection loss, and the other is the key point feature description loss. The specific calculation formula is:

[0059] ,

[0060] ,

[0061] ,

[0062] Among them, A represents the key points of the original image extracted through the network; B represents the key points of the original image extracted through the network after geometric transformation; Represents the true value label of key point A; represents the true value label of the key point B; λ represents the weight constant; represents the key point detection loss, represents the cross entropy loss function; Represents the key point feature description loss; represents the hinge loss function; It represents the number of pixels on the feature map, that is, the number of feature points; h and w represent the coordinates of the pixels on the feature map, which are used to indicate the position of the feature points on the feature map.

[0063] In step S2, the constructed feature matching network includes a graph neural network based on an attention mechanism and a matching layer;

[0064] The attention graph neural network encodes the position coordinates and feature description of each key point in each frame of the image into an initial feature expression vector, and alternately uses self-attention and cross-attention to enhance the feature matching performance of the initial feature expression vector to obtain the matching description vector of each key point in each frame of the image; the matching layer calculates the inner product of each matching description vector to obtain a matching score matrix, and optimizes the score matrix. According to the preset value of the score matrix, the key point pairs with successful matching are screened.

[0065] Specifically, the position coordinates of the key points are first encoded using a multi-layer perceptron and upgraded to a size that matches the key point feature description vector. The key point feature description vector is then coupled with the encoded key point position coordinates to obtain the initial feature expression vector. The specific calculation formula is:

[0066] ,

[0067] in, represents the position coordinates of the key point k, Indicates feature encoding through a multi-layer perceptron. The feature description vector representing the key point, Represents the initial feature expression vector.

[0068] Assume that each key point in each frame of the video sequence is a node, and each input image frame can be abstracted as a complete graph. The self-attention operation can connect all the key points in the frame, while the cross-attention operation can connect the key points of this graph with all the key points of another complete graph. The attention calculation process is similar to the query operation in the database. The query information query is input, and the value corresponding to the attribute is retrieved according to the target attribute key, and the output is finally obtained by weighted average.

[0069] When performing self-attention calculation, the query, key and value are all calculated from the input corresponding to the same image; when performing cross-attention calculation, the query needs to be calculated for the input corresponding to the current image, while the key and value are calculated from the input corresponding to the next frame image.

[0070] In the feature matching network described in the present invention, self-attention calculation and cross-attention calculation are performed alternately, and are performed L times in total. Assume is the calculation result of the kth initial feature on image A at layer l, and its corresponding update calculation at layer l+1 is:

[0071] ,

[0072] Here, || represents the concatenation of self-attention and cross-attention, and ε represents the type of attention calculation.

[0073] For image A, the output of the attention graph neural network is:

[0074] ,

[0075] Among them, W represents the weight of the feedforward neural network, b represents the bias, is the matching description vector of the kth key point on image A, which is used for feature matching; the key points in image B also have a similar output form.

[0076] The matching layer aggregates the inner product and The features of are used to obtain the score matrix S, and the soft assignment matrix P is constructed by maximizing the overall score.

[0077] Setting the last row / column of the score matrix S to redundant terms yields , an underflow mechanism is introduced. When the score is less than a certain value, the matching point pair is removed from the feature set to filter out incorrect matching points and improve the accuracy of the model.

[0078] The matching point pairs are filtered by the set threshold to obtain the successfully matched feature point pairs:

[0079] ,

[0080] in, Represents the coordinates of the kth key point on images A and B.

[0081] By analogy, the matching point pairs of consecutive frames can be obtained. , where Q represents the frame number of the video sequence, and the potential target information of consecutive frames is obtained.

[0082] The feature matching network is trained using supervised learning, that is, the matching truth value is , the matching relationship can be obtained by the relative pose transformation of the true value, and the feature points without matching can also be obtained. When the true value label is given, the distribution matrix can be minimized The negative log-likelihood function Loss is used to maximize the precision and matching recall. Loss is expressed as:

[0083] ,

[0084] In step S3, the constructed secondary classifier includes a pixel value feature classifier and a motion feature classifier;

[0085] The pixel value feature classifier F is constructed based on the following two principles: the pixel values ​​of the same target in adjacent frames of the video sequence change slowly; there is a large gradient between the pixels of the target and the background. The pixel value feature classifier F first sets the pixel value change threshold condition according to the characteristics of the pixel value change of the same target in adjacent frames in the video sequence, filters the key point pairs of adjacent frames that are successfully matched in step S2, and removes the key points whose pixel values ​​are greater than the threshold; then, according to the characteristics of the gradient between the pixels of the target and the background, sets the pixel gradient threshold condition, and removes the key points whose pixel values ​​are less than the gradient threshold.

[0086] The motion feature classifier T is constructed based on the principle that the motion trajectory of the same target in the video sequence is continuous and the displacement changes are within a certain range. Therefore, the target trajectory and its motion characteristics should conform to certain rules, including the displacement and speed of the same key point in consecutive frames. The motion feature classifier T further classifies the key point pairs filtered by the pixel value feature classifier F under the constraints of the above motion rules, filters out noise and interference targets, and finally obtains the required minimal target.

[0087] The specific description of the secondary feature classifier is as follows: According to the matching point pairs obtained by matching the features of continuous frames You can get the target pixel value Gradient Value Movement trajectory Build potential target data vector .in It is the key point for successful matching in step S2 The corresponding pixel value, motion trajectory is the displacement vector formed by the continuous successful matching of the same key point in the video frame. Assume that the successfully matched feature point pairs in images A and B are , and The corresponding displacement vector is By analogy, the pixel displacement vector of the key points of continuous matching is Then the second-level classifier can be expressed as:

[0088] ,

[0089] Among them, threF represents the threshold of target pixel value change in adjacent frames, threG represents the gradient threshold of pixels between the target and the background, Represents the feature space of target trajectory in consecutive frames.

[0090] The present invention utilizes the characteristics of large gradients between pixels of the target and the background, the correlation of the target in the adjacent frames, the pixel value characteristics of the target and the motion characteristics of the target, and transforms the target detection problem into a feature matching and classification problem by constructing a feature extraction network, a feature matching network and a feature classifier, thereby realizing the detection of extremely small targets in the sky and space background. The present invention focuses on the detection of extremely small targets with the number of effective pixels of the target less than 12*12 in the sky and space background, effectively solving the problem that it is difficult for neural networks to characterize extremely small targets, and improving the precision and recall rate of extremely small target detection.

[0091] In order to verify the effectiveness of the present invention, a test verification data set is constructed by using real video data and public data. The method described in the present invention and the general target detection algorithm YoloV5 are used to perform extremely small target detection on each image in the data set, so as to evaluate and verify the experimental performance of the two methods. The evaluation results are shown in the following table.

[0092]

[0093] Figure 3 It is the original video sequence image. It can be seen from the figure that the extremely small target in the sky background appears as an isolated point cluster in the imaging plane and there are differences in pixels with the background. The target and the background are unrelated, and the gradient of the local area of ​​the target is large.

[0094] Figure 4 This is a measured effect diagram obtained by detecting extremely small targets in the original video sequence image using the method of the present invention, where the triangle represents the potential target calculated by the feature extraction network and the feature matching network, and the circle represents the extremely small target detection result obtained by the final screening and calculation of the secondary feature classifier. From the detection results of the experimental video, it can be seen that the method of the present invention can effectively detect extremely small targets in the sky and space background.

[0095] The embodiments described above are merely descriptions of preferred implementations of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope of the present invention.

Claims

1. A method for detecting extremely small targets in sky and space background based on video sequence feature matching, characterized in that: The following steps are involved: S1. Construct a feature extraction network, take the video sequence as input, and extract key point information of continuous frame images of the video sequence, wherein the key point information includes the position coordinates of the key point and the feature description vector of the key point; S2, construct a feature matching network, input the key point coordinates and key point feature description vectors extracted in step S1 into the feature matching network for association matching of continuous frames, and output key point pairs matched by continuous frames; S3. According to the pixel value characteristics and motion characteristics of the target, a secondary classifier is constructed to further screen and classify the successfully matched key point pairs. The final key point obtained is the minimal target in the video sequence. In step S2, the constructed feature matching network includes a graph neural network based on an attention mechanism and a matching layer; the attention graph neural network encodes the position coordinates and feature description vectors of each key point in each frame image into an initial feature expression vector, and uses the attention mechanism to enhance the feature matching performance of the initial feature expression vector; the matching layer calculates the inner product of each matching description vector to obtain a matching score matrix, and optimizes the score matrix to screen out key point pairs that are successfully matched.

2. The method for detecting extremely small targets in sky and space background based on video sequence feature matching according to claim 1, characterized in that: In step S1, the constructed feature extraction network includes an input layer, a feature encoding layer, a feature extraction layer, and an output layer connected in sequence; the feature extraction layer includes a key point feature extraction branch and a feature description extraction branch.

3. The method for detecting extremely small targets in sky and space background based on video sequence feature matching according to claim 2, characterized in that: In step S1, continuous frame images of the video sequence input into the feature extraction network are uniformly feature encoded in the feature encoding layer to obtain a feature map containing key point feature information corresponding to each frame image, the key point feature extraction branch extracts the key point features of each feature map, and the feature description extraction branch extracts the key point feature description vector of each feature map to obtain the position coordinates and feature description vector of the key points contained in each frame image.

4. The method for detecting extremely small targets in sky and space background based on video sequence feature matching according to claim 3, characterized in that: In step S1, the output of the key point feature extraction branch is a key point probability map with the same size as the input, and the key point position coordinates are obtained by decoding the probability map; the key point feature description extraction branch outputs a feature description vector of uniform length for each pixel on each feature map, and uses the decoded key point position coordinate index to obtain the feature description vector corresponding to the key point.

5. The method for detecting extremely small targets in sky and space background based on video sequence feature matching according to claim 1, characterized in that: In step S2, the constructed feature matching network uses a multi-layer perceptron to encode the position coordinates of the key points and upgrade the dimensions to a size that matches the key point feature description vector. The key point feature description vector is then coupled with the encoded key point position coordinates to obtain the initial feature expression vector.

6. The method for detecting extremely small targets in sky and space background based on video sequence feature matching according to claim 1, characterized in that: In step S2, the constructed feature matching network uses alternating self-attention and cross-attention calculations to enhance the feature matching performance of the initial feature expression vector.

7. The method for detecting extremely small targets in sky and space background based on video sequence feature matching according to claim 1, characterized in that: In step S3, the constructed secondary classifier includes a connected pixel value feature classifier and a motion feature classifier.

8. The method for detecting extremely small targets in sky and space background based on video sequence feature matching according to claim 7, characterized in that: In step S3, the pixel value feature classifier filters the key point pairs of adjacent frames successfully matched in step S2 according to the characteristics of the change of the same target pixel values ​​in adjacent frames in the video sequence, and further filters the filtered key point pairs of adjacent frames according to the characteristics of the gradient of pixels between the target and the background.

9. The method for detecting extremely small targets in sky and space background based on video sequence feature matching according to claim 7, characterized in that: In step S3, the motion feature classifier further classifies the screening results of the pixel value feature classifier according to the motion trajectory characteristics of the same target in the video sequence to obtain the final key point, which is the extremely small target in the video sequence.

Images