A dark and weak spatial small target detection method based on spatial guidance

Abstract

The present application belongs to the field of space target detection, and relates to a dark and weak space small target detection method based on space domain guidance, comprising: inputting a space target image sequence to be detected into a dark and weak space small target time sequence evolution feature perception network based on space domain guidance which is pre-established and trained, to obtain a target prediction frame, so as to realize dark and weak space small target detection; the dark and weak space small target time sequence evolution feature perception network extracts initial space-time features of the image sequence through a lightweight three-dimensional convolution network, enhances target response and suppresses star clutter, builds a double-branch structure to model target time sequence evolution features and enhance target space region features, adopts a double attention feature fusion module based on a gate to selectively enhance and fuse features output by the branch structure, and improves the joint expression capability of position and motion information.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of space target detection, and particularly relates to a dark and weak space small target detection method based on space domain guidance. BACKGROUND

[0002] The dark and weak space small target mainly refers to that in the remote space-based passive optical observation, the space target (such as artificial satellite, artificial spacecraft, space debris, etc.) is far away from the observation platform, small in size and weak in signal strength, only occupies a few pixel regions on the imaging plane, and the imaging signal-to-noise ratio is low. The target is difficult to identify from the complex background, and the target is dark in brightness, weak in energy and small in scale in the image. In recent years, with the rapid development of space technology, space situation awareness has gradually become the research focus of various countries. Among them, the detection of dark and weak space small targets as one of the key technologies of space situation awareness system has important application value in the fields of satellite monitoring, space safety warning, orbit maintenance and space defense. However, in the complex deep space background, the detection of dark and weak space target under the condition of low signal-to-noise ratio still faces severe challenges.

[0003] The existing dark and weak target detection methods can be roughly divided into model-driven and data-driven two categories. In the past few decades, the detection of dark and weak small targets mainly relies on model-driven methods, that is, through the artificially designed detector, the target is recognized based on the significant feature difference between the target and the background. Typical methods include ephemeris calculation, Hough transform, multi-frame time sequence projection method and detection before tracking, in addition to methods based on filter, human visual system and low-rank sparse decomposition. This kind of method is usually established on the basis of mature mathematical theory, and has good interpretability in target feature modeling. However, in the face of strong star clutter and strong noise interference in complex deep space environment, the low signal-to-noise ratio target is easy to be submerged by the background, which leads to the difficulty of traditional method in effectively identifying the target and the significant decline of detection performance.

[0004] In recent years, with the rapid development of convolutional neural networks, the field of target detection has made significant progress, bringing new solutions for dim and weak target detection under low signal-to-noise ratio conditions. With the powerful feature expression and learning ability of deep learning, data-driven methods can automatically extract target features from large-scale data sets, effectively improving detection accuracy, and gradually becoming an important research direction for dim and weak small target detection. Current data-driven methods can be divided into single-frame and multi-frame image-based methods. Single-frame detection methods mainly rely on the gray distribution characteristics of the target, and learn spatial local features through neural networks to locate the target, with the advantages of low computational complexity and fast detection speed. However, in a complex deep space background, the target is usually dim and weak, and has similar features with the background star clutter in the local space, making single-frame detection face serious false alarm and missed detection problems. In contrast, multi-frame detection methods further introduce target motion information, making full use of the distribution characteristics of the target in time and space dimensions, thereby extracting more discriminative features and improving the detectability of weak and small targets. However, existing multi-frame detection methods are mostly designed for stable background and rich texture scenes such as infrared unmanned aerial vehicle target detection. These methods usually process target spatial and temporal features separately, ignoring the cooperative relationship between the two, making it difficult to adapt to deep space exploration scenarios with multiple target star clutters, weak texture features, and strong background interference. SUMMARY

[0005] The purpose of the present application is to overcome the defects of the prior art and provide a dim and weak spatial small target detection method based on spatial domain guidance.

[0006] Therefore, the present application provides a dim and weak spatial small target detection method based on spatial domain guidance, which comprises:

[0007] The spatial target image sequence to be detected is input into the dim and weak spatial small target time series evolution feature perception network based on spatial domain guidance which is pre-established and trained, to obtain a target prediction frame, thereby realizing dim and weak spatial small target detection.

[0008] The dim and weak spatial small target time series evolution feature perception network extracts the initial spatio-temporal features of the image sequence through a lightweight three-dimensional convolutional network, enhances target response and suppresses star clutter, builds a double-branch structure to model target time series evolution features and enhance target spatial region features, and uses a double-attention feature fusion module based on gating to selectively enhance and fuse the features output by the branch structure, thereby improving the joint expression ability of position and motion information.

[0009] Preferably, the input of the dark and weak spatial small target time sequence evolution feature perception network is a continuous 5-frame image sequence, and the output is a target prediction box of the last frame of the current image sequence. The dark and weak spatial small target time sequence evolution feature perception network comprises an initial feature extraction module, a space-time dual-branch feature extraction module, a double attention feature fusion module based on a gate, and a target detection head, wherein

[0010] The initial feature extraction module is configured to perform initial space-time feature extraction on the continuous 5-frame image sequence of the spatial target to be detected, so as to obtain corresponding initial space-time features.

[0011] The space-time dual-branch feature extraction module is configured to extract target time domain and space domain features respectively through a time sequence evolution feature perception branch and a space region enhancement branch based on cross attention, so as to obtain time domain features and space domain features.

[0012] The double attention feature fusion module based on the gate is configured to fuse the time domain features and the space domain features, so as to obtain target space-time correlation features.

[0013] The target detection head is configured to process the target space-time correlation features to obtain a final target prediction box result.

[0014] Preferably, the initial feature extraction module comprises time dimension stacking, three groups of three-dimensional convolution blocks and down-sampling connected in sequence, and two two-dimensional convolution blocks. The three-dimensional convolution block comprises a three-dimensional convolution layer, batch normalization and a nonlinear activation function connected in sequence. The down-sampling comprises a three-dimensional convolution layer with a step of 2, batch normalization and a nonlinear activation function connected in sequence. The two-dimensional convolution block comprises a two-dimensional convolution layer, batch normalization and a nonlinear activation function connected in sequence.

[0015] The specific processing process comprises:

[0016] The input is a continuous 5-frame image sequence, wherein, is a reference frame, is a key frame, and a space-time tensor is obtained through time dimension stacking. is a channel number of the input image, is a row number of the image, is a column number of the image.

[0017] After the three groups of three-dimensional convolution blocks and down-sampling are connected in sequence, target space-time features are obtained.​​​​​​​​​​​​ Perform time-dimensional feature decomposition to obtain the first... i Spatiotemporal features corresponding to frame images ,in ;

[0018] Will By iteratively inputting two concatenated two-dimensional convolutional blocks, we obtain the... i Initial spatiotemporal features of frame images .

[0019] Preferably, the spatial region enhancement branch based on cross-attention includes: feature connection, two cascaded two-dimensional convolutional blocks, three parallel two-dimensional convolutional blocks, cross-attention, two-dimensional convolutional blocks, three parallel two-dimensional convolutional blocks, and parameter normalization. The specific processing includes:

[0020] Initial spatiotemporal features of the reference frame First, feature concatenation along the channel dimension is performed, and spatial difference features of the reference frame are further extracted through two cascaded 2D convolutional blocks. Then, features of the query Q corresponding to the keyframe, the key K and value V corresponding to the reference frame are generated through three parallel 2D convolutional blocks. Based on the attention mechanism, the spatial features of the keyframe are obtained after weighting by cross-attention coefficients. ;

[0021] Initial spatiotemporal features of keyframes and Residual connections are performed, and the residually connected features are passed through three parallel 2D convolutional blocks to extract multi-scale features of the target region. The multi-scale features are then summed and normalized to obtain spatial weighting coefficients. These coefficients are used to weight the keyframe features, outputting the spatial domain features. .

[0022] Preferably, the temporal evolution feature perception branch includes: a temporal memory network and a temporal context guidance module, wherein,

[0023] The temporal memory network includes: feature connections, two-dimensional convolutional blocks, and multi-layer cascaded memory blocks;

[0024] The temporal context guidance module includes: feature connection, channel rearrangement, and two-dimensional convolutional block.

[0025] Preferably, the processing procedure of the temporal memory network includes:

[0026] Initial spatiotemporal features of the first frame image First, feature alignment is performed using two-dimensional convolutional blocks, and then the hidden state of the Lth layer is obtained through L layers of memory blocks. and memory state ;

[0027] Initial spatio-temporal features of other frame images , wherein , first with the hidden state of the Lth layer of the previous frame Feature connection is performed in the channel dimension, and then a two-dimensional convolution block is used for feature alignment, after which the hidden state of each layer memory block of the previous frame is updated to obtain the updated hidden state of the Lth layer and memory state ;

[0028] The hidden state of the Lth layer of each frame and the memory state of the Lth layer of the key frame are respectively passed through a two-dimensional convolution block to obtain the local temporal feature of each frame image and the global temporal feature of the image sequence .

[0029] Preferably, the processing process of the temporal context guidance module includes:

[0030] The local temporal feature of each frame image and the global temporal feature of the image sequence First, feature enhancement is performed through channel-position collaborative attention, and then feature connection and channel rearrangement are performed;

[0031] The channel rearrangement enhanced feature is processed through a series of two-dimensional convolution blocks to obtain a feature with higher temporal sensitivity , wherein ;

[0032] The are connected, and two two-dimensional convolution blocks are used to obtain the temporal feature of the time series evolution feature perception branch output .

[0033] Preferably, the dual attention feature fusion module based on gating includes channel dimension average pooling, channel dimension maximum pooling, global average pooling, feature connection, two-dimensional convolution block and parameter normalization; the specific processing process includes:

[0034] Temporal feature Two two-dimensional features are obtained through two parallel channel dimension average pooling and channel dimension maximum pooling, and then the two two-dimensional features are connected, and then a two-dimensional convolution block and a parameter normalization are sequentially passed through to obtain a spatial attention coefficient, and the temporal feature is weighted to obtain the temporal feature after selective feature enhancement ;

[0035] Spatial feature First, a global average pooling is performed, then two two-dimensional convolution blocks in series and a parameter normalization are performed to obtain channel attention coefficients, and the spatial domain features are weighted to obtain selective feature enhanced spatial domain features

[0036] The enhanced temporal domain features and the spatial domain features are connected through a feature connection, and then a two-dimensional convolution block in series and a parameter normalization in series are performed to obtain weighting coefficients , and the complementary fusion of and is performed to obtain spatiotemporal correlation features through two two-dimensional convolution blocks in series .

[0037] Preferably, the target detection head comprises a two-dimensional convolution block, three parallel detection branches and a feature connection, wherein the first detection branch and the second detection branch each comprise three two-dimensional convolution blocks in series, and the third detection branch comprises a two-dimensional convolution block. The three detection branches respectively output the regression result, the classification result and the confidence result of the target, and the final target prediction frame result is obtained after the feature connection is connected in the channel dimension.

[0038] Preferably, the method further comprises training of the dark and weak spatial small target time evolution feature perception network, and the overall loss function of the training is the sum of regression loss, confidence loss and classification loss, wherein

[0039] The regression loss adopts a IoU loss function, and the confidence loss and the classification loss each adopt a binary cross-entropy loss function;

[0040] The Adam optimizer is used to update and predict the network parameters, and the learning rate is set to 1e-4, and the training batch size is 4;

[0041] The training comprises the step of preparing a training set, specifically comprising:

[0042] Step A1: setting simulation parameters of the observation position, observation start and end time, line-of-sight pointing, observation field angle, maximum detection distance, limit detection magnitude, camera aperture, exposure time, optical system quantum efficiency, optical transmittance, camera gain, camera gray scale bias, readout noise, dark current noise, image sensor target surface size, image resolution, image frame number and target imaging signal-to-noise ratio of a space-based long-range passive optical observation platform;

[0043] ​​Step A2: Select the visible stars in the field of view according to the star catalog and all parameters set in step A1, and correct the star positions using the position, proper motion, and precession information in the star catalog, and convert them to the imaging coordinate system;

[0044] Step A3: Calculate the imaging gray value of the visible stars in the field of view using the apparent magnitude information in the star catalog and the simulation parameters, and generate star points with different diffusion degrees and widths using the Gaussian point spread function, and add them to the pure black background image according to the imaging coordinates to generate a background star image;

[0045] Step A4: Establish a noise model for the known imaging noise, model the imaging noise with zero-mean Gaussian white noise, generate imaging noise with different mean and variance, and superimpose it on the background star image to generate a background image;

[0046] Step A5: Calculate the position of different targets with known orbital information, obtain the position information in the imaging coordinate system through coordinate transformation, and derive the target imaging gray value according to the set target imaging signal-to-noise ratio, Gaussian point spread function, and mean and standard deviation of the generated background image as follows:

[0047] ;

[0048] ;

[0049] wherein, is the average gray value of a single target on the image, is the gray standard deviation of the generated background image, is the gray mean value of the background image, is the set target imaging signal-to-noise ratio in the simulation parameters, is the imaging gray value at the center of the target, is the number of pixels occupied by the target, is the imaging coordinate at the center of the target, is the natural logarithm, is the k th point of the target imaging area, is the imaging coordinate of the k th point of the target imaging area;

[0050] Step A6: Superimpose the gray value of the target region on the generated background image to obtain a target simulation image sequence, and superimpose the target on the pure black image as a label;

[0051] Step A7: Construct the generated target simulation image and its label into a data set, and divide it into a training set, a validation set, and a test set.

[0052] Compared with the prior art, the application has the advantages that:

[0053] 1、 In the network initial feature extraction stage, the lightweight three-dimensional convolution network is used to replace the traditional two-dimensional spatial feature extraction network, which can jointly extract the initial space-time features of the target from continuous multiple frames of images, supplement the spatial representation of the target with the inter-frame timing information, effectively alleviate the high similarity of the target and the stationary background star in the spatial domain, and thus improve the distinguishability and detection performance of the low signal-to-noise ratio dim target.

[0054] 2、 In the space-time collaborative modeling aspect, a space-time double-branch structure is used to extract the target time domain and spatial domain features. Among them, the time sequence evolution feature perception branch models the target motion trend by analyzing the dynamic changes between continuous frames, and the spatial region enhancement branch uses the spatial difference information in the reference frame to enhance the target region in the current frame based on the cross-attention mechanism, so as to realize the collaborative enhancement of space-time features under the guidance of space domain, and improve the accuracy and robustness of target positioning.

[0055] 3、 In order to further excavate the deep space-time correlation features of the target, a dual attention feature fusion module based on gating is used, which selectively enhances the target response ability of the spatial domain features through channel attention, and highlights the target significant area in the time domain features through spatial attention, further improving the discriminability of the feature representation; At the same time, the gating mechanism is introduced to fuse the enhanced features, realize the effective complementation of space-time information, and enhance the expression ability and detection performance of the small target. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 is the space-time evolution feature perception network structure diagram based on space domain guidance in the method of the application;

[0057] Figure 2 is the structure diagram of the time sequence memory network, the dual attention feature fusion module based on gating and the target detection head in the method of the application. DETAILED DESCRIPTION

[0058] To solve these problems, the application provides a dark and weak spatial small target detection method based on space domain guidance, which inputs a spatial target image sequence to be detected into a dark and weak spatial small target time evolution feature perception network based on space domain guidance which is established and trained in advance, obtains a target prediction frame, and realizes dark and weak spatial small target detection. The core invention point is the dark and weak spatial small target time evolution feature perception network based on space domain guidance. The network first extracts the initial space-time features of the image sequence through a lightweight three-dimensional convolution network, enhances the target response and suppresses the star clutter. Secondly, a double-branch structure is constructed to model the target time evolution features and enhance the target spatial region features. Then, a double attention feature fusion module based on gating is used to selectively enhance and fuse the features output by the branch structure, improving the joint expression ability of position and motion information. Finally, the end-to-end dark and weak spatial small target detection is realized through the detection head.

[0059] To solve the problem of difficult feature extraction of dark and weak spatial small targets in a complex deep space background, the application provides a dark and weak spatial small target time evolution feature perception network based on space domain guidance, which includes the following steps

[0060] Step A: An optical imaging model is established for target observation images in a space-based long-distance passive optical detection scene, and simulation data of spatial motion target sequence images with different observation positions, different observation directions, different motion trajectories and different signal-to-noise ratios are generated through the optical imaging model to construct a simulation data set.

[0061] Step B: A dark and weak spatial small target time evolution feature perception network based on space domain guidance is constructed. The network includes four modules: an initial feature extraction module, a space-time double-branch feature extraction module, a double attention feature fusion module based on gating, and a target detection head. First, the initial space-time features of the target are extracted through the initial feature extraction module. Secondly, the time domain and spatial domain features of the target are extracted through the space-time double-branch feature extraction module. The space-time double-branch feature extraction module includes two branches, one is a time evolution feature perception branch, and the other is a spatial region enhancement branch based on cross attention. Then, the target time domain and spatial domain features obtained by the space-time double-branch feature extraction module are fused through the double attention feature fusion module based on gating to obtain the target space-time correlation features. Finally, the target space-time correlation features are processed through the target detection head to obtain the final target detection result.

[0062] Step C: The loss function and training hyperparameters of the dark and weak spatial small target time evolution feature perception network based on space domain guidance are set, the simulation data set generated in step A is used to train the network, and the optimal network parameter model is obtained.

[0063] Step D: loading the network parameter model, inputting the spatial target image sequence to be detected into the dark and weak spatial small target time evolution feature perception network based on spatial domain guidance, obtaining a target prediction box, and realizing dark and weak spatial small target detection

[0064] The technical solutions of the present application will be described in detail below in combination with the drawings and embodiments.

[0065] Embodiments

[0066] The embodiments of the present application propose a dark and weak spatial small target detection method based on spatial domain guidance.

[0067] The implementation scheme consists of four parts: first, generate spatial target sequence image simulation data set under space-based long-distance detection scene; then construct dark and weak spatial small target time evolution feature perception network based on spatial domain guidance; then set loss function and training hyperparameters, train the network using the simulation data set to obtain the optimal model; finally, load the optimal model in the network, input the image sequence to be detected into the dark and weak spatial small target time evolution feature perception network based on spatial domain guidance, and obtain the target detection result.

[0068] First step: making data set

[0069] Step A: establishing an optical imaging model for the target observation image under the space-based long-distance passive optical detection scene, generating spatial motion target sequence image simulation data with different observation positions, different observation directions, different motion trajectories, and different signal-to-noise ratios through the optical imaging model, and constructing a simulation data set.

[0070] Step A1: setting the observation position, observation start and end time, visual axis direction, observation field of view angle, maximum detection distance, limit detection magnitude, camera aperture, exposure time, optical system quantum efficiency, optical transmittance, camera gain, camera gray scale bias, readout noise, dark current noise, image sensor target surface size, image resolution, image frame number, and target imaging signal-to-noise ratio simulation parameters of the observation platform.

[0071] Step A2: selecting the visible stars in the field of view according to the star catalog and all the parameters set in step A1, and correcting the star positions using the position, proper motion, and precession information in the star catalog, and converting them to the imaging coordinate system.

[0072] Step A3: calculating the imaging gray scale values of the visible stars in the field of view using the visual magnitude information in the star catalog and the simulation parameters, and generating stars with different diffusion degrees and widths using a Gaussian point spread function, and adding them to the pure black background image according to the imaging coordinates to generate a background star image.

[0073] Step A4: Establish a noise model for the imaging noise with known distribution, model the imaging noise with zero-mean Gaussian white noise, generate imaging noise with different mean and variance, and superimpose it on the background star map to generate the background map.

[0074] Step A5: Calculate the position of different targets with known orbital information, obtain the position information in the imaging coordinate system through coordinate transformation, and derive the target imaging gray value according to the set target imaging signal-to-noise ratio, Gaussian point spread function and the mean and standard deviation of the generated background map. The derivation process is shown in equations (1) and (2):

[0075] (1)

[0076] (2)

[0077] wherein, is the average gray value of a single target on the image, is the gray standard deviation of the generated background map, is the gray mean value of the background map, is the target imaging signal-to-noise ratio set in the simulation parameters, is the imaging gray value at the center point of the target, is the number of pixels occupied by the target, is the imaging coordinate at the center of the target, is the natural logarithm, is the k th point in the target imaging area, is the imaging coordinate of the k th point in the target imaging area.

[0078] Step A6: Superimpose the gray value of the target region on the generated background map to obtain a sequence of target simulation images, and superimpose the target on a pure black image as a label.

[0079] Step A7: Construct the generated target simulation images and their labels into a dataset, and further divide them into a training set, a validation set and a test set.

[0080] Second step: Constructing a dark and weak spatial small target time evolution feature perception network based on spatial guidance

[0081] Step B: Constructing a dark and weak spatial small target time series evolution feature perception network based on spatial guidance. The network includes four modules: an initial feature extraction module, a spatio-temporal double-branch feature extraction module, a gated double-attention feature fusion module, and a target detection head. First, the initial spatio-temporal feature of the target is extracted through the initial feature extraction module. Second, the time domain and spatial domain features of the target are extracted through the spatio-temporal double-branch feature extraction module. The spatio-temporal double-branch feature extraction module includes two branches, one is the time series evolution feature perception branch, and the other is the spatial region enhancement branch based on cross attention. Then, the target time domain and spatial domain features obtained by the spatio-temporal double-branch feature extraction module are fused through the gated double-attention feature fusion module to obtain the target spatio-temporal correlation feature. Finally, the target spatio-temporal correlation feature is processed through the target detection head to obtain the final target detection result.

[0082] Step B1: The dark and weak spatial small target time series evolution feature perception network based on spatial guidance includes four modules: an initial feature extraction module, a spatio-temporal double-branch feature extraction module, a gated double-attention feature fusion module, and a target detection head.

[0083] Step B2: In the initial feature extraction module, the input of the continuous 5-frame image sequence is processed through the initial feature extraction module to obtain the initial spatio-temporal feature of each frame of image. The initial feature extraction module, as shown in Figure 1 , includes a time dimension stacking operation, three three-dimensional convolution blocks, three downsampling, and two two-dimensional convolution blocks. The input of the continuous 5-frame image sequence is , where is the reference frame, is the key frame. After the time dimension stacking operation, the input image sequence obtains the spatio-temporal tensor is the channel number of the input image, is the number of image rows, is the number of image columns.

[0084] Step B3: After obtaining the spatio-temporal tensor ​After that, the target spatiotemporal features are extracted layer by layer and the spatial resolution of the features is gradually reduced through three groups of three-dimensional convolution blocks (C3BS) and downsampling (DownSample Conv3d-BatchNorm3d-SiLU, D3BS). The three-dimensional convolution block is sequentially connected by a three-dimensional convolution layer, batch normalization and a nonlinear activation function (Conv3d-BatchNorm3d-SiLU, C3BS). The downsampling is sequentially connected by a three-dimensional convolution layer with a step of 2, batch normalization and a nonlinear activation function (DownSample Conv3d-BatchNorm3d-SiLU, D3BS). The processing result is shown in equation (3).

[0085] (3)

[0086] wherein, is the target spatiotemporal feature obtained after the third downsampling.

[0087] Step B4: After obtaining the target spatiotemporal feature , the feature is split in the time dimension to obtain the spatiotemporal feature corresponding to each frame of image, which is input into two two-dimensional convolution blocks in series to obtain the initial spatiotemporal feature of each frame of image. The two-dimensional convolution block is sequentially connected by a two-dimensional convolution layer, batch normalization and a nonlinear activation function (Conv2d-BatchNorm2d-SiLU, C2BS). The extraction process of the initial spatiotemporal feature is shown in equation (4).

[0088] (4)

[0089] Step B5: After obtaining the initial spatiotemporal feature of the input image sequence, the target temporal and spatial features are extracted by the spatiotemporal double-branch feature extraction module. The spatiotemporal double-branch feature extraction module includes two branches, one is a time series evolution feature perception branch and the other is a spatial region enhancement branch based on cross attention, as shown in Figure 1 . The spatial region enhancement branch based on cross attention mainly consists of feature connection, two-dimensional convolution block, cross attention and parameter normalization. The initial spatiotemporal feature ​After inputting the cross-attention-based spatial region enhancement branch, first, the feature concatenation (Concat) in the channel dimension is performed, and the spatial difference features of the reference frame are further extracted through two two-dimensional convolution blocks in series. Subsequently, three two-dimensional convolution blocks in parallel are used to generate the query (Q) corresponding to the key frame, the key (K) and the value (V) corresponding to the reference frame, respectively. Based on the calculation formula of the attention mechanism, the spatial features of the key frame weighted by the cross-attention coefficient are further obtained, so as to realize the enhancement of the target spatial region. The processing process is shown in equations (5) to (8).

[0090] (5)

[0091] (6)

[0092] (7)

[0093] (8)

[0094] wherein, is the cross-attention coefficient, is the Hadamard product operation (i.e., element-wise multiplication), is the spatial feature of the key frame weighted by the cross-attention coefficient.

[0095] Step B6: In order to make full use of the spatial difference information of the reference frame and retain the features of the key frame itself, the initial spatio-temporal features of the key frame are residual connected with the spatial features of the key frame weighted by the cross-attention coefficient . After that, the features after the residual connection are extracted through three two-dimensional convolution blocks in parallel to obtain the multi-scale features of the target region, and the multi-scale features are added and then parameter normalized to obtain the spatial weighting coefficient, which is used to weight the key frame features, so as to enhance the response to different sizes of target regions and improve the discriminability of the spatial features. The processing process is shown in equations (9) and (10).

[0096] (9)

[0097] (10)

[0098] wherein, is the initial spatio-temporal features of the key frame and the spatial features of the key frame weighted by the cross-attention coefficient after the residual connection, are the kernel sizes of , and Two-dimensional convolutional blocks, For parameter normalization operation, To enhance the spatial features of branch outputs based on cross-attention spatial regions.

[0099] Step B7: Simultaneously, the temporal evolution feature perception branch extracts the temporal features of the target. The temporal evolution feature perception branch mainly consists of a temporal memory network and a temporal context guidance module, such as... Figure 1 As shown. The temporal context guidance module consists of feature connections, channel shuffle (CS), and two-dimensional convolutional blocks. The temporal memory network consists of feature connections, two-dimensional convolutional blocks, and multi-layered cascaded memory blocks, as shown. Figure 2 As shown. The memory block is a Spatiotemporal Long Short-Term Memory (STLSTM) network structure. The initial spatiotemporal features output by the initial feature extraction module are shown. First, the images are input into a temporal memory network in chronological order. After passing through L layers of memory blocks, the local temporal features of each frame are obtained. Global temporal features of image sequences The specific implementation process is as follows: the initial spatiotemporal features of the first frame image. First, feature alignment is performed using two-dimensional convolutional blocks, and then the hidden state of the Lth layer is obtained through L layers of memory blocks. and memory state The initial spatiotemporal features of subsequent frames. ( First, compare the hidden state of layer L in the previous frame. Feature concatenation is performed along the channel dimension, followed by feature alignment using a 2D convolutional block. Then, the hidden states of each memory block from the previous frame are updated to obtain the updated hidden states of the Lth layer. and memory state Finally, the hidden states of layer L in each frame are obtained. and the memory state of the Lth layer of the keyframe The local temporal features of each frame are obtained by passing each frame through a two-dimensional convolutional block. Global temporal features of image sequences The process is shown in formulas (11) to (16).

[0100] (11)

[0101] (12)

[0102] (13)

[0103] (14)

[0104] (15)

[0105] (16)

[0106] wherein, is the initial spatio-temporal feature of the first frame image is the feature after feature alignment is the initial spatio-temporal feature of the first frame image is the hidden state of the previous frame L-th layer is the feature after feature alignment and feature connection is the cell state of the L-th layer memory block of the first frame image is the hidden state of the L-th layer memory block of the first frame image is the L-th layer memory block is the L-th layer memory block

[0107] Step B8: In the temporal context guidance module, the local temporal feature of each frame and the global temporal feature of the image sequence are first enhanced in feature (using Channel-Position Collaborative Attention, CPCA) to enhance the feature representation ability of the network. Then, the local temporal feature of each frame and the global temporal feature of the image sequence are connected in feature, and the information interaction between the features is enhanced through channel rearrangement to realize the guidance and enhancement of the local temporal feature of each frame. Subsequently, the features after channel rearrangement are processed through a two-dimensional convolution block in series to obtain features with higher temporal sensitivity . Finally, the features are connected in feature, and two two-dimensional convolution blocks in series are used to obtain the temporal feature output by the temporal evolution feature perception branch. The processing process is shown in equations (17) and (18).

[0108] (17)

[0109] (18) ​​​​​​​​

[0110] Step B9: The gating-based dual-attention feature fusion module is composed of channel-wise average pooling (CAP), channel-wise max pooling (CMP), global average pooling (GAP), feature concatenation, a two-dimensional convolution block, and parameter normalization, as shown in Figure 2 The spatial domain features and the time domain features are selectively enhanced and fused by the gating-based dual-attention feature fusion module. The time domain features are first obtained by two parallel channel-wise average pooling and channel-wise max pooling to obtain two two-dimensional features, then the two two-dimensional features are concatenated, and then a two-dimensional convolution block and a parameter normalization are sequentially passed to obtain a spatial attention coefficient, and the time domain features are weighted to obtain the selectively enhanced time domain features . The processing process is shown in equations (19) and (20).

[0111] (19)

[0112] (20)

[0113] wherein, is the spatial attention coefficient.

[0114] Step B10: The spatial domain features are first processed by a global average pooling, then two two-dimensional convolution blocks in series and a parameter normalization are passed to obtain a channel attention coefficient, and finally the spatial domain features are weighted to obtain the selectively enhanced spatial domain features . The processing process is shown in equations (21) and (22).

[0115] (21)

[0116] (22)

[0117] wherein, is the channel attention coefficient.

[0118] Step B11: The enhanced time domain features and the spatial domain features are concatenated, then a two-dimensional convolution block in series and a parameter normalization in series are passed to obtain a weighting coefficient , then the two features are complementarily fused, and finally the spatio-temporal correlation features output by the gated dual attention feature fusion module are obtained through two two-dimensional convolution blocks in series . The processing process is shown in formulas (23) and (24).

[0119] (23)

[0120] (24)

[0121] Step B12: The spatio-temporal correlation features output by the gated dual attention feature fusion module are finally sent into the target detection head to obtain the final target detection result. The structure of the target detection head is shown in Figure 2 , which includes one two-dimensional convolution block, three parallel detection branches and one feature connection. The first detection branch and the second detection branch are both composed of three two-dimensional convolution blocks in series, and the third detection branch is composed of one two-dimensional convolution block. The three detection branches respectively output the regression result, the classification result and the confidence result of the target, and the final target detection result is obtained after the feature connection in the channel dimension.

[0122] Step 3: Network training

[0123] Step C: Setting a loss function and training hyperparameters for the dark and weak spatial small target time series evolution feature perception network based on spatial domain guidance, training the network using the simulation data set generated in step A, and obtaining the optimal network parameter model.

[0124] Step C1: The overall loss function of the dark and weak spatial small target time series evolution feature perception network based on spatial domain guidance is the sum of regression loss, confidence loss and classification loss. Among them, the regression loss adopts the IoU loss function, which is used to measure the overlap degree between the predicted box and the real box; the confidence loss and the classification loss both adopt the BCE loss function, which are respectively used to optimize the accuracy of target existence discrimination and class prediction. And in order to adapt to the single classification task, the hyperparameter is used to weight the classification loss, so as to reduce the influence of the classification loss on the overall loss function. And the hyperparameter is used to weight the regression loss, so as to enhance the influence of the regression loss on the overall loss function, thereby improving the accuracy of the predicted box.

[0125] Step C2: The Adam optimizer is used to update and predict the network parameters, and the learning rate The training batch size is 4. The training parameters of the dark and weak spatial small target time evolution feature perception network based on spatial domain guidance are initialized, the training set and the verification set are loaded, the network is trained, and the optimal parameter model of the dark and weak spatial small target time evolution feature perception network based on spatial domain guidance is obtained.

[0126] Step D: loading the network parameter model, inputting the spatial target image sequence to be detected into the dark and weak spatial small target time evolution feature perception network based on spatial domain guidance, and obtaining the target image sequence after detection.

[0127] Step D1: creating a dark and weak spatial small target time evolution feature perception network based on spatial domain guidance, and loading the optimal parameter model obtained in step C.

[0128] Step D2: inputting the spatial target image sequence to be detected into the dark and weak spatial small target time evolution feature perception network based on spatial domain guidance, obtaining the target prediction box, and drawing the target prediction box in the original target image to obtain the target detection result.

[0129] It is worth noting that in the above embodiment of the system, each module included is only divided according to functional logic, but is not limited to the above division, as long as the corresponding function can be realized; in addition, the specific names of each functional module are only for easy mutual differentiation, and do not serve to limit the protection scope of the present application.

[0130] Summary:

[0131] The present application provides a dark and weak spatial small target detection method based on spatial domain guidance, and designs a dark and weak spatial small target time evolution feature perception network based on spatial domain guidance. The method replaces the traditional two-dimensional convolution network with a lightweight three-dimensional convolution network to jointly extract the space-time features of the target from the image sequence to be detected, thereby effectively addressing the problems of weak target features and high similarity to star clutter. To model the dynamic change process of the target between consecutive frames, the network introduces a time evolution feature perception branch; at the same time, a spatial region enhancement branch based on cross-attention is designed, which uses the semantic information of the reference frame to guide the spatial positioning of the current frame target, thereby solving the problem of insufficient spatial discrimination caused by independent processing of spatial and temporal features. To fully integrate multi-dimensional features, a dual attention feature fusion module based on gating is further introduced to dynamically integrate the space-time information of the target and enhance the space-time collaborative expression ability, thereby achieving accurate detection of dark and weak targets. Therefore, the present application can significantly improve the performance of dark and weak spatial small target detection in low signal-to-noise ratio and complex deep space scenes, and has good engineering practical value and promotion prospects.

[0132] Finally, it should be noted that the above examples are merely used to illustrate the technical solutions of the present application but not to limit. Although the present application is explained in detail with reference to the examples, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, and all of them should be covered in the scope of the claims of the present application.

Claims

1. A method for detecting small targets in dim space based on spatial guidance, comprising: The sequence of spatial target images to be detected is input into a pre-established and trained spatial domain-guided temporal evolution feature perception network for small targets in dim space to obtain target prediction boxes and realize the detection of small targets in dim space. The dim space small target temporal evolution feature perception network extracts the initial spatiotemporal features of the image sequence through a lightweight three-dimensional convolutional network, enhances the target response and suppresses star clutter. By constructing a dual-branch structure, it models the temporal evolution features of the target and enhances the spatial region features of the target. A gate-based dual attention feature fusion module is used to selectively enhance and fuse the features output by the branch structure, thereby improving the joint expression ability of position and motion information. The input to the temporal evolution feature perception network for small targets in dim space is a sequence of 5 consecutive image frames, and the output is the target prediction bounding box of the last image in the current image sequence. The network includes: an initial feature extraction module, a spatiotemporal dual-branch feature extraction module, a gated dual-attention feature fusion module, and a target detection head. The initial feature extraction module is used for a continuous 5-frame image sequence of the spatial target to be detected. Initial spatiotemporal feature extraction is performed to obtain the corresponding initial spatiotemporal features. ; The spatiotemporal dual-branch feature extraction module is used to extract the target temporal and spatial features separately through a temporal evolution feature perception branch and a spatial region enhancement branch based on cross-attention, to obtain temporal features. and spatial characteristics ; The gated dual attention feature fusion module is used for temporal features. and spatial characteristics By fusing the data, the spatiotemporal correlation features of the target are obtained. ; The target detection head is used to detect the spatiotemporal correlation features of the target. The final target prediction box result is obtained through processing.

2. The method for detecting small targets in dim space based on spatial guidance according to claim 1, characterized in that, The initial feature extraction module includes: a time-dimensional stack, three sets of alternating cascaded three-dimensional convolutional blocks and downsampling, and two two-dimensional convolutional blocks; wherein, the three-dimensional convolutional block includes a three-dimensional convolutional layer, batch normalization and non-linear activation function connected in sequence; the downsampling includes a three-dimensional convolutional layer with a stride of 2 connected in sequence, batch normalization and non-linear activation function, and the two-dimensional convolutional block includes a two-dimensional convolutional layer, batch normalization and non-linear activation function connected in sequence; The specific processing steps include: Input a sequence of 5 consecutive images ,in, As a reference frame, As keyframes, stacking them along the time dimension yields the spatiotemporal tensor. ,in, The number of channels in the input image, For image row number, Number of columns in the image; After three sets of alternating cascaded 3D convolutional blocks and downsampling, the spatiotemporal features of the target are obtained. Perform time-dimensional feature decomposition to obtain the first... i Spatiotemporal features corresponding to frame images ,in ; Will By iteratively inputting two concatenated two-dimensional convolutional blocks, we obtain the... i Initial spatiotemporal features of frame images .

3. The method for detecting small targets in dim space based on spatial guidance according to claim 1, characterized in that, The spatial region enhancement branch based on cross-attention includes: feature connection, two cascaded two-dimensional convolutional blocks, three parallel two-dimensional convolutional blocks, cross-attention, two-dimensional convolutional blocks, three parallel two-dimensional convolutional blocks, and parameter normalization. The specific processing steps include: Initial spatiotemporal features of the reference frame First, feature concatenation along the channel dimension is performed. Then, spatial difference features of the reference frame are extracted using two cascaded 2D convolutional blocks. Next, features for the query Q corresponding to the keyframe, and the key K and value V corresponding to the reference frame are generated using three parallel 2D convolutional blocks. Finally, based on an attention mechanism, keyframe spatial features weighted by cross-attention coefficients are obtained. ; Initial spatiotemporal features of keyframes and Residual connections are performed, and the residually connected features are passed through three parallel 2D convolutional blocks to extract multi-scale features of the target region. The multi-scale features are then summed and normalized to obtain spatial weighting coefficients. These coefficients are used to weight the keyframe features, outputting the spatial domain features. .

4. The method for detecting small targets in dim space based on spatial guidance according to claim 1, characterized in that, The temporal evolution feature perception branch includes: a temporal memory network and a temporal context guidance module, wherein... The temporal memory network includes: feature connections, two-dimensional convolutional blocks, and multi-layer cascaded memory blocks; The temporal context guidance module includes: feature connection, channel rearrangement, and two-dimensional convolutional block.

5. The method for detecting small targets in dim space based on spatial guidance according to claim 4, characterized in that, The processing steps of the temporal memory network include: Initial spatiotemporal features of the first frame image First, feature alignment is performed using two-dimensional convolutional blocks, and then the hidden state of the Lth layer is obtained through L layers of memory blocks. and memory state ; Initial spatiotemporal features of other frames ,in First, compare it with the hidden state of the Lth layer in the previous frame. Feature concatenation is performed along the channel dimension, followed by feature alignment using a 2D convolutional block. Then, the hidden states of each memory block from the previous frame are updated to obtain the updated hidden states of the Lth layer. and memory state ; The hidden states of the Lth layer obtained in each frame and the memory state of the Lth layer of the keyframe The local temporal features of each frame are obtained by passing each frame through a two-dimensional convolutional block. Global temporal features of image sequences .

6. The method for detecting small targets in dim space based on spatial guidance according to claim 5, characterized in that, The processing procedure of the time context guidance module includes: Local temporal features of each frame image Global temporal features of image sequences First, feature enhancement is performed through channel-location collaborative attention, followed by feature concatenation and channel rearrangement. The enhanced features after channel rearrangement are processed by a cascaded two-dimensional convolutional block to obtain more time-sensitive features. ,in ; Will Feature concatenation is performed, and the temporal features output by the temporal evolution feature-aware branch are obtained through two cascaded two-dimensional convolutional blocks. .

7. The method for detecting small targets in dim space based on spatial guidance according to claim 1, characterized in that, The gated dual-attention feature fusion module includes: channel-dimensional average pooling, channel-dimensional max pooling, global average pooling, feature concatenation, two-dimensional convolutional blocks, and parameter normalization; the specific processing steps include: Temporal characteristics Two two-dimensional features are obtained through two parallel channel-dimensional average pooling and channel-dimensional max pooling operations. These two two-dimensional features are then concatenated, followed by a two-dimensional convolutional block and parameter normalization to obtain spatial attention coefficients. Finally, the temporal features are processed... Weighting is performed to obtain the temporal features after selective feature enhancement. ; airspace features First, a global average pooling is applied, followed by two cascaded 2D convolutional blocks and a parameter normalization to obtain channel attention coefficients for spatial features. Weighted summation yields spatial features enhanced with selective features. ; Enhanced temporal features and spatial characteristics The weighting coefficients are obtained by passing a feature connection, a cascaded two-dimensional convolutional block, and a cascaded parameter normalization. ,right and Complementary fusion is performed to obtain spatiotemporal correlation features through two cascaded two-dimensional convolutional blocks. .

8. The method for detecting small targets in dim space based on spatial guidance according to claim 1, characterized in that, The target detection head includes: a two-dimensional convolutional block, three parallel detection branches, and a feature connection. The first and second detection branches are each composed of three cascaded two-dimensional convolutional blocks, and the third detection branch is composed of a two-dimensional convolutional block. The three detection branches output the target regression result, classification result, and confidence result, respectively. After being connected in the channel dimension through the feature connection, the final target prediction box result is obtained.

9. The method for detecting small targets in dim space based on spatial guidance according to claim 1, characterized in that, The method also includes training a perceptual network for the temporal evolution features of small targets in dim space. The overall loss function of the training is the sum of three parts: regression loss, confidence loss, and classification loss. The regression loss uses the intersection-over-union loss function; the confidence loss and classification loss both use the binary cross-entropy loss function. The Adam optimizer is used to update and predict network parameters, with a learning rate of [missing information]. Set to 1e-4, with a training batch size of 4; The training includes the step of creating a training set, specifically including: Step A1: Set the observation position, observation start and end time, line of sight pointing, observation field of view, maximum detection distance, limiting detection magnitude, camera aperture, exposure time, optical system quantum efficiency, optical transmittance, camera gain, camera grayscale bias, readout noise, dark current noise, image sensor target size, image resolution, number of image frames, and target imaging signal-to-noise ratio simulation parameters for the space-based long-distance passive optical observation platform; Step A2: Based on the star catalog and all parameters set in Step A1, select the visible stars in the field of view, and use the position, proper motion, and precession information in the star catalog to correct the star positions, and then transform them into the imaging coordinate system; Step A3: Calculate the imaging grayscale values ​​of visible stars in the field of view using the apparent magnitude information and simulation parameters in the star catalog, and use the Gaussian point spread function to generate star points with different spread degrees and widths. Add these points to the pure black background image according to the imaging coordinates to generate a background star map. Step A4: Establish a noise model for the known distribution of imaging noise. Use zero-mean Gaussian white noise to model the imaging noise, generate imaging noise with different means and variances, and superimpose it onto the background star image to generate a background image. Step A5: Calculate the positions of different targets with known orbital information, obtain the position information in the imaging coordinate system through coordinate transformation, and derive the target imaging grayscale value as follows based on the set target imaging signal-to-noise ratio, Gaussian point spread function, and the mean and standard deviation of the generated background image: ； ； in, The average gray value of a single target in the image. The grayscale standard deviation of the generated background image. The average grayscale value of the background image. The target imaging signal-to-noise ratio is set in the simulation parameters. The image grayscale value at the center point of the target. The number of pixels occupied by the target. The imaging coordinates at the center of the target. It is the natural logarithm. For the target imaging region k One point, For the target imaging region k The imaging coordinates of each point; Step A6: Overlay the grayscale values ​​of the target region onto the generated background image to obtain a sequence of target simulation images, and overlay the target as a label onto the pure black image; Step A7: Construct a dataset from the generated target simulation images and their labels, and divide it into a training set, a validation set, and a test set.

Images