Feature point detection method, device and equipment for ISAR images of space targets

Abstract

This application relates to a method, apparatus, and device for feature point detection in ISAR images of space targets. It constructs a primitive image dataset and a space target dataset through simulation using physical optics. A basic feature point detection model is trained using the primitive image dataset. Then, each simulated space target image in the space target dataset undergoes multiple affine transformations. The trained basic feature point detection model is then used to detect basic feature points in the transformed images, obtaining pseudo-ground value labels for the corresponding simulated space target images. The feature point detection model is then trained again using the space target dataset and the corresponding pseudo-ground value labels. Finally, the trained feature point detection model is directly used to detect ISAR images, thus obtaining the feature points of the space targets. This method overcomes the problem of insufficient labeled images during training while improving the accuracy of space target detection results.

Description

Technical Field

[0001] This application relates to the field of ISAR image target recognition technology, and in particular to a method, apparatus and device for feature point detection of ISAR images of space targets. Background Technology

[0002] Inverse Synthetic Aperture Radar (ISAR) can operate around the clock and in all weather conditions and is currently widely used in the field of space target surveillance. By observing space targets, ISAR can obtain high-resolution ISAR images of the targets. During ISAR imaging, increasing the transmission signal bandwidth can improve the radial resolution of the ISAR image, while accumulating a large rotation angle through long-term observation can improve the lateral resolution of the ISAR image. An ISAR image is a two-dimensional projection of the target onto the radar imaging plane, reflecting the target's contour shape and scattering characteristics. Therefore, detecting feature points from ISAR images is a very important task, providing technical support for subsequent space target attitude estimation and 3D reconstruction.

[0003] Traditional feature point detection methods perform poorly when dealing with the low texture and uneven brightness characteristics of ISAR images. Neural networks, due to their nonlinear mapping capabilities, can effectively learn features in ISAR images of space targets, thereby improving the performance of feature point detection.

[0004] However, the availability of labeled ISAR image data of spatial targets is relatively scarce, which poses a challenge to the detection of target feature points using neural networks. Summary of the Invention

[0005] Therefore, it is necessary to provide a feature point detection method, apparatus, and device for space target ISAR images that can solve the problem of the relative scarcity of labeled space target ISAR image data, in order to address the above-mentioned technical problems.

[0006] A feature point detection method for ISAR images of space targets, the method comprising:

[0007] Based on various different spatial target primitive structures and spatial targets, multiple primitive structure ISAR simulation images and spatial target ISAR simulation images are obtained by using the physical optics method. Corresponding ISAR primitive image datasets and spatial target ISAR datasets are then constructed.

[0008] The basic feature point detection model is trained using the ISAR primitive image dataset to obtain the trained basic feature point detection model;

[0009] After performing multiple affine transformations on each space target ISAR simulation image in the space target ISAR dataset, the trained basic feature point detection model is used to detect basic feature points on the images after multiple affine transformations, and the pseudo-true value label of the corresponding space target ISAR simulation image is obtained based on multiple detection results.

[0010] The feature point detection model is trained using the aforementioned space target ISAR dataset and the corresponding pseudo-ground value labels, resulting in the trained feature points detected by the model.

[0011] An ISAR image of a space target is acquired, and the ISAR image is input into the trained feature point detection model to perform feature point detection, thereby obtaining multiple feature points of the space target in the ISAR image.

[0012] In one embodiment, when simulating the basic structure of a space target using physical optics:

[0013] The spatial target primitive structure includes cylindrical structure, cubic structure and planar structure, and the spatial target primitive structure includes multiple scales;

[0014] The true values ​​of feature point locations are labeled for each of the simulated ISAR images of the primitive structure obtained from the simulation.

[0015] In one embodiment, the basic feature point detection model is a fully convolutional neural network. The convolutional kernel size in the basic feature point detection model is 3×3, batch normalization is used for data normalization, the nonlinear activation function is the ReLU function, and the fully connected convolutional layer is a 1×1 convolution.

[0016] In one embodiment, multiple affine transformations, including translation, rotation, and scaling operations, are performed on each space target ISAR simulation image in the space target ISAR dataset.

[0017] In one embodiment, the trained basic feature point detection model is used to detect basic feature points in the image after multiple affine transformations, and the pseudo-ground value label of the corresponding space target ISAR simulation image is obtained based on multiple detection results, including:

[0018] The pseudo-true value label of a space target ISAR simulation image is obtained by aggregating the basic feature point detection results of the images after performing different affine transformations on the simulation image.

[0019] In one embodiment, the feature point detection model includes a shared encoder and two independent decoders, wherein the two decoders are a feature point detection decoder for detecting feature points in an image and a feature point descriptor decoder for calculating feature point descriptors.

[0020] The shared encoder includes a convolutional layer, a downsampling pooling layer, and a nonlinear activation function connected in sequence.

[0021] The feature point detection decoder adopts an explicit decoding structure and outputs a feature point probability image.

[0022] The feature point descriptor decoder uses a UCN network structure to generate sparse descriptors, and then performs bicubic interpolation and L2 normalization on the sparse descriptors to obtain unit pixel descriptors of uniform length.

[0023] In one embodiment, when training the feature point detection model, the loss function used is the total loss function after balancing the feature point detection decoder loss and the feature point descriptor decoder loss through weight coefficients.

[0024] This application also provides a feature point detection device for ISAR images of space targets, the device comprising:

[0025] The ISAR dataset construction module is used to simulate multiple primitive structure ISAR images and space target ISAR images based on various different space target primitive structures and space targets using the physical optics method, and to construct the corresponding ISAR primitive image dataset and space target ISAR dataset respectively.

[0026] The basic feature point detection model training module is used to train the basic feature point detection model using the ISAR primitive image dataset to obtain the trained basic feature point detection model.

[0027] The pseudo-true value labeling module is used to perform multiple affine transformations on each space target ISAR simulation image in the space target ISAR dataset, and then use the trained basic feature point detection model to detect basic feature points on the images after multiple affine transformations, and obtain the pseudo-true value label of the corresponding space target ISAR simulation image based on multiple detection results.

[0028] The feature point detection model training module is used to train the feature point detection model using the space target ISAR dataset and the corresponding pseudo-ground value labels to obtain the trained feature point detection model.

[0029] The feature point detection module is used to acquire ISAR images of space targets, input the ISAR images into the trained feature point detection model to perform feature point detection, and obtain multiple feature points of space targets in the ISAR images.

[0030] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0031] Based on various different spatial target primitive structures and spatial targets, multiple primitive structure ISAR simulation images and spatial target ISAR simulation images are obtained by using the physical optics method. Corresponding ISAR primitive image datasets and spatial target ISAR datasets are then constructed.

[0032] The basic feature point detection model is trained using the ISAR primitive image dataset to obtain the trained basic feature point detection model;

[0033] After performing multiple affine transformations on each space target ISAR simulation image in the space target ISAR dataset, the trained basic feature point detection model is used to detect basic feature points on the images after multiple affine transformations, and the pseudo-true value label of the corresponding space target ISAR simulation image is obtained based on multiple detection results.

[0034] The feature point detection model is trained using the aforementioned space target ISAR dataset and the corresponding pseudo-ground value labels, resulting in the trained feature points detected by the model.

[0035] An ISAR image of a space target is acquired, and the ISAR image is input into the trained feature point detection model to perform feature point detection, thereby obtaining multiple feature points of the space target in the ISAR image.

[0036] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0037] Based on various different spatial target primitive structures and spatial targets, multiple primitive structure ISAR simulation images and spatial target ISAR simulation images are obtained by using the physical optics method. Corresponding ISAR primitive image datasets and spatial target ISAR datasets are then constructed.

[0038] The basic feature point detection model is trained using the ISAR primitive image dataset to obtain the trained basic feature point detection model;

[0039] After performing multiple affine transformations on each space target ISAR simulation image in the space target ISAR dataset, the trained basic feature point detection model is used to detect basic feature points on the images after multiple affine transformations, and the pseudo-true value label of the corresponding space target ISAR simulation image is obtained based on multiple detection results.

[0040] The feature point detection model is trained using the aforementioned space target ISAR dataset and the corresponding pseudo-ground value labels, resulting in the trained feature points detected by the model.

[0041] An ISAR image of a space target is acquired, and the ISAR image is input into the trained feature point detection model to perform feature point detection, thereby obtaining multiple feature points of the space target in the ISAR image.

[0042] The aforementioned feature point detection method, apparatus, and device for ISAR images of space targets utilize physical optics to simulate and construct ISAR primitive image datasets and space target ISAR datasets. First, a basic feature point detection model is trained using the ISAR primitive image dataset to obtain the trained model. Then, multiple affine transformations are performed on each simulated ISAR image of a space target in the space target ISAR dataset. The trained model is then used to detect basic feature points on these transformed images, obtaining pseudo-ground truth labels for the corresponding simulated ISAR images. The feature point detection model is then trained again using the space target ISAR dataset and the corresponding pseudo-ground truth labels. Finally, the trained model is directly used to detect ISAR images, thus obtaining the feature points of the space targets. This method overcomes the problem of insufficient labeled images during neural network training while improving the accuracy of ISAR space target detection results. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating a feature point detection method for ISAR images of space targets in one embodiment.

[0044] Figure 2 This is a schematic diagram of spatial target motion decomposition in one embodiment;

[0045] Figure 3 This is a schematic diagram of an ISAR observation and imaging model of a space target in one embodiment;

[0046] Figure 4 This is a schematic diagram of the ISAR image projection mechanism in one embodiment;

[0047] Figure 5 This is a schematic diagram of the framework of a feature point detection method for ISAR images of space targets in one embodiment;

[0048] Figure 6 This is a schematic diagram of an ISAR primitive image in one embodiment. Figure 6 (a) is an ISAR image obtained from simulation with a cube as the basic structure. Figure 6 (b) is an ISAR image obtained from a planar simulation of the primitive structure;

[0049] Figure 7 This is a schematic diagram of the basic feature point detection model structure in one embodiment;

[0050] Figure 8 This is a schematic diagram of the feature map channels in one embodiment;

[0051] Figure 9 This is a schematic diagram of affine transformation in one embodiment;

[0052] Figure 10 This is a schematic diagram of affine transformation decomposition in one embodiment;

[0053] Figure 11 This is a schematic diagram of the feature point detection model in one embodiment;

[0054] Figure 12 This is a schematic diagram of a shared encoder structure in one embodiment;

[0055] Figure 13 This is a schematic diagram illustrating the feature point detection performance of the SIFT, SURF, KLT, and SFPD algorithms in an experiment. Figure 13 (a) is a schematic diagram showing the number of feature points detected. Figure 13 (b) is a schematic diagram of feature point detection accuracy;

[0056] Figure 14 Here are example images showing the feature point detection results of the SIFT, SURF, KLT, and SFPD algorithms in an experiment. Figure 14 (a) Figure 14 (b) Figure 14 (c) and Figure 14 (d) Examples of feature point detection results for SIFT algorithm, SURF algorithm, KLT algorithm and SFPD model respectively;

[0057] Figure 15 This diagram illustrates the feature point matching accuracy of the SIFT, SURF, KLT, and SFPD algorithms under different matching thresholds in an experiment. Figure 15 (a) Figure 15 (b) Figure 15 (c) These are schematic diagrams showing the feature point matching accuracy when the matching threshold is 1 pixel, 3 pixels, and 5 pixels, respectively.

[0058] Figure 16 This diagram illustrates the feature point matching recall rates of the SIFT, SURF, KLT, and SFPD algorithms under different matching thresholds in an experiment. Figure 16 (a) Figure 16 (b) Figure 16 (c) These are schematic diagrams showing the feature point matching recall rates when the matching threshold is 1 pixel, 3 pixels, and 5 pixels, respectively.

[0059] Figure 17 This diagram illustrates the F1 scores of feature point matching using the SIFT, SURF, KLT, and SFPD models at different matching thresholds in an experiment. Figure 17 (a) Figure 17 (b) Figure 17 (c) These are schematic diagrams of the F1 scores for feature point matching with a matching threshold of 1 pixel, a matching threshold of 3 pixels, and a matching threshold of 5 pixels, respectively.

[0060] Figure 18 Here are example images of feature point matching using the SIFT, SURF, KLT, and SFPD models when the matching threshold is set to 3 pixels in an experiment. Figure 18 (a) Figure 18 (b) Figure 18 (c) and Figure 18 (d) are example diagrams of feature point matching for SIFT algorithm, SURF algorithm, KLT algorithm and SFPD model respectively;

[0061] Figure 19 This is a structural block diagram of a feature point detection device for ISAR images of space targets in one embodiment.

[0062] Figure 20 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0064] In existing technologies, the nonlinear mapping capabilities of neural networks are used to extract features from ISAR images. By learning complex image feature representations, neural networks can effectively capture detailed information in ISAR images, thereby improving the accuracy of feature point detection. For example, one ISAR image keypoint feature extraction network can detect the corner points of a sailboat and the endpoints of a subject in ISAR images, and the effectiveness of keypoint extraction has been verified through pose estimation experiments. Another network is a multi-scale semantic feature extraction network, which can extract semantic features from ISAR images as keypoints, improving the reliability of ISAR feature extraction. Yet another method is a deep learning-based instance segmentation method used to extract structural keypoints from associated ISAR images. However, current methods for extracting feature points from ISAR images using neural networks still face some limitations. As can be seen from the above, current research mainly focuses on extracting structural keypoints from ISAR images, failing to cover all feature points in ISAR images. Therefore, these methods are not suitable for downstream tasks such as 3D reconstruction. There are two main reasons for this phenomenon: First, there are relatively few publicly available ISAR datasets, which do not support extensive research and algorithm verification; second, due to the anisotropy of scatterers, it is difficult to accurately calibrate the true location of feature points when creating ISAR image datasets, which limits the applicability of supervised neural network methods in detecting ISAR image feature points.

[0065] In response to the above problems, such as Figure 1 As shown, a feature point detection method for ISAR images of space targets is provided, including the following steps:

[0066] Step S100: Based on various different spatial target primitive structures and spatial targets, multiple primitive structure ISAR simulation images and spatial target ISAR simulation images are obtained by using physical optics method, and corresponding ISAR primitive image datasets and spatial target ISAR datasets are constructed respectively.

[0067] Step S110: Train the basic feature point detection model using the ISAR primitive image dataset to obtain the trained basic feature point detection model.

[0068] Step S120: After performing multiple affine transformations on each space target ISAR simulation image in the space target ISAR dataset, the trained basic feature point detection model is used to detect basic feature points on the images after multiple affine transformations, and the pseudo-true value label of the corresponding space target ISAR simulation image is obtained based on multiple detection results.

[0069] Step S130: Use the ISAR dataset of spatial targets and the corresponding pseudo-ground value labels to train the feature point detection model, and obtain the trained feature point detection model.

[0070] Step S140: Obtain the ISAR image of the space target, input the ISAR image into the trained feature point detection model to perform feature point detection, and obtain multiple feature points of the space target in the ISAR image.

[0071] This application proposes a self-supervised feature point detection method for ISAR images of space targets. This method can accurately detect the location of feature points in ISAR images of space targets without providing ground truth feature point locations during the training of the neural network. The method first simulates the ISAR primitive dataset and uses this dataset to train the proposed basic feature point detection model. Then, the basic feature point detection model is used to annotate the pseudo-ground truth values ​​of the ISAR images of space targets, and an affine transformation method is used to improve the accuracy of the annotation. Finally, the feature point detection model is trained on the ISAR image dataset of space targets. Experimental results show that the proposed method can be trained without ground truth values ​​from the ISAR image dataset of space targets and exhibits good feature point detection performance. Furthermore, experimental results verify the effectiveness of the proposed method; compared with traditional feature point detection methods, the proposed method achieves higher feature point detection accuracy and better feature point matching performance.

[0072] In this embodiment, the method is as follows: Figure 2 As shown, the method comprises three parts. First, a basic feature point detection model is trained on a spatial target primitive dataset. Next, affine transformations are combined with the basic feature point detection model to automatically label the spatial target ISAR dataset, generating pseudo-ground values. Finally, the proposed fully convolutional neural network is trained using the generated labels and the spatial target ISAR dataset. This network can extract feature point locations and their corresponding feature descriptors from spatial target ISAR images.

[0073] In step S100, the datasets needed to train the two neural networks are first constructed. Both datasets are constructed using simulated ISAR images obtained from the simulation of the ISAR imaging model of space targets based on the physical optics method.

[0074] In this embodiment, the space target ISAR imaging model used in the physical optics method is constructed as follows:

[0075] In inverse synthetic aperture radar (ISAR) imaging of space targets, the distance between the observed space target and the radar is relatively large, and the target size is very small relative to this distance, satisfying the far-field condition. Therefore, the electromagnetic waves received by the target can be considered as plane waves. Under this assumption, the motion of the space target can be described as follows: Figure 3The diagram shows the decomposition process, where the red trajectory represents the target's motion trajectory, and the blue trajectory represents the decomposed motion trajectory. Assuming a spatial target moves from point A to point C, this motion can be decomposed into three parts. First, the target rotates around its geometric center, causing each scattering point on the target to produce a different Doppler effect. Second, the target translates from point A to point B. During this process, the distance between each scattering point and the radar changes consistently, resulting in a consistent Doppler effect, which can be corrected through translational compensation. Finally, from point B to point C, the target rotates in a circular motion around the radar center. During this process, the distance between the target and the radar remains constant, and no Doppler effect occurs.

[0076] After translational compensation is completed, the ISAR observation imaging model is as follows: Figure 4 As shown, O-XYZ is the target body coordinate system, and O is the target's equivalent rotation center. Figure 4 In the diagram, the red line represents the radar line-of-sight (LOS), which is determined by the azimuth and elevation angles, and can be represented as:

[0077] r(t)=[-cosφ(t)cosθ(t),-cosφ(t)sinθ(t),-sinφ(t)] T (1)

[0078] In formula (1), θ(t) and φ(t) are the azimuth and elevation angles at time t. k Let p be a scattering point on the target. k =(x k ,y k ,z k ) T Assuming the distance between the radar and the target's equivalent center of rotation is r0(t), then the feature point p k The distance to the radar can be expressed as:

[0079] r p (t)=r0(t)+r(t) T p k (2)

[0080] In formula (2), for r p Taking the derivative of (t) yields the instantaneous velocity of the feature point, which can be expressed as:

[0081]

[0082] In formula (3), Indicates r p (t) is the derivative with respect to t, and the other signs are similar. In the body coordinate system, the position and direction of the scattering point do not change with the rotation of the target, so formula (3) can be expressed as:

[0083]

[0084] In the early formula (4), × is the cross product symbol for vectors, and ω radar (t) represents the radar's rotational angular velocity relative to the target. ω radar (t) can be decomposed along LOS into an angular velocity ω perpendicular to LOS. n (t) and the angular velocity ω parallel to LOS r (t). The parallel component of the rotational angular velocity along LOS does not affect the calculation results, therefore formula (4) can be expressed as:

[0085]

[0086] In formula (5), 2 represents the second norm of the vector.

[0087] Based on the above analysis, the imaging plane of an ISAR image can be represented as:

[0088]

[0089] In formula (6), ρ r With ρ a The projection vectors of the scattering point in the range and azimuth directions are represented by the normal vector of the imaging plane, which is perpendicular to ω. n The directions of (t) are consistent. Therefore, the projection relationship of the scattering points onto the ISAR image can be expressed as:

[0090]

[0091] r k With a k For the scattering point p k Coordinates in the range and azimuth directions of the ISAR image. Figure 5 The diagram below illustrates the projection mechanism of ISAR images.

[0092] In this embodiment, it is worth noting that there is currently no large-scale ISAR image feature point labeling dataset for spatial targets. To train the BasicPoint detection model, an ISAR primitive image dataset was constructed. This dataset was built based on simulations of spatial target primitive structures, including cylindrical, cubic, and planar structures. Furthermore, to ensure the model trained on the ISAR primitive image dataset has good data generalization ability and can learn feature representations of targets of different sizes, multi-scale spatial target primitive structures were designed and introduced. Additionally, ground-valued feature point locations in the ISAR primitive images were labeled during the simulation. Figure 6An example of an ISAR primitive image dataset is given in the document.

[0093] In addition, the process of constructing the ISAR dataset for space targets is the same as that of constructing the ISAR primitive image dataset, except that the primitive structure of the space targets is replaced with space targets, such as satellites and spacecraft.

[0094] Furthermore, after obtaining the ISAR primitive image dataset, in step S110, the BasicPoint detection model is trained on this dataset. BasicPoint can effectively detect the location of feature points in ISAR primitive images, and its network structure is as follows: Figure 7 As shown in the image.

[0095] from Figure 7 As can be seen, BasicPoint only accepts single-channel images as input. During training, the images in the ISAR primitive data set need to be converted to grayscale images before being input into BasicPoint. Traditional dense prediction networks typically employ an encoder-decoder structure, where the encoder uses convolution or pooling operations to reduce data dimensionality, and then the decoder restores the data to its original dimensionality through upsampling. However, upsampling significantly increases computational cost, which is detrimental to model training. Therefore, in this embodiment, BasicPoint uses an explicit decoder structure to restore data dimensionality and reduce model computation. First, BasicPoint uses a VGG encoder to encode the M×N ISAR primitive image into an M / 8×N / 8 feature map, where each point contains 65 channels. Then, a Softmax layer converts the feature map into probabilities.

[0096] Specifically, the meaning of each channel in the feature map is as follows: Figure 8 As shown, the first 64 channels in the feature map correspond to the probability that each pixel in an 8×8 local pixel region of the input image may be a feature point. The 65th channel is a discard channel, representing that no feature point was detected in this local pixel region. To restore the data to its original dimensions, BasicPoint discards the 65th channel in the feature map and then transforms the 64×M / 8×N / 8 feature map into an M×N feature map through a Reshape layer. Each element at a position in the feature map corresponds to the probability that that position is a feature point in the input image. Finally, the ISAR primitive image is mapped onto a feature point response image with the same resolution. The feature point response image is presented as a heatmap, and the detected feature points show obvious red features in the feature point response image, forming a clear contrast with the surrounding points.

[0097] Specifically, the BasicPoint detection model is a fully convolutional neural network. The convolutional kernel size in the BasicPoint detection model is 3×3. Batch normalization is used for data normalization. The non-linear activation function is the ReLU function. The fully connected convolutional layer is a 1×1 convolution.

[0098] In this embodiment, when training the basic feature point detection model, cross-entropy is used as the loss function, which can be expressed as:

[0099]

[0100] In formula (8). p m,n Let q be the probability distribution of the true feature points. m,n Here, M and N represent the feature point probability distribution detected by BasicPoint, where M and N are the dimensions of the feature point probability distribution map.

[0101] To provide training data for the feature point detection model, pseudo-ground values ​​of the space target ISAR image dataset are labeled using the trained BasicPoint. Since BasicPoint is trained on the ISAR primitive image dataset, it cannot detect all feature points in the space target ISAR images, leading to poor performance of the feature point detection model. To address this issue, in step S120, an affine transformation is performed on each image in the space target ISAR dataset.

[0102] Affine transformation is a geometric transformation composed of linear transformations and translations, including operations such as translation, rotation, and scaling. When performing an affine transformation on an image, the original geometric relationships within the image are not altered. After performing an affine transformation on an ISAR image of a space target, BasicPoint is used to annotate the image, enabling BasicPoint to effectively detect the locations of feature points in different receptive fields. These annotation results are then accumulated and summed to obtain the pseudo-true values ​​of the image's feature point locations. A schematic diagram of the affine transformation is shown in Figure 9.

[0103] Specifically, multiple affine transformations, including translation, rotation, and scaling operations, are performed on each space target ISAR simulation image in the space target ISAR dataset.

[0104] Furthermore, by utilizing the trained basic feature point detection model to detect basic feature points in images after multiple affine transformations, and obtaining the pseudo-ground truth label for the corresponding space target ISAR simulation image based on multiple detection results, the process involves: detecting the basic feature points of a space target ISAR simulation image after performing different affine transformations; then inversely transforming the feature point results detected on the affine images back to the original image. This allows for the acquisition of multiple feature points under different receptive fields from multiple affine images, and the aggregation of these feature points from different receptive fields yields the pseudo-ground truth label for the space target ISAR simulation image.

[0105] Specifically, assuming f() is the basic feature point detection function (i.e., the basic feature point detection network), I is the input image, and P is the set of detected feature points, it can be expressed as:

[0106] P = f(I) (9)

[0107] An affine transformation of an input image can be represented as:

[0108] AP=f(A(I)) (10)

[0109] In formula (10), A is the affine transformation matrix, and A(I) is the affine transformed ISAR image of the spatial target. A is multiplied by A on both sides of formula (10). -1 Another representation of the feature point set P can be obtained:

[0110] P = A -1 f(A(I)) (11)

[0111] Therefore, by summing up a large number of affine transformation samples, a better feature point detection function can be obtained, which can be expressed as:

[0112]

[0113] In formula (12), N a Let A be the order of the affine transformation. i Let be the affine transformation matrix of the i-th order.

[0114] It's important to note that not all 3×3 matrices are suitable as affine matrices. To select a suitable affine matrix, the affine transformation is decomposed, performing simple transformations such as rotation, translation, and scaling within a predetermined range. These simple transformations are then combined with the initial clipping center. The specific steps are as follows: Figure 10 As shown above, the number of affine transformations affects the feature point detection performance; therefore, N... a N represents the hyperparameter of the method proposed in this paper. N is determined through corresponding experiments. a The optimal value, Na The optimal value is 1000.

[0115] In step S130, the structure of the Feature Point Detection Model (SFPD) is as follows: Figure 11 As shown, the network includes a shared encoder and two independent decoders. The two decoders are adapted for different downstream tasks: one is a feature point detection decoder for detecting feature points in an image, and the other is a feature point descriptor decoder for calculating feature point descriptors. Traditional feature point detection algorithms typically employ a pipeline approach, first performing feature point detection and then generating the corresponding descriptors. However, this approach suffers from insufficient coupling between tasks. In contrast, the proposed SFPD does not have this problem. The network design allows most network parameters to be shared between the two different decoders, and feature point detection and descriptor computation can be performed simultaneously during the forward pass.

[0116] In this embodiment, a VGG encoder is used in the feature point detection model to reduce the data dimensionality. Figure 12 The paper presents the shared encoder structure of SFPD, which consists of convolutional layers, downsampling pooling layers, and nonlinear activation functions. Specifically, the shared encoder uses a 3×3 convolutional kernel, while three 2×2 downsampling pooling layers are used to reduce the input data size from H×W to H / 8×W / 8. In summary, the shared encoder will... The input image is mapped to a tensor with smaller dimensions and greater channel depth.

[0117]

[0118] The output data of the feature point detection decoder represents the probability that each pixel in the image is a feature point. Typical network structures usually reduce the data dimensionality at the encoder level and then restore it through the decoder. However, the upsampling operation in traditional decoders significantly increases computational cost, affecting network convergence speed. Therefore, in this embodiment, the feature point detection decoder adopts the same explicit decoding structure as BasicPoint. In this structure, the input tensor of the feature point detection decoder... The 65 channels in this model are the same as those in BasicPoint, and will not be described in detail here. Next, the last channel is discarded through a softmax layer, and the tensor of H / 8×W / 8×64 is reshaped into a feature point probability image of H×W.

[0119] In this embodiment, the feature point descriptor decoder uses a UCN network structure to generate sparse descriptors, and then performs bicubic interpolation and L2 normalization on the sparse descriptors to obtain unit pixel descriptors of uniform length.

[0120] Specifically, the input to the feature point descriptor decoder is a tensor. The output is a descriptor of length H×W×D. To reduce computation, the feature point descriptor decoder first uses the UCN

[44] network structure to generate sparse descriptors of 8 pixels each, and then performs bicubic interpolation and L2 normalization to obtain a unit pixel descriptor of uniform length.

[0121] In this embodiment, the loss function used when training the feature point detection model is the loss of the feature point detection decoder calculated using weight coefficients. and feature point descriptor decoder loss The total loss function after balancing the two parts is expressed as:

[0122]

[0123] Specifically, It can be represented as:

[0124]

[0125] In formula (14), For ISAR images of space targets, for The images obtained after affine transformation, Y and Y′ are the pseudo-true value labels of feature points in the two images. s () represents the cross-entropy loss, which can be expressed as:

[0126]

[0127] In formula (15), y h,w ∈Y.

[0128] Specifically, It can be represented as:

[0129]

[0130] In formula (16), and For the descriptor of the input image pair,

[0131] S represents the correspondence between image pairs. d () represents the hinge loss, which can be expressed as:

[0132]

[0133] In formula (17), the weight λ d Used to balance positive and negative samples, mp With m n The key loss represents the positive and negative boundaries of the network. s represents the correspondence between descriptors, used to determine whether d and d' are related. s can be expressed as:

[0134]

[0135] In formula (18), p h,w Let (h,w) be the center coordinates of the pixel grid corresponding to point (h,w). If the Euclidean distance between corresponding points after the affine transformation is less than 6 pixels, then the two points are considered to be related.

[0136] Finally, in step S140, that is, in the application of the trained feature point detection model, the feature points of the space target can be obtained simply by inputting the measured ISAR image into the model.

[0137] This paper also presents experimental results demonstrating the effectiveness of the proposed method, i.e., the effectiveness of the feature point detection model.

[0138] Specifically, the experiment compared this method with the commonly used feature point detection algorithm SURF (Speeded Up RobustFeatures). Furthermore, it was also compared with the improved SIFT (Scale Invariant Feature Transform) algorithm and the KLT (Kanade-Lucas-Tomasi) algorithm. The test data consisted of 100 randomly selected ISAR images of space targets from the test set. The experiment used the number of correctly detected feature points and the feature point detection accuracy as evaluation metrics. Figure 13Specific experimental results are presented. The results show that the proposed method outperforms the other three methods in both the number of correctly detected feature points and the feature point detection accuracy, fully demonstrating the effectiveness of the proposed method. Meanwhile, observing the experimental results of the other three methods reveals that the SIFT and SURF algorithms have lower numbers of correctly detected feature points and lower feature point detection accuracy. This is mainly due to the significant differences in imaging mechanisms between optical images and ISAR images, resulting in significant differences in their image features and a higher likelihood of false positives and false negatives during feature point detection. In contrast, although the KLT algorithm detects fewer feature points correctly, its detection accuracy surpasses that of the SIFT and SURF algorithms. This indicates that the KLT algorithm's feature point detection performance is superior to the SIFT and SURF algorithms because false positives have a greater impact than false negatives in downstream tasks of feature point detection. The statistical results of the feature point detection performance of the four algorithms are shown in Table 1. Compared to the other three methods, the SFPD model improves the number of correctly detected feature points by more than 96.8% and the feature point detection accuracy by more than 47.48%. Furthermore, in Figure 14 The document demonstrates feature point detection examples using the SIFT, SURF, KLT, and SFPD algorithms.

[0139] Table 1. Statistical results of feature point detection performance of SIFT, SURF, KLT, and SFPD algorithms.

[0140]

[0141] Furthermore, considering that in space target surveillance missions, feature point detection often serves as a prerequisite for downstream tasks such as feature point matching and 3D reconstruction, the proposed method can not only detect the location of feature points in ISAR images of space targets but also output the feature descriptors corresponding to the feature point locations, facilitating subsequent feature point matching. To verify the feature descriptor capability of the proposed method, 50 images were randomly selected from the test set, and affine transformations were randomly performed on each image to obtain 50 pairs of images to be matched. The proposed method and the three algorithms mentioned above were used to perform feature point matching on these 50 pairs of images. Precision, recall, and F1 score were used as evaluation metrics in the experiment. In addition, the following evaluation criteria were designed to judge whether the feature point matching was correct:

[0142]

[0143] In the above formula, p represents the feature point detected in the original image, p' represents the feature point in the image after affine transformation, and A is the affine transformation matrix between the two images. If the Euclidean distance between a feature point in the original image and its matching point after affine transformation is less than the matching threshold, it is considered a correct match. Feature point matching experiments were conducted with matching thresholds of 1 pixel, 3 pixels, and 5 pixels. The experimental results are as follows: Figures 15 to 17 As shown.

[0144] Experimental results show that the proposed method exhibits good feature point matching performance under different matching thresholds. The results indicate that the smaller the matching threshold, the more significant the performance improvement of the proposed method compared to the other three methods. This demonstrates that the feature descriptors generated by the proposed method can accurately describe the image features of the feature points, resulting in higher feature point matching accuracy. When the matching threshold increases, the algorithm's requirement for matching accuracy becomes less stringent, and the performance improvement of the proposed method relatively decreases. Figures 15 to 17 The experimental results clearly show that the SIFT and SURF algorithms exhibit relatively stable curves, while the performance of the KLT algorithm and the proposed method fluctuates significantly. The KLT algorithm, in particular, achieves a peak F1 score of around 0.9, but its trough is below 0.2. This phenomenon is primarily due to the random affine transformation causing significant viewpoint differences between some matched image pairs. Large viewpoint differences alter the features of image features, impacting the performance of the feature point matching algorithm. Table 2 presents the performance statistics of the feature point matching experiments. As can be seen, even with a matching threshold of 5 pixels, the proposed method achieves an F1 score improvement of over 15% compared to the other three algorithms, further validating the feature point matching performance of the proposed algorithm. To more intuitively demonstrate the feature point matching effect, in... Figure 18 Examples of feature point matching using the SIFT, SURF, KLT algorithms, and SFPD models with a matching threshold of 3 pixels are given.

[0145] Table 2. Statistical results of feature point matching performance of SIFT, SURF, KLT, and SFPD models under different matching thresholds.

[0146]

[0147] The aforementioned feature point detection method for ISAR images of space targets first constructs an ISAR primitive dataset and trains a basic feature point detection model on it. This model can be used to annotate the pseudo-ground truth values ​​of feature point locations in the ISAR image dataset. To improve the accuracy of pseudo-ground truth annotation, this method introduces affine transformation into the pseudo-ground truth annotation process. Feature points in the affine-transformed image are detected, and the feature points are inversely transformed back to the original image. By aggregating feature point locations through multiple transformations, the accuracy of feature point location annotation is improved. Subsequently, the feature point detection model is trained on the annotated ISAR image dataset of space targets. This method first simulates the space target primitive model to generate an ISAR primitive dataset and trains on this dataset to obtain the basic feature point detection model. Simultaneously, combining the basic feature point detection model with affine transformation to annotate the pseudo-ground truth values ​​of ISAR images of space targets improves the confidence of the pseudo-ground truth annotation. This method also proposes a fully convolutional neural network called SFPD, which is trained on the annotated ISAR image dataset of space targets and can output the locations of feature points in ISAR images and their corresponding feature descriptors. Simulation experiments show that our method has higher feature point detection accuracy compared with existing methods, and the feature point detection results are also applicable to subsequent tasks such as feature point matching.

[0148] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0149] In one embodiment, such as Figure 19 As shown, a feature point detection device for ISAR images of space targets is provided, including: an ISAR dataset construction module 200, a basic feature point detection model training module 210, a pseudo-ground value label acquisition module 220, a feature point detection model training module 230, and a feature point detection module 240, wherein:

[0150] The ISAR dataset construction module 200 is used to simulate multiple primitive structure ISAR images and space target ISAR images based on various different space target primitive structures and space targets using the physical optics method, and to construct the corresponding ISAR primitive image dataset and space target ISAR dataset respectively.

[0151] The basic feature point detection model training module 210 is used to train the basic feature point detection model using the ISAR primitive image dataset to obtain the trained basic feature point detection model.

[0152] The pseudo-true value labeling module 220 is used to perform multiple affine transformations on each space target ISAR simulation image in the space target ISAR dataset, and then use the trained basic feature point detection model to detect basic feature points on the images after multiple affine transformations, and obtain the pseudo-true value label of the corresponding space target ISAR simulation image based on multiple detection results.

[0153] The feature point detection model training module 230 is used to train the feature point detection model using the space target ISAR dataset and the corresponding pseudo-ground value labels, so as to obtain the trained feature points detected by the model.

[0154] The feature point detection module 240 is used to acquire ISAR images of space targets, input the ISAR images into the trained feature point detection model to perform feature point detection, and obtain multiple feature points of space targets in the ISAR images.

[0155] Specific limitations regarding the feature point detection device for ISAR images of space targets can be found in the limitations of the feature point detection method for ISAR images of space targets described above, and will not be repeated here. Each module in the aforementioned feature point detection device for ISAR images of space targets can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0156] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 20As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a feature point detection method for ISAR images of space targets. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0157] Those skilled in the art will understand that Figure 20 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0158] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0159] Based on various different spatial target primitive structures and spatial targets, multiple primitive structure ISAR simulation images and spatial target ISAR simulation images are obtained by using the physical optics method. Corresponding ISAR primitive image datasets and spatial target ISAR datasets are then constructed.

[0160] The basic feature point detection model is trained using the ISAR primitive image dataset to obtain the trained basic feature point detection model;

[0161] After performing multiple affine transformations on each space target ISAR simulation image in the space target ISAR dataset, the trained basic feature point detection model is used to detect basic feature points on the images after multiple affine transformations, and the pseudo-true value label of the corresponding space target ISAR simulation image is obtained based on multiple detection results.

[0162] The feature point detection model is trained using the aforementioned space target ISAR dataset and the corresponding pseudo-ground value labels, resulting in the trained feature points detected by the model.

[0163] An ISAR image of a space target is acquired, and the ISAR image is input into the trained feature point detection model to perform feature point detection, thereby obtaining multiple feature points of the space target in the ISAR image.

[0164] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0165] Based on various different spatial target primitive structures and spatial targets, multiple primitive structure ISAR simulation images and spatial target ISAR simulation images are obtained by using the physical optics method. Corresponding ISAR primitive image datasets and spatial target ISAR datasets are then constructed.

[0166] The basic feature point detection model is trained using the ISAR primitive image dataset to obtain the trained basic feature point detection model;

[0167] After performing multiple affine transformations on each space target ISAR simulation image in the space target ISAR dataset, the trained basic feature point detection model is used to detect basic feature points on the images after multiple affine transformations, and the pseudo-true value label of the corresponding space target ISAR simulation image is obtained based on multiple detection results.

[0168] The feature point detection model is trained using the aforementioned space target ISAR dataset and the corresponding pseudo-ground value labels, resulting in the trained feature points detected by the model.

[0169] An ISAR image of a space target is acquired, and the ISAR image is input into the trained feature point detection model to perform feature point detection, thereby obtaining multiple feature points of the space target in the ISAR image.

[0170] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0171] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0172] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A feature point detection method for ISAR images of space targets, characterized in that, The method includes: Based on various different spatial target primitive structures and spatial targets, multiple primitive structure ISAR simulation images and spatial target ISAR simulation images are obtained by using the physical optics method. Corresponding ISAR primitive image datasets and spatial target ISAR datasets are then constructed. The basic feature point detection model is trained using the ISAR primitive image dataset to obtain the trained basic feature point detection model; After performing multiple affine transformations on each space target ISAR simulation image in the space target ISAR dataset, the trained basic feature point detection model is used to detect basic feature points on the images after multiple affine transformations, and the pseudo-true value label of the corresponding space target ISAR simulation image is obtained based on multiple detection results. The feature point detection model is trained using the aforementioned space target ISAR dataset and the corresponding pseudo-ground value labels, resulting in the trained feature points detected by the model. An ISAR image of a space target is acquired, and the ISAR image is input into the trained feature point detection model to perform feature point detection, thereby obtaining multiple feature points of the space target in the ISAR image.

2. The feature point detection method for ISAR images of space targets according to claim 1, characterized in that, When using the physical optics method to simulate the primitive structure of a space target: The spatial target primitive structure includes cylindrical structure, cubic structure and planar structure, and the spatial target primitive structure includes multiple scales; The true values ​​of feature point locations are labeled for each of the simulated ISAR images of the primitive structure obtained from the simulation.

3. The feature point detection method for ISAR images of space targets according to claim 2, characterized in that, The basic feature point detection model is a fully convolutional neural network. The convolutional kernel size in the basic feature point detection model is 3×3. Batch normalization is used for data normalization. The non-linear activation function is the ReLU function. The fully connected convolutional layer is a 1×1 convolution.

4. The feature point detection method for ISAR images of space targets according to claim 3, characterized in that, Multiple affine transformations, including translation, rotation, and scaling operations, are performed on each space target ISAR simulation image in the aforementioned space target ISAR dataset.

5. The feature point detection method for ISAR images of space targets according to claim 4, characterized in that, The trained basic feature point detection model is used to detect basic feature points in images after multiple affine transformations, and the pseudo-ground value labels of the corresponding space target ISAR simulation images are obtained based on multiple detection results, including: The pseudo-true value label of a space target ISAR simulation image is obtained by aggregating the basic feature point detection results of the images after performing different affine transformations on the simulation image.

6. The feature point detection method for ISAR images of space targets according to any one of claims 1-5, characterized in that, The feature point detection model includes a shared encoder and two independent decoders, wherein the two decoders are a feature point detection decoder for detecting feature points in an image, and a feature point descriptor decoder for calculating feature point descriptors. The shared encoder includes a convolutional layer, a downsampling pooling layer, and a nonlinear activation function connected in sequence. The feature point detection decoder adopts an explicit decoding structure and outputs a feature point probability image. The feature point descriptor decoder uses a UCN network structure to generate sparse descriptors, and then performs bicubic interpolation and L2 normalization on the sparse descriptors to obtain unit pixel descriptors of uniform length.

7. The feature point detection method for ISAR images of space targets according to any one of claims 1-5, characterized in that, When training the feature point detection model, the loss function used is the total loss function after balancing the loss of the feature point detection decoder and the loss of the feature point descriptor decoder through weight coefficients.

8. A feature point detection device for ISAR images of space targets, characterized in that, The device includes: The ISAR dataset construction module is used to simulate multiple primitive structure ISAR images and space target ISAR images based on various different space target primitive structures and space targets using the physical optics method, and to construct the corresponding ISAR primitive image dataset and space target ISAR dataset respectively. The basic feature point detection model training module is used to train the basic feature point detection model using the ISAR primitive image dataset to obtain the trained basic feature point detection model. The pseudo-true value labeling module is used to perform multiple affine transformations on each space target ISAR simulation image in the space target ISAR dataset, and then use the trained basic feature point detection model to detect basic feature points on the images after multiple affine transformations, and obtain the pseudo-true value label of the corresponding space target ISAR simulation image based on multiple detection results. The feature point detection model training module is used to train the feature point detection model using the space target ISAR dataset and the corresponding pseudo-ground value labels, so as to obtain the trained feature points detected by the model. The feature point detection module is used to acquire ISAR images of space targets, input the ISAR images into the trained feature point detection model to perform feature point detection, and obtain multiple feature points of space targets in the ISAR images.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images