Feature point detection method, device and equipment oriented to space target ISAR (Inverse Synthetic Aperture Radar) image

Abstract

The invention relates to a space target ISAR image-oriented feature point detection method, device and equipment, and the method comprises the steps: carrying out the simulation construction of a primitive image data set and a space target data set through employing a physical optical method, and training a basic feature point detection model through employing the primitive image data set, after multiple affine transformation is carried out on each space target simulation image in the space target data set, basic feature point detection is carried out on the images subjected to multiple affine transformation by utilizing the trained basic feature point detection model, and pseudo-true value labels corresponding to the space target simulation images are obtained; according to the method, a space target data set is obtained, a feature point detection model is trained by using the space target data set and a corresponding pseudo-truth value label, and finally, the trained feature point detection model is directly adopted to detect an ISAR image, so that the feature points of the space target can be obtained. And the precision of a space target detection result is improved.

Description

Technical Field

[0001] The present application relates to the technical field of ISAR image target recognition, and in particular to a method, device and equipment for detecting feature points of ISAR images of space targets. Background Art

[0002] Inverse synthetic aperture radar can work all day and all weather, and has been widely used in the field of space target monitoring. By observing space targets, inverse synthetic aperture radar can obtain high-resolution ISAR images of targets. In the ISAR imaging process, increasing the transmission signal bandwidth can improve the radial resolution of ISAR images, and accumulating larger angles through long-term observation can improve the lateral resolution of ISAR images. ISAR images are two-dimensional projections of targets on the radar imaging plane, reflecting the contour shape and scattering characteristics of the target. Therefore, detecting feature points from ISAR images is a very important task, which can provide technical support for subsequent space target attitude estimation and three-dimensional reconstruction.

[0003] Traditional feature point detection methods perform poorly when faced with low texture and uneven brightness characteristics of ISAR images. Due to its nonlinear mapping ability, neural networks can effectively learn the features of space target ISAR images, thereby improving the performance of feature point detection.

[0004] However, the existing annotated ISAR image data of space targets is relatively scarce, which brings challenges to the detection of target feature points using neural networks. Summary of the invention

[0005] Based on this, it is necessary to provide a feature point detection method, device and equipment for space target ISAR images that can solve the problem of relative scarcity of annotated space target ISAR image data in response to the above technical problems.

[0006] A feature point detection method for space target ISAR images, the method comprising:

[0007] Based on a variety of different space target primitive structures and space targets, the physical optics method is used to simulate and obtain multiple primitive structure ISAR simulation images and space target ISAR simulation images, and the corresponding ISAR primitive image datasets and space target ISAR datasets are constructed respectively;

[0008] Using the ISAR primitive image data set to train a basic feature point detection model to obtain a trained basic feature point detection model;

[0009] After performing multiple affine transformations on each space target ISAR simulation image in the space target ISAR data set, basic feature point detection is performed on the image after the multiple affine transformations using the trained basic feature point detection model, and pseudo-true value labels of the corresponding space target ISAR simulation images are obtained according to the multiple detection results;

[0010] Using the space target ISAR data set and the corresponding pseudo-true value labels to train a feature point detection model to obtain a trained feature point detection model;

[0011] An ISAR image of a space target is acquired, and the ISAR image is input into the trained feature point detection model to perform feature point detection, so as to obtain a plurality of feature points of the space target in the ISAR image.

[0012] In one embodiment, when simulating based on a space target primitive structure using a physical optics method:

[0013] The space target primitive structure includes a cylindrical structure, a cubic structure and a plane structure, and the space target primitive structure includes multiple scales;

[0014] And the true value of the feature point position is marked on each of the primitive structure ISAR simulation images obtained by simulation.

[0015] In one embodiment, the basic feature point detection model is a fully convolutional neural network, the convolution kernel size in the basic feature point detection model is 3×3, batch normalization is used for data normalization, the nonlinear activation function uses the ReLU function, and the fully connected convolution layer is 1×1 convolution.

[0016] In one embodiment, multiple affine transformations are performed on each space target ISAR simulation image in the space target ISAR data set, including translation, rotation, and scaling operations.

[0017] In one embodiment, the trained basic feature point detection model is used to perform basic feature point detection on an image after multiple affine transformations, and a pseudo-true value label of an ISAR simulation image of a corresponding space target is obtained according to multiple detection results, including:

[0018] The basic feature point detection results of an ISAR simulation image of a space target after different affine transformations are aggregated to obtain the pseudo-true value label of the ISAR simulation image of the space target.

[0019] In one embodiment, the feature point detection model includes a shared encoder and two independent decoders, wherein the two decoders are a feature point detection decoder for detecting feature points in an image, and a feature point descriptor decoder for calculating feature point descriptors;

[0020] The shared encoder includes a convolutional layer, a downsampling pooling layer and a nonlinear activation function connected in sequence;

[0021] The feature point detection decoder adopts a display decoding structure, and the output is a feature point probability image;

[0022] The feature point descriptor decoder uses a UCN network structure to generate a sparse descriptor, and then performs bicubic interpolation and L2 standardization on the sparse descriptor to obtain a unit pixel descriptor of uniform length.

[0023] In one embodiment, when training the feature point detection model, the loss function used is a total loss function that balances the feature point detection decoder loss and the feature point descriptor decoder loss through a weight coefficient.

[0024] The present application also provides a feature point detection device for space target ISAR images, the device comprising:

[0025] ISAR dataset construction module, which is used to obtain multiple primitive structure ISAR simulation images and space target ISAR simulation images based on a variety of different space target primitive structures and space targets by using physical optics method, and to construct corresponding ISAR primitive image datasets and space target ISAR datasets respectively;

[0026] A basic feature point detection model training module is used to train the basic feature point detection model using the ISAR primitive image data set to obtain a trained basic feature point detection model;

[0027] A pseudo-truth value label obtaining module is used to perform multiple affine transformations on each space target ISAR simulation image in the space target ISAR data set, perform basic feature point detection on the image after the multiple affine transformations using the trained basic feature point detection model, and obtain the pseudo-truth value label of the corresponding space target ISAR simulation image according to the multiple detection results;

[0028] A feature point detection model training module is used to train the feature point detection model using the space target ISAR data set and the corresponding pseudo-true value labels to obtain a trained feature point detection model;

[0029] The feature point detection module is used to obtain an ISAR image of a space target, input the ISAR image into the trained feature point detection model to perform feature point detection, and obtain multiple feature points of the space target in the ISAR image.

[0030] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0031] Based on a variety of different space target primitive structures and space targets, the physical optics method is used to simulate and obtain multiple primitive structure ISAR simulation images and space target ISAR simulation images, and the corresponding ISAR primitive image datasets and space target ISAR datasets are constructed respectively;

[0032] Using the ISAR primitive image data set to train a basic feature point detection model to obtain a trained basic feature point detection model;

[0033] After performing multiple affine transformations on each space target ISAR simulation image in the space target ISAR data set, basic feature point detection is performed on the image after the multiple affine transformations using the trained basic feature point detection model, and pseudo-true value labels of the corresponding space target ISAR simulation images are obtained according to the multiple detection results;

[0034] Using the space target ISAR data set and the corresponding pseudo-true value labels to train a feature point detection model to obtain a trained feature point detection model;

[0035] An ISAR image of a space target is acquired, and the ISAR image is input into the trained feature point detection model to perform feature point detection, so as to obtain a plurality of feature points of the space target in the ISAR image.

[0036] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0037] Based on a variety of different space target primitive structures and space targets, the physical optics method is used to simulate and obtain multiple primitive structure ISAR simulation images and space target ISAR simulation images, and the corresponding ISAR primitive image datasets and space target ISAR datasets are constructed respectively;

[0038] Using the ISAR primitive image data set to train a basic feature point detection model to obtain a trained basic feature point detection model;

[0039] After performing multiple affine transformations on each space target ISAR simulation image in the space target ISAR data set, basic feature point detection is performed on the image after the multiple affine transformations using the trained basic feature point detection model, and pseudo-true value labels of the corresponding space target ISAR simulation images are obtained according to the multiple detection results;

[0040] Using the space target ISAR data set and the corresponding pseudo-true value labels to train a feature point detection model to obtain a trained feature point detection model;

[0041] An ISAR image of a space target is acquired, and the ISAR image is input into the trained feature point detection model to perform feature point detection, so as to obtain a plurality of feature points of the space target in the ISAR image.

[0042] The above-mentioned feature point detection method, device and equipment for space target ISAR images use the physical optics method to simulate and construct an ISAR primitive image data set and a space target ISAR data set, first use the ISAR primitive image data set to train the basic feature point detection model to obtain the trained basic feature point detection model, then perform multiple affine transformations on each space target ISAR simulation image in the space target ISAR data set, use the trained basic feature point detection model to perform basic feature point detection on the image after multiple affine transformations to obtain the pseudo-true value label of the corresponding space target ISAR simulation image, and use the space target ISAR data set and the corresponding pseudo-true value label to train the feature point detection model, finally directly use the trained feature point detection model to detect the ISAR image to obtain the feature points of the space target, and use this method to overcome the problem of too few labeled images during neural network training, while improving the accuracy of ISAR space target detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 1 is a flow chart of a feature point detection method for ISAR images of space targets in one embodiment;

[0044] Figure 2 A schematic diagram of the decomposition of the motion of a space target in one embodiment;

[0045] Figure 3 A schematic diagram of an ISAR observation imaging model for a space target in an embodiment;

[0046] Figure 4 A schematic diagram of an ISAR image projection mechanism in one embodiment;

[0047] Figure 5 Schematic diagram of a framework of a feature point detection method for ISAR images of space targets in one embodiment;

[0048] Figure 6 is a schematic diagram of an ISAR primitive image in an embodiment, Figure 6 (a) is the ISAR image obtained by simulating a cube as the basic structure. Figure 6 (b) is the ISAR image obtained by simulation when the basic unit structure is a plane;

[0049] Figure 7 A schematic diagram of a basic feature point detection model structure in an embodiment;

[0050] Figure 8 A schematic diagram of a feature map channel in one embodiment;

[0051] Fig. 9 A schematic diagram of affine transformation in one embodiment;

[0052] Fig.10 It is a schematic diagram of affine transformation decomposition in one embodiment;

[0053] Fig.11 is a structural schematic diagram of a feature point detection model in an embodiment;

[0054] Fig.12 A schematic diagram of a shared encoder structure in one embodiment;

[0055] Fig.13 This is a schematic diagram of the feature point detection performance of the SIFT algorithm, SURF algorithm, KLT algorithm and SFPD model in an experiment. Fig.13 (a) is a schematic diagram of the number of feature point detections. Fig.13 (b) is a schematic diagram of feature point detection accuracy;

[0056] Fig.14 This is an example of the feature point detection results of the SIFT algorithm, SURF algorithm, KLT algorithm and SFPD model in an experiment, where: Fig.14 (a) Fig.14 (b) Fig.14 (c) and Fig.14 (d) are example graphs of feature point detection results using SIFT algorithm, SURF algorithm, KLT algorithm and SFPD model;

[0057] Fig.15 This is a schematic diagram of the feature point matching accuracy of the SIFT algorithm, SURF algorithm, KLT algorithm and SFPD model under different matching thresholds in an experiment, where: Fig.15 (a) Fig.15 (b) Fig.15 (c) Schematic diagram of feature point matching accuracy when the matching threshold is 1 pixel, the matching threshold is 3 pixels, and the matching threshold is 5 pixels;

[0058] Fig.16 This is a schematic diagram of the feature point matching recall rate of the SIFT algorithm, SURF algorithm, KLT algorithm and SFPD model under different matching thresholds in an experiment, where: Fig.16 (a) Fig.16 (b) Fig.16 (c) Schematic diagram of feature point matching recall rate when the matching threshold is 1 pixel, the matching threshold is 3 pixels, and the matching threshold is 5 pixels;

[0059] Fig.17 This is a schematic diagram of the F1 score of feature point matching of SIFT algorithm, SURF algorithm, KLT algorithm and SFPD model under different matching thresholds in an experiment, where: Fig.17 (a) Fig.17 (b) Fig.17 (c) Schematic diagram of the F1 score of feature point matching when the matching threshold is 1 pixel, the matching threshold is 3 pixels, and the matching threshold is 5 pixels;

[0060] Fig.18 This is an example of feature point matching between SIFT algorithm, SURF algorithm, KLT algorithm and SFPD model when the matching threshold is 3 pixels in an experiment, where: Fig.18 (a) Fig.18 (b) Fig.18 (c) and Fig.18 (d) Example images of feature point matching of SIFT algorithm, SURF algorithm, KLT algorithm and SFPD model respectively;

[0061] Fig.19 It is a structural block diagram of a feature point detection device for space target ISAR images in one embodiment;

[0062] Fig. 20 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] In the prior art, the nonlinear mapping capability of neural networks is used to extract features from ISAR images. By learning complex image feature representations, neural networks can effectively capture detailed information in ISAR images, thereby improving the accuracy of feature point detection. For example, a network for extracting key point features from ISAR images can detect sailboard corners and body endpoints in ISAR images, and the effectiveness of key point extraction is verified through pose estimation experiments. There is also a multi-scale semantic feature extraction network that can extract semantic features from ISAR images as key points, improving the reliability of ISAR feature extraction. Or an instance segmentation method based on deep learning for extracting and associating structural key points in ISAR images. However, there are still some limitations in the use of neural networks to extract ISAR image feature points. From the above content, it can be seen that current research work mainly focuses on extracting structural key points in ISAR images, but fails to cover all feature points in ISAR images. Therefore, these methods are not suitable for downstream tasks such as three-dimensional reconstruction. There are two main reasons for this phenomenon: first, there are relatively few publicly available ISAR datasets, which do not support extensive research and algorithm verification; second, due to the anisotropy of scatterers, it is difficult to accurately calibrate the true positions of feature points when making ISAR image datasets, which limits the applicability of supervised neural network methods in detecting ISAR image feature points.

[0065] In response to the above problems, Figure 1 As shown, a feature point detection method for space target ISAR images is provided, comprising the following steps:

[0066] Step S100, based on a plurality of different space target primitive structures and space targets, a physical optics method is used to perform simulation to obtain a plurality of primitive structure ISAR simulation images and space target ISAR simulation images, and corresponding ISAR primitive image datasets and space target ISAR datasets are constructed respectively.

[0067] Step S110, training a basic feature point detection model using the ISAR primitive image data set to obtain a trained basic feature point detection model.

[0068] Step S120, after performing multiple affine transformations on each space target ISAR simulation image in the space target ISAR data set, basic feature point detection is performed on the image after multiple affine transformations using the trained basic feature point detection model, and pseudo-true value labels of the corresponding space target ISAR simulation images are obtained based on multiple detection results.

[0069] Step S130, training a feature point detection model using the space target ISAR data set and corresponding pseudo-truth labels to obtain a trained feature point detection model.

[0070] Step S140, acquiring an ISAR image of the space target, inputting the ISAR image into the trained feature point detection model to perform feature point detection, and obtaining a plurality of feature points of the space target in the ISAR image.

[0071] In this application, a self-supervised feature point detection method for space target ISAR images is proposed. This method can accurately detect the feature point positions in space target ISAR images without providing the true value of the feature point position during the training process of the neural network. This method first simulates the ISAR primitive data set and uses the data set to train the proposed basic feature point detection model. Then, the basic feature point detection model is used to annotate the pseudo-true value of the space target ISAR image, and the affine transformation method is used to improve the accuracy of the annotation. Finally, the feature point detection model is trained on the space target ISAR image data set. At the same time, the experimental results show that this method can complete the training without the true value of the space target ISAR image data set, and shows good feature point detection performance. At the same time, the experimental results verify the effectiveness of the proposed method. Compared with the traditional feature point detection method, the method proposed in this article has higher feature point detection accuracy and better feature point matching performance.

[0072] In this embodiment, the method is as follows Figure 2 As shown in the figure, it specifically includes three parts. First, it is trained on the space target primitive dataset to obtain the basic feature point detection model. Then, the space target ISAR dataset is automatically labeled by combining affine transformation with the basic feature point detection model to generate pseudo-true values. Finally, the generated labels and the space target ISAR dataset are used to train the fully convolutional neural network proposed in this paper. The network can extract the feature point positions and their corresponding feature descriptors from the space target ISAR images.

[0073] In step S100, firstly, data sets needed for training two neural networks are constructed. Both data sets are constructed by using a physical optics method based on simulated ISAR images obtained by simulating a space target ISAR imaging model.

[0074] In this embodiment, the space target ISAR imaging model used in the physical optics method is constructed as follows:

[0075] In the inverse synthetic aperture radar (ISAR) imaging of space targets, the distance between the observed space target and the radar is relatively far, and the target size is very small relative to this distance, satisfying the far field condition. Therefore, the electromagnetic wave received by the target can be regarded as a plane wave. Under this assumption, the motion of the space target can be described as Figure 3The decomposition is shown in the figure, where the red track is the target motion track and the blue track is the decomposed motion track. Assuming that the space target moves from point A to point C, the motion can be decomposed into three parts. First, the target rotates around its geometric center, resulting in different Doppler effects at each scattering point on the target. Secondly, the translation motion from point A to point B. During this process, the distance change between each scattering point and the radar is consistent, so the Doppler effect produced is also consistent, which can be corrected by translation compensation. Finally, from point B to point C, the target rotates in an equal circle around the center of the radar. During this process, the distance between the target and the radar remains unchanged, and no Doppler effect is produced.

[0076] After completing the translation compensation, the ISAR observation imaging model is as follows: Figure 4 As shown in the figure, O-XYZ is the target body coordinate system, and O is the equivalent rotation center of the target. Figure 4 In the figure, the red line represents the radar line of sight (LOS), which is determined by the azimuth and elevation angles and can be expressed as:

[0077] r(t)=[-cosφ(t)cosθ(t),-cosφ(t)sinθ(t),-sinφ(t)] T (1)

[0078] In formula (1), θ(t) and φ(t) are the azimuth and elevation angles at time t. k is a scattering point on the target, and its position can be expressed as p k =(x k ,y k ,z k ) T Assume that the distance between the radar and the target equivalent rotation center is r 0 (t), then the feature point p k The distance to the radar can be expressed as:

[0079] r p (t) = r 0 (t)+r(t) T p k (2)

[0080] In formula (2), for r p (t) can be differentiated to obtain the instantaneous velocity of the feature point, which can be expressed as:

[0081]

[0082] In formula (3), Represents r p(t) is the derivative with respect to t, and the other symbols are the same. In the body coordinate system, the position and direction of the scattering point do not change with the rotation of the target, so formula (3) can be expressed as:

[0083]

[0084] In formula (4), × is the vector cross product symbol, ω radar (t) is the angular velocity of the radar relative to the target. radar (t) can be decomposed along the LOS into the angular velocity ω perpendicular to the LOS n (t) and the angular velocity ω parallel to LOS r (t). The parallel component of the angular velocity along the LOS will not affect the calculation results, so formula (4) can be expressed as:

[0085]

[0086] In formula (5), 2 Represents the two-norm of a vector.

[0087] Combined with the above analysis, the imaging plane of the ISAR image can be expressed as:

[0088]

[0089] In formula (6), ρ r and ρ a Represents the projection vector of the scattering point in the range and azimuth directions. The normal vector direction of the imaging plane is the same as ω n Therefore, the projection relationship of the scattering points on the ISAR image can be expressed as:

[0090]

[0091] r k with a k is the scattering point p k The coordinates in the range and azimuth directions of the ISAR image. Figure 5 A schematic diagram of the projection mechanism of ISAR images is given in FIG.

[0092] In this embodiment, it is worth noting that there is currently no large-scale space target ISAR image feature point labeling dataset. In order to train the basic feature point detection model (BasicPoint), an ISAR primitive image dataset is constructed. When constructing the ISAR primitive image dataset, it is simulated based on the space target primitive structure, wherein the space target primitive structure includes a cylindrical structure, a cubic structure, and a plane structure. At the same time, in order to ensure that the model trained by the ISAR primitive image dataset has good data generalization ability and can learn the feature representation of targets of different sizes, a multi-scale space target primitive structure is designed and introduced. In addition, the true value of the feature point position in the ISAR primitive image is annotated during the simulation process. Figure 6 An example of an ISAR primitive image dataset is given in .

[0093] In addition, when constructing a space target ISAR dataset, the process is the same as that of constructing an ISAR primitive image dataset, except that the space target primitive structure is replaced with space targets, such as satellites, spacecraft, etc.

[0094] Further, after obtaining the ISAR primitive image data set, in step S110, the basic feature point detection model (BasicPoint) is trained on the data set. BasicPoint can effectively detect the location of feature points in the ISAR primitive image. Its network structure is as follows: Figure 7 as shown in .

[0095] from Figure 7 It can be seen that the input of BasicPoint only accepts single-channel images. During the training process, the images in the ISAR primitive data set need to be converted into grayscale images before being input into BasicPoint. Traditional dense prediction networks usually adopt an encoder-decoder structure, in which the encoder uses convolution operations or pooling operations to reduce the data dimension, and then the decoder restores the data to the original dimension through upsampling operations. However, the upsampling operation will greatly increase the amount of calculation, which is not conducive to model training. Therefore, in this embodiment, BasicPoint uses a display decoder structure to restore the data dimension and reduce the amount of model calculation. First, BasicPoint uses a VGG structure encoder to encode the ISAR primitive image of size M×N into a feature map of M / 8×N / 8. Each point on the feature map contains 65 channels, and then the feature map is converted into a probability through the Softmax layer.

[0096] Specifically, the meaning of each channel in the feature map is as follows Figure 8As shown in the figure, the first 64 channels in the feature map correspond to the probability that each pixel in an 8×8 local pixel area in the input image may be a feature point, and the 65th channel is a discarded channel, which means that no feature point is detected in this local pixel area. In order to restore the data to the original dimension, BasicPoint discards the 65th channel in the feature map, and then converts the 64×M / 8×N / 8 feature map into an M×N feature map through the Reshape layer. The element at each position on the feature map corresponds to the probability that the position in the input image is a feature point. Finally, the ISAR primitive image is mapped to a feature point response image with the same resolution. The feature point response image is presented as a heat map. The detected feature points show obvious red features in the feature point response image, which is in sharp contrast with the surrounding points.

[0097] Specifically, the basic feature point detection model (BasicPoint) is a fully convolutional neural network. The convolution kernel size in the basic feature point detection model is 3×3, batch normalization is used for data normalization, the nonlinear activation function uses the ReLU function, and the fully connected convolution layer is 1×1 convolution.

[0098] In this embodiment, when training the basic feature point detection model, cross entropy is used as the loss function, which can be expressed as:

[0099]

[0100] In formula (8), p m,n is the probability distribution of true feature points, q m,n is the probability distribution of feature points detected by BasicPoint, and M and N are the sizes of the feature point probability distribution map.

[0101] In order to provide training data for the feature point detection model, the trained BasicPoint is used to annotate the pseudo-true value of the space target ISAR image dataset. Since BasicPoint is trained on the ISAR primitive image dataset, it is impossible to detect all the feature points in the space target ISAR image, which will lead to poor effect of the feature point detection model. To solve this problem, in step S120, an affine transformation is performed on each image in the space target ISAR dataset.

[0102] Affine transformation is a geometric transformation composed of linear transformation and translation, including translation, rotation, scaling and other operations. When an image is affine transformed, the original geometric relationship in the image will not be changed. After the space target ISAR image is affine transformed, the space target ISAR image is annotated using BasicPoint, so that BasicPoint can effectively detect the position of feature points under different receptive fields. Then, the cumulative summation of these annotation results is performed to obtain the pseudo-true value of the position of the feature point of the image. The schematic diagram of the affine transformation is shown in Figure 9.

[0103] Specifically, multiple affine transformations including translation, rotation, and scaling operations are performed on each space target ISAR simulation image in the space target ISAR data set.

[0104] Furthermore, using the trained basic feature point detection model to perform basic feature point detection on images after multiple affine transformations, and obtaining the pseudo-true value label of the corresponding space target ISAR simulation image according to multiple detection results includes: performing basic feature point detection results of images after different affine transformations on a space target ISAR simulation image, and then inversely transforming the feature point results detected on the affine image to the original image, so that multiple feature points under different receptive fields can be obtained through multiple affine images, and the feature points under these different receptive fields are aggregated. Then, the pseudo-true value label of the space target ISAR simulation image can be obtained.

[0105] Specifically, assuming that f() is the basic feature point detection function (i.e., the basic feature point detection network), I is the input image, and P is the detected feature point set, it can be expressed as:

[0106] P=f(I) (9)

[0107] The affine transformation of the input image can be expressed as:

[0108] AP=f(A(I)) (10)

[0109] In formula (10), A is the affine transformation matrix, A(I) is the ISAR image of the space target after affine transformation, and A is multiplied on both sides of formula (10) -1 Another representation of the feature point set P can be obtained:

[0110] P=A -1 f(A(I)) (11)

[0111] Therefore, after accumulating and summing a large number of affine transformation samples, a better feature point detection function can be obtained, which can be expressed as:

[0112]

[0113] In formula (12), N a is the number of affine transformations, A i is the affine transformation matrix of the i-th order.

[0114] It is worth noting that not all 3×3 matrices are suitable as affine matrices. In order to select a suitable affine matrix, the affine transformation is decomposed, and simple transformations such as rotation, translation, and scaling are performed within a predetermined range, and these simple transformations are combined with the initial cropping center. The specific steps are as follows: Fig.10 As shown in the above content, we can know that the number of affine transformations will affect the performance of feature point detection, so N a is the hyperparameter of the method proposed in this paper. N is determined by corresponding experiments. a The best value of N a The best value for is 1000.

[0115] In step S130, the structure of the feature point detection model (SFPD) is as follows Fig.11 As shown in the figure, it includes a shared encoder and two independent decoders, wherein the two decoders are adapted for different downstream tasks respectively, one is a feature point detection decoder for detecting feature points in an image, and the other is a feature point descriptor decoder for calculating feature point descriptors. The two decoders are adapted for different downstream tasks respectively. Traditional feature point detection algorithms usually adopt a pipeline process, that is, feature point detection is performed first, and then the corresponding descriptors are generated. However, this algorithm has the problem of insufficient coupling between tasks during the processing. In contrast, the SFPD proposed in this paper does not have this problem. The design of the network allows most network parameters to be shared between two different decoders, and feature point detection and descriptor calculation can be achieved simultaneously during the forward pass.

[0116] In this embodiment, in the feature point detection model, an encoder with a VGG structure is used to reduce the data dimension. Fig.12 The shared encoder structure of SFPD is given in , which consists of convolutional layers, downsampling pooling layers, and nonlinear activation functions. Specifically, the shared encoder uses a convolution kernel size of 3×3, and three 2×2 downsampling pooling layers are used to reduce the input data size from H×W to H / 8×W / 8. In summary, the shared encoder The input image is mapped into a tensor with smaller dimensions and greater channel depth

[0117]

[0118] The output data of the feature point detection decoder represents the probability that each pixel in the image is a feature point. The typical network structure usually reduces the data dimension at the encoder end and then restores the data dimension through the decoder. However, the upsampling operation in the traditional decoder will cause a significant increase in the amount of calculation, affecting the network convergence speed. Therefore, in this embodiment, the feature point detection decoder adopts the same display decoding structure as BasicPoint. In this structure, the input tensor of the feature point detection decoder is The 65 channels in are the same as those in BasicPoint and will not be described in detail here. Next, the last channel is discarded through the softmax layer and the H / 8×W / 8×64 tensor is reshaped into an H×W feature point probability image.

[0119] In this embodiment, the feature point descriptor decoder uses a UCN network structure to generate a sparse descriptor, and then performs bicubic interpolation and L2 normalization on the sparse descriptor to obtain a unit pixel descriptor of uniform length.

[0120] Specifically, the input of the feature point descriptor decoder is a tensor The output is a descriptor of size H×W×D. In order to reduce the amount of computation, the feature point descriptor decoder first uses the UCN

[44] network structure to generate a sparse descriptor for every 8 pixels, and then performs bicubic interpolation and L2 normalization to obtain a unit pixel descriptor of uniform length.

[0121] In this embodiment, when training the feature point detection model, the loss function used is to multiply the feature point detection decoder loss by the weight coefficient. And the feature point descriptor decoder loss The total loss function after balancing the two parts is expressed as:

[0122]

[0123] Specifically, It can be expressed as:

[0124]

[0125] In formula (14), is the ISAR image of the space target, for The image obtained after affine transformation, Y and Y′ are the pseudo-true value labels of the feature points of the two images. s () is the cross entropy loss, which can be expressed as:

[0126]

[0127] In formula (15), yh,w ∈Y.

[0128] Specifically, It can be expressed as:

[0129]

[0130] In formula (16), and is the descriptor of the input image pair,

[0131] S is the correspondence between image pairs. d () is the hinge loss, which can be expressed as:

[0132]

[0133] In formula (17), the weight λ d Used to balance positive and negative samples, m p With m n is the key loss of the positive and negative boundaries of the network. s is the correspondence between descriptors, which is used to determine whether d and d' are associated. s can be expressed as:

[0134]

[0135] In formula (18), p h,w are the center coordinates of the pixel grid corresponding to the point (h, w). If the Euclidean distance between corresponding points after affine transformation is less than 6 pixels, the two points are considered to be associated.

[0136] Finally, in step S140, that is, in the application process of the trained feature point detection model, it is only necessary to input the measured ISAR image into the model to obtain the feature points of the space target.

[0137] In this paper, experimental results are also given to prove the effectiveness of this method, that is, the effectiveness of the feature point detection model.

[0138] Specifically, this method is compared with the commonly used feature point detection algorithm SURF (Speeded Up Robust Features) in the experiment. In addition, it is also compared with the improved SIFT (Scale Invariant Feature Transform) algorithm and KLT (Kanade-Lucas-Tomasi) algorithm. The test data is 100 randomly selected ISAR images of space targets in the test set. The experiment selects the number of correctly detected feature points and the accuracy of feature point detection as evaluation indicators. Fig.13Specific experimental results are given. It can be seen from the experimental results that the method proposed in this paper is superior to the other three methods in terms of the number of correctly detected feature points and the accuracy of feature point detection, which fully proves the effectiveness of the proposed method. At the same time, observing the experimental results of the other three methods, it can be found that the number of correctly detected feature points and the accuracy of feature point detection of the SIFT algorithm and the SURF algorithm are both low. The main reason for this phenomenon is that there are large differences in the imaging mechanism between optical images and ISAR images. Therefore, there are also large differences in the image features of the two, which leads to false detection and missed detection in the feature point detection process. Relatively speaking, although the number of correctly detected feature points of the KLT algorithm is small, its detection accuracy exceeds that of the SIFT algorithm and the SURF algorithm. This shows that the feature point detection performance of the KLT algorithm is better than that of the SIFT algorithm and the SURF algorithm, because in the downstream tasks of feature point detection, false detection has a greater impact than missed detection. The statistical results of the feature point detection performance of the above four algorithms are shown in Table 1. Compared with the other three methods, the number of correctly detected feature points of the SFPD model has increased by more than 96.8%, and the accuracy of feature point detection has increased by more than 47.48%. In addition, in Fig.14 The feature point detection examples of SIFT algorithm, SURF algorithm, KLT algorithm and SFPD model are shown in the figure.

[0139] Table 1 1 Statistical results of feature point detection performance of SIFT algorithm, SURF algorithm, KLT algorithm and SFPD model

[0140]

[0141] Furthermore, considering that in the space target monitoring task, the purpose of feature point detection is often to provide a prerequisite for downstream tasks such as feature point matching and three-dimensional reconstruction. The method proposed in this paper can not only detect the feature point positions in the ISAR images of space targets, but also output the feature descriptors corresponding to the feature point positions, which is convenient for subsequent feature point matching. In order to verify the feature description ability of the method proposed in this paper, 50 images were randomly selected from the test set, and affine transformation was randomly performed on each image to obtain 50 groups of images to be matched. The proposed method and the above three algorithms were used to match the feature points of these 50 pairs of images. The experiment uses precision, recall and F1 score as evaluation indicators. In addition, in order to judge whether the feature point matching is correct, the following evaluation criteria are also designed:

[0142]

[0143] In the above formula, p is the feature point detected in the original image, p' is the feature point in the image after affine transformation, and A is the affine transformation matrix between the two images. If the Euclidean distance between the feature point in the original image and its matching point after affine transformation is less than the matching threshold, it is considered to be a correct match. Feature point matching experiments were conducted under the conditions of matching thresholds of 1 pixel, 3 pixels, and 5 pixels. The experimental results are shown in Figure 2. Figures 15 to 17 shown.

[0144] According to the experimental results, the proposed method shows good feature point matching effect under different matching thresholds. The experimental results show that the smaller the matching threshold, the more significant the performance improvement of the proposed method compared with the other three methods. This shows that the feature descriptor generated by the proposed method can accurately describe the image features of the feature points, making the feature point matching more accurate. When the matching threshold increases, the algorithm's requirements for matching accuracy are relaxed, and the performance improvement of the proposed method is relatively reduced. Observation Figures 15 to 17 It is not difficult to find from the experimental results that the curves of the SIFT algorithm and the SURF algorithm are relatively stable, while the performance of the KLT algorithm and the method proposed in this paper fluctuates greatly, especially the KLT algorithm, whose F1 score peak value of the matching result can reach about 0.9, but the valley value is less than 0.2. The main reason for this phenomenon is that the random affine transformation causes a large difference in perspective between some matching image pairs. When the perspective difference is large, the feature point features in the image will change, affecting the performance of the feature point matching algorithm. Table 2 shows the performance statistics of the feature point matching experiment. It can be seen from the table that even when the matching threshold is 5 pixels, the F1 score of the method proposed in this paper is improved by more than 15% compared with the other three algorithms, which once again verifies the feature point matching performance of the algorithm proposed in this paper. In order to more intuitively demonstrate the feature point matching effect, Fig.18 An example of feature point matching of SIFT algorithm, SURF algorithm, KLT algorithm and SFPD model is given when the matching threshold is 3 pixels.

[0145] Table 2 Statistical results of feature point matching performance of SIFT algorithm, SURF algorithm, KLT algorithm and SFPD model under different matching thresholds

[0146]

[0147] In the above-mentioned feature point detection method for space target ISAR images, an ISAR primitive dataset is first constructed, and a basic feature point detection model is trained on this basis. The model can be used to annotate the pseudo-true values ​​of the feature point positions of the space target ISAR image dataset. In order to improve the accuracy of pseudo-true value annotation, this method introduces affine transformation into the pseudo-true value annotation process, detects feature points in the image after affine transformation, and inversely transforms the feature points to the original image. The feature point positions are aggregated through multiple transformations, thereby improving the accuracy of feature point position annotation. Subsequently, the feature point detection model training is completed on the annotated space target ISAR image dataset. This method first simulates the space target primitive model, generates an ISAR primitive dataset, and trains the dataset to obtain a basic feature point detection model. At the same time, the basic feature point detection model is combined with the affine transformation to annotate the pseudo-true values ​​of the space target ISAR image, thereby improving the confidence of the pseudo-true value annotation. This method also proposes a fully convolutional neural network named SFPD, and trains it on the annotated space target ISAR image dataset, which can output the positions of feature points in the ISAR image and their corresponding feature descriptors. Through simulation experiments, the comparison between this method and existing methods shows that this method has higher feature point detection accuracy, and the feature point detection results are also suitable for subsequent tasks such as feature point matching.

[0148] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0149] In one embodiment, Fig.19 As shown, a feature point detection device for space target ISAR images is provided, comprising: an ISAR data set construction module 200, a basic feature point detection model training module 210, a pseudo-true value label acquisition module 220, a feature point detection model training module 230 and a feature point detection module 240, wherein:

[0150] An ISAR data set construction module 200 is used to obtain a plurality of primitive structure ISAR simulation images and space target ISAR simulation images based on a plurality of different space target primitive structures and space targets by using a physical optics method, and to respectively construct a corresponding ISAR primitive image data set and a space target ISAR data set;

[0151] A basic feature point detection model training module 210 is used to train a basic feature point detection model using the ISAR primitive image data set to obtain a trained basic feature point detection model;

[0152] A pseudo-truth label obtaining module 220 is used to perform multiple affine transformations on each space target ISAR simulation image in the space target ISAR data set, perform basic feature point detection on the image after the multiple affine transformations using the trained basic feature point detection model, and obtain a pseudo-truth label of the corresponding space target ISAR simulation image according to multiple detection results;

[0153] The feature point detection model training module 230 is used to train the feature point detection model using the space target ISAR data set and the corresponding pseudo-true value labels to obtain a trained feature point detection model;

[0154] The feature point detection module 240 is used to obtain an ISAR image of a space target, input the ISAR image into the trained feature point detection model to perform feature point detection, and obtain multiple feature points of the space target in the ISAR image.

[0155] The specific definition of the feature point detection device for space target ISAR images can be found in the definition of the feature point detection method for space target ISAR images mentioned above, which will not be repeated here. Each module in the above-mentioned feature point detection device for space target ISAR images can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0156] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig. 20As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a feature point detection method for ISAR images of space targets is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse, etc.

[0157] Those skilled in the art will understand that Fig. 20 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0158] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0159] Based on a variety of different space target primitive structures and space targets, the physical optics method is used to simulate and obtain multiple primitive structure ISAR simulation images and space target ISAR simulation images, and the corresponding ISAR primitive image datasets and space target ISAR datasets are constructed respectively;

[0160] Using the ISAR primitive image data set to train a basic feature point detection model to obtain a trained basic feature point detection model;

[0161] After performing multiple affine transformations on each space target ISAR simulation image in the space target ISAR data set, basic feature point detection is performed on the image after the multiple affine transformations using the trained basic feature point detection model, and pseudo-true value labels of the corresponding space target ISAR simulation images are obtained according to the multiple detection results;

[0162] Using the space target ISAR data set and the corresponding pseudo-true value labels to train a feature point detection model to obtain a trained feature point detection model;

[0163] An ISAR image of a space target is acquired, and the ISAR image is input into the trained feature point detection model to perform feature point detection, so as to obtain a plurality of feature points of the space target in the ISAR image.

[0164] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0165] Based on a variety of different space target primitive structures and space targets, the physical optics method is used to simulate and obtain multiple primitive structure ISAR simulation images and space target ISAR simulation images, and the corresponding ISAR primitive image datasets and space target ISAR datasets are constructed respectively;

[0166] Using the ISAR primitive image data set to train a basic feature point detection model to obtain a trained basic feature point detection model;

[0167] After performing multiple affine transformations on each space target ISAR simulation image in the space target ISAR data set, basic feature point detection is performed on the image after the multiple affine transformations using the trained basic feature point detection model, and pseudo-true value labels of the corresponding space target ISAR simulation images are obtained according to the multiple detection results;

[0168] Using the space target ISAR data set and the corresponding pseudo-true value labels to train a feature point detection model to obtain a trained feature point detection model;

[0169] An ISAR image of a space target is acquired, and the ISAR image is input into the trained feature point detection model to perform feature point detection, so as to obtain a plurality of feature points of the space target in the ISAR image.

[0170] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0171] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0172] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A feature point detection method for ISAR images of space targets, characterized in that: The method comprises: Based on a variety of different space target primitive structures and space targets, the physical optics method is used to simulate and obtain multiple primitive structure ISAR simulation images and space target ISAR simulation images, and the corresponding ISAR primitive image datasets and space target ISAR datasets are constructed respectively; Using the ISAR primitive image data set to train a basic feature point detection model to obtain a trained basic feature point detection model; After performing multiple affine transformations on each space target ISAR simulation image in the space target ISAR data set, basic feature point detection is performed on the image after the multiple affine transformations using the trained basic feature point detection model, and pseudo-true value labels of the corresponding space target ISAR simulation images are obtained according to the multiple detection results; Using the space target ISAR data set and the corresponding pseudo-true value labels to train a feature point detection model to obtain a trained feature point detection model; An ISAR image of a space target is acquired, and the ISAR image is input into the trained feature point detection model to perform feature point detection, so as to obtain a plurality of feature points of the space target in the ISAR image.

2. The feature point detection method for space target ISAR images according to claim 1, characterized in that: When simulating based on the primitive structure of a space target using the physical optics method: The space target primitive structure includes a cylindrical structure, a cubic structure and a plane structure, and the space target primitive structure includes multiple scales; And the true value of the feature point position is marked on each of the primitive structure ISAR simulation images obtained by simulation.

3. The feature point detection method for space target ISAR images according to claim 2, characterized in that: The basic feature point detection model is a fully convolutional neural network. The convolution kernel size in the basic feature point detection model is 3×3, batch normalization is used for data normalization, the nonlinear activation function uses the ReLU function, and the fully connected convolution layer is 1×1 convolution.

4. The feature point detection method for space target ISAR images according to claim 3, characterized in that: Multiple affine transformations including translation, rotation and scaling operations are performed on each space target ISAR simulation image in the space target ISAR data set.

5. The feature point detection method for space target ISAR images according to claim 4, characterized in that: Using the trained basic feature point detection model to perform basic feature point detection on the image after multiple affine transformations, and obtaining the pseudo-true value label of the corresponding space target ISAR simulation image according to the multiple detection results includes: The basic feature point detection results of an ISAR simulation image of a space target after different affine transformations are aggregated to obtain the pseudo-true value label of the ISAR simulation image of the space target.

6. The feature point detection method for space target ISAR images according to any one of claims 1 to 5, characterized in that: The feature point detection model includes a shared encoder and two independent decoders, wherein the two decoders are a feature point detection decoder for detecting feature points in an image and a feature point descriptor decoder for calculating feature point descriptors; The shared encoder includes a convolutional layer, a downsampling pooling layer and a nonlinear activation function connected in sequence; The feature point detection decoder adopts a display decoding structure, and the output is a feature point probability image; The feature point descriptor decoder uses a UCN network structure to generate a sparse descriptor, and then performs bicubic interpolation and L2 standardization on the sparse descriptor to obtain a unit pixel descriptor of uniform length.

7. The feature point detection method for space target ISAR images according to any one of claims 1 to 5, characterized in that: When training the feature point detection model, the adopted loss function is a total loss function after balancing the feature point detection decoder loss and the feature point descriptor decoder loss through a weight coefficient.

8. A feature point detection device for ISAR images of space targets, characterized in that: The device comprises: ISAR dataset construction module, which is used to obtain multiple primitive structure ISAR simulation images and space target ISAR simulation images based on a variety of different space target primitive structures and space targets by using physical optics method, and to construct corresponding ISAR primitive image datasets and space target ISAR datasets respectively; A basic feature point detection model training module is used to train the basic feature point detection model using the ISAR primitive image data set to obtain a trained basic feature point detection model; A pseudo-truth value label obtaining module is used to perform multiple affine transformations on each space target ISAR simulation image in the space target ISAR data set, perform basic feature point detection on the image after the multiple affine transformations using the trained basic feature point detection model, and obtain the pseudo-truth value label of the corresponding space target ISAR simulation image according to the multiple detection results; A feature point detection model training module is used to train the feature point detection model using the space target ISAR data set and the corresponding pseudo-true value labels to obtain a trained feature point detection model; The feature point detection module is used to obtain an ISAR image of a space target, input the ISAR image into the trained feature point detection model to perform feature point detection, and obtain multiple feature points of the space target in the ISAR image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images