A method for segmenting on-orbit spacecraft components based on ISAR images

Abstract

This invention discloses an on-orbit spacecraft component segmentation method based on ISAR images, comprising: constructing a spacecraft component segmentation network model, wherein the segmentation network model includes an image preprocessing module, an image feature extraction module, and a feature fusion module; the image preprocessing module is used to perform morphological preprocessing on the image input to the segmentation network model; the image feature extraction module is used to extract target edge features, target keypoint features, and target semantic features from the input image respectively; the feature fusion module is used to fuse the target edge features, target keypoint features, and target semantic features, and output the component segmentation result; constructing a training sample set; training the segmentation network model using the training sample set; and inputting the original image to be identified into the trained segmentation network model to obtain the spacecraft component segmentation result. This invention can further classify spacecraft solar panels, providing a foundation for tasks such as spacecraft behavior recognition.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of space target recognition, and particularly relates to a method for segmenting in-orbit spacecraft components based on an ISAR image. BACKGROUND

[0002] Thanks to the development of high-resolution imaging technology of space targets based on optical or radar sensors, spacecraft information interpretation based on high-resolution images has become an important technical means for in-orbit spacecraft behavior analysis. ISAR (Inverse Synthetic Aperture Radar) imaging has the characteristics of high resolution, good penetration, strong anti-interference ability, etc., and can make up for the shortcomings of visible light and infrared imaging technology, solve the imaging problem of moving targets, and obtain non-cooperative target images at a long distance. In the field of spacecraft information interpretation, the research and judgment technology of spacecraft target structure, size and attitude based on ISAR imaging has gradually become a research hotspot and difficulty. Component segmentation refers to the research and analysis of spacecraft structure to distinguish important components such as spacecraft main body and sailboard, which can be used to extract the shape and structural features of the target, so as to research and judge the in-orbit working state of the spacecraft, and has important application value.

[0003] Most of the existing spacecraft image segmentation methods are based on optical images, or component detection and attitude estimation based on ISAR images, and there is less research on ISAR image component segmentation. Lin Caiyong et al. in the patent with application number CN110910413A and title "ISAR image segmentation method based on U-Net" adopts an encoder-decoder structure, respectively uses a shallow network to solve the pixel positioning problem and a deep network to solve the pixel classification problem, eliminates the influence of coherent spots, sidelobes and other factors in ISAR images, and realizes the target segmentation of ISAR images. Du Lan et al. in the literature "ISAR image semantic segmentation method based on GAN[J]. Radar Science and Technology, 2021, 19(5): 1672-2337." proposed an ISAR image semantic segmentation method based on generative adversarial network, which adopts the idea of adversarial learning to learn the mapping relationship from ISAR image distribution to semantic segmentation image distribution, and ensures the accuracy of spacecraft main body and sailboard segmentation by constructing local information and global information of segmentation image. Wu Yuanyuan et al. in the literature "Space target image segmentation method based on watershed algorithm[J]. Computer Simulation, 2011, 28(2): 1006-9348." proposed to reconstruct the morphological gradient image of the original optical image by using morphological opening and closing reconstruction operation, then label the target in the original image by using the difference in fractal characteristics between the target and the space background, and extract the region containing the label after segmentation of the reconstructed image by using the watershed algorithm. Wang Jingjing et al. in the literature "Complementary enhanced space moving target high precision detection and segmentation[J]. Journal of Harbin Institute of Technology, 2016, 48(3)." proposed a complementary enhanced algorithm based on saliency calculation and optical flow detection, which uses saliency region detection and optical flow detection to determine reasonable target candidate region, uses conjunction operation to confirm the target, and uses morphological filtering to realize high precision detection and segmentation of space moving target. Li Linze et al. in the literature "Non-cooperative target feature detection and recognition based on deep learning[J]. Journal of Intelligent Systems, 2020, 15(6): 1154-1162." proposed to use the improved deep learning method Mask RCNN for mask detection and recognition of space non-cooperative target simulation.

[0004] However, the above spacecraft segmentation detection methods have some shortcomings, mainly in the following two aspects: first, the above spacecraft segmentation detection methods only segment and distinguish the spacecraft and the space background, or use semantic segmentation methods to distinguish the main body and the sailboard, and do not further detect the left and right wing properties of the sailboard. In spacecraft information interpretation, it is necessary to realize the segmentation of important components of spacecraft to judge the target structure property and morphology. Second, most of the above segmentation detection methods are based on optical simulation images, and do not consider the problem of difficult access to high-quality optical images under the influence of time and weather, which cannot realize all-weather data support and research. SUMMARY

[0005] In order to solve the above problems existing in the prior art, the present application provides an on-orbit spacecraft component segmentation method based on an ISAR image, so as to enrich the current segmentation technology of the ISAR image, realize high-precision instance segmentation of important components of the spacecraft, and achieve higher segmentation precision than the prior art. The technical problem to be solved by the present application is realized by the following technical scheme:

[0006] The present application provides an on-orbit spacecraft component segmentation method based on an ISAR image, comprising:

[0007] S1: constructing a segmentation network model of a spacecraft component, the segmentation network model comprising an image preprocessing module, an image feature extraction module and a feature fusion module, wherein the image preprocessing module is used for morphological preprocessing of an image input into the segmentation network model to obtain a preprocessed image; the image feature extraction module is used for extracting target edge features, target key point features and target semantic features of the input image respectively; and the feature fusion module is used for fusing the target edge features, the target key point features and the target semantic features and outputting a component segmentation result;

[0008] S2: constructing a training sample set composed of spacecraft ISAR images;

[0009] S3: training the segmentation network model by using the training sample set to obtain a trained segmentation network model;

[0010] S4: inputting an original spacecraft ISAR image to be recognized into the trained segmentation network model to obtain a spacecraft component segmentation result.

[0011] In an embodiment of the present application, the image preprocessing module comprises a dilation unit and an erosion unit, wherein,

[0012] the dilation unit is used for morphological dilation of the image input into the segmentation network model for multiple times, and the erosion unit is used for erosion operation of the dilated image for multiple times to obtain the preprocessed image.

[0013] In an embodiment of the present application, the image feature extraction module comprises a target edge feature extraction unit, a target key point feature extraction unit and a target semantic feature extraction unit, wherein,

[0014] The target edge feature extraction unit comprises a convolution layer, two convolution structures and two residual blocks connected in sequence, each convolution structure comprises a convolution layer, an activation layer and a pooling layer connected in sequence; each residual block comprises a plurality of residual structures, each residual structure comprises three convolution layers and a residual operation; the feature dimension output by each residual block is unified to the output dimension of the second convolution structure by using up-sampling and feature connection operation, and the two residual blocks and the features of the second convolution structure are merged to form a target edge feature set;

[0015] The target semantic feature extraction unit is a pyramid structure composed of a bottom-up network and a top-down network, the bottom-up network is used to extract basic features of different scales of an image, and the top-down network is used to down-sample the output result of the bottom-up network and up-fuse the basic features of different scales extracted by the bottom-up network to obtain features including rich semantic information.

[0016] The target key point feature extraction unit is connected to the output end of the last residual block of the target edge feature extraction unit, comprises a residual module and two deconvolution operations connected in sequence, and is used for extracting target key point features of an input image.

[0017] In an embodiment of the present application, the feature fusion module is specifically used for:

[0018] The mixed features after the feature channel connection of the target edge features, the target key point features and the target semantic features Input a first convolution layer, output a feature matrix A, and fuse semantic features Input a second convolution layer, output a feature matrix B; perform inner product operation on the feature matrix A and the feature matrix B to obtain a feature matrix S; input the feature matrix S into a third convolution layer, and perform convolution on the feature matrix S and the target semantic features Add pixel by pixel to obtain the final features

[0019]

[0020]

[0021]

[0022]

[0023] wherein, the target edge feature set is represented by E, the target semantic feature is represented by S, the target key point feature is represented by K, the convolution operation is represented by conv,​ represents a pixel-level addition operation, represents softmax a function.

[0024] In an embodiment of the present application, the S2 comprises:

[0025] S2.1: Based on the 3D model of the spatial target, the orbit elements are used to generate the on-orbit motion state of the target, and the ISAR simulation image sample set of the on-orbit spacecraft is generated by combining the imaging simulation method;

[0026] S2.2: The images in the image sample set are pixel-level labeled, and the spacecraft main body, left sailboard and right sailboard three component class labels are set;

[0027] S2.3: The key points in the ISAR simulation image sample set of the on-orbit spacecraft are labeled, and the main body 1, main body 2, left sailboard 1, left sailboard 2, right sailboard 1, right sailboard 2 are used to represent the component key point position in an ISAR image, so as to obtain a large number of images with class labels and key point labels, and form a training sample set.

[0028] In an embodiment of the present application, the S3 comprises:

[0029] The images with class labels and key point labels in the training sample set are input into the segmentation network model, and the network parameters in the image feature extraction module and the feature fusion module are updated respectively by using different loss functions, and when the preset update iteration number is reached, the trained segmentation network model is obtained.

[0030] In an embodiment of the present application, the target edge feature extraction unit uses a cross-entropy loss function to calculate the edge feature loss for model training, and the loss function expression is:

[0031]

[0032] Wherein, represents the loss function of the target edge feature extraction unit, represents the true label of the image edge feature point; represents the probability that the current edge feature point is predicted as a positive sample.

[0033] In an embodiment of the present application, the loss function of the target semantic feature extraction unit is:

[0034]

[0035] Wherein, is a weight coefficient; a true label representing an image semantic feature point; a probability that a current semantic feature point is predicted as a positive sample.

[0036] In an embodiment of the present application, the target key point feature extraction unit uses a mean square error loss function to calculate key point feature loss for model training, and the expression of the loss function is:

[0037]

[0038] wherein, L represents a loss function of the target key point feature extraction unit, N represents a number of key points, Xi represents a true coordinate of an i-th target key point, Yi represents a predicted label of the target key point feature extraction unit.

[0039] In an embodiment of the present application, the loss function of the feature fusion module is:

[0040] ,

[0041] wherein, is a preset coefficient.

[0042] Compared with the prior art, the present application has the following beneficial effects:

[0043] 1. The on-orbit spacecraft component segmentation method based on an ISAR image provided by the present application constructs a spacecraft component segmentation network model, which can further classify spacecraft sails compared with semantic segmentation in other related researches, and can provide a basis for spacecraft behavior cognition and other tasks.

[0044] 2. The method provided by the present application has a segmentation accuracy of 90.5%, and has high engineering application value.

[0045] 3. The present application verifies the effectiveness of artificial intelligence methods in the field of aerospace, which can provide strong support for judging the on-orbit operation state of space non-cooperative targets, and has high application value.

[0046] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is a flowchart of an on-orbit spacecraft component segmentation method based on an ISAR image provided by an embodiment of the present application;

[0048] Figure 2 is a structural schematic diagram of a spacecraft component segmentation network model provided by an embodiment of the present application;​

[0049] Figure 3 is a data processing process schematic diagram of a spacecraft component segmentation network model provided by an embodiment of the present application;

[0050] Figure 4 is a schematic diagram of a residual structure provided by an embodiment of the present application;

[0051] Figure 5 is an on-orbit spacecraft component segmentation result map obtained by using the on-orbit spacecraft component segmentation method of an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to further illustrate the technical means and effects adopted by the present application to achieve the predetermined purpose, a kind of on-orbit spacecraft component segmentation method based on ISAR image according to the present application is described in detail below in combination with the drawings and specific embodiments.

[0053] The foregoing and other technical contents, features and effects of the present application can be clearly presented in the specific embodiment description below in combination with the drawings. Through the description of the specific embodiments, the technical means and effects adopted by the present application to achieve the predetermined purpose can be more deeply and specifically understood. However, the attached drawings are provided for reference and explanation only, and are not intended to limit the technical solutions of the present application.

[0054] It should be noted that in this document, relational terms such as first and second are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any such actual relationship or order between such entities or actions. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a list of elements does not necessarily include only those elements in the list, but can include other elements not expressly listed or included. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus including the element.

[0055] Embodiment one

[0056] Please refer to Figure 1 , Figure 1 is a flowchart of an on-orbit spacecraft component segmentation method based on ISAR image provided by an embodiment of the present application. The method comprises:

[0057] S1: a segmentation network model of a spacecraft component is constructed, the segmentation network model comprising an image preprocessing module, an image feature extraction module and a feature fusion module, wherein the image preprocessing module is configured to perform morphological preprocessing on an image input into the segmentation network model to obtain a preprocessed image; the image feature extraction module is configured to extract target edge features, target key point features and target semantic features of the input image respectively; and the feature fusion module is configured to fuse the target edge features, the target key point features and the target semantic features and output a component segmentation result.

[0058] Specifically, referring to Figure 2 and Figure 3 , Figure 2 is a structural schematic diagram of a spacecraft component segmentation network model provided by an embodiment of the present application; Figure 3 is a data processing process schematic diagram of a spacecraft component segmentation network model provided by an embodiment of the present application. The image preprocessing module of the embodiment comprises an inflation unit and an erosion unit, wherein the inflation unit is configured to perform morphological inflation on an image input into the segmentation network model for multiple times, and the erosion unit is configured to perform erosion operation on the inflated image for multiple times to obtain a preprocessed image.

[0059] The image feature extraction module of the embodiment comprises a target edge feature extraction unit, a target key point feature extraction unit and a target semantic feature extraction unit, wherein

[0060] The target edge feature extraction unit comprises a convolution layer, two convolution structures and two residual blocks connected in sequence, each convolution structure comprises a convolution layer, an activation layer and a pooling layer connected in sequence; each residual block comprises a plurality of residual structures, each residual structure comprises three convolution layers and a residual operation; the feature dimension output by each residual block is unified to the output dimension of the second convolution structure by using up-sampling and feature connection operation, and the two residual blocks and the features of the second convolution structure are merged to form a target edge feature set; the target semantic feature extraction unit is a pyramid structure composed of a bottom-up network and a top-down network, the bottom-up network is configured to extract basic features of different scales of an image, and the top-down network is configured to down-sample the output result of the bottom-up network and up-fuse the basic features of different scales extracted by the bottom-up network to obtain features comprising rich semantic information; and the target key point feature extraction unit is connected to the output end of the last residual block of the target edge feature extraction unit and comprises a residual module and two deconvolution operations connected in sequence, and is configured to extract target key point features of an input image.

[0061] The feature fusion module of the embodiment is specifically configured to:

[0062] The mixed feature after the feature channel connection of the target edge feature, the target key point feature and the target semantic feature Input a first convolutional layer, output a feature matrix A, and fuse the semantic feature Input a second convolutional layer, output a feature matrix B; perform an inner product operation on the feature matrix A and the feature matrix B to obtain a feature matrix S; input the feature matrix S into a third convolutional layer, and perform convolution on the target semantic feature Pixel-wise addition to obtain the final feature .

[0063] Specifically, the image feature extraction module of the embodiment is composed of three parts, namely a target edge feature extraction module, a target key point feature extraction module and a target semantic feature extraction module, and the specific composition is as follows:

[0064] (1) The target edge feature extraction module includes one convolutional layer and two convolutional structures, and the convolutional structure is composed of: a convolutional layer -> an activation layer -> a pooling layer. In order to better extract the target feature, the network depth is increased by using residual calculation, and two residual modules are used after the convolutional structure, and each residual module includes a plurality of residual structures. Each residual structure includes three convolutional layers and one residual operation, which can make the network not degenerate with the increase of depth, as shown in Figure 4 , wherein HxWxC represents the height, width and channel size of the feature map; 1x1, C respectively represent the size of the network layer convolution kernel and the channel number of the output feature map; Relu represents the activation function used. The feature dimensions output by each residual module are unified to the output dimension of the second convolutional structure by using upsampling and feature connection operations, and the features of different layers are combined together to form a target edge feature set .

[0065] (2) The target semantic feature extraction module is a pyramid structure composed of a bottom-up network and a top-down network, and each layer of the pyramid structure includes a lateral connection and an upsampling operation. In the specific image processing process, first, the basic features are extracted by the bottom-up network, and then the output features of the latter network layer are fused upwards (specifically, in the bottom-up network, the feature map scale of the latter layer is basically 1 / 2 smaller than that of the former layer, and in the top-down network, the upward fusion means increasing the feature map from top to bottom layer by layer, and then performing pixel-level addition fusion with the feature map of the same scale in the bottom-up network), and finally obtaining a feature with a larger scale and rich semantic information. In network implementation, average pooling and maximum pooling operations are used in spatial dimension and channel dimension respectively, and then feature fusion is completed through feature splicing operation, a convolutional layer is designed to adjust the feature scale, and then a next residual block is connected to enhance the network feature expression ability and obtain a target semantic feature with stronger semantic information .

[0066] (3) Based on the target edge feature extraction module constructed in (1), a residual module including three residual structures is added after the module, and two deconvolution operations are performed on the final output feature map of the model to extract target key point features .

[0067] Further, the feature fusion module of the embodiment is composed of four convolution layers, one feature connection, one matrix multiplication operation and one pixel-level addition operation. The mixed features of the target edge features, the target key point features and the target semantic features after channel connection Input a convolution layer, output a feature matrix A, and input the target semantic features Input a convolution layer, output a feature matrix B; the inner product operation of the feature matrices A and B obtains a feature matrix S; input S into a convolution layer, and after convolution, add the semantic features Pixel by pixel, the final feature is obtained . Wherein:

[0068]

[0069]

[0070]

[0071]

[0072] Wherein, represents a target edge feature set, represents a target semantic feature, represents a target key point feature, represents a convolution operation, represents a pixel-level addition operation, represents softmax function.

[0073] S2: Construct a training sample set composed of spacecraft ISAR images.

[0074] Specifically, step S2 includes:

[0075] S2.1: Based on the 3D model of the space target, generate the on-orbit motion state of the target using the orbital elements, and generate an ISAR simulation image sample set of the on-orbit spacecraft by combining the imaging simulation method;

[0076] S2.2: Pixel-level labeling is performed on the images in the image sample set, and three component class labels of the spacecraft main body, the left sail plate and the right sail plate are set;

[0077] S2.3: Key point labeling is performed on the images in the ISAR simulation image sample set of the on-orbit spacecraft, six labels of subject 1, subject 2, left sailboard 1, left sailboard 2, right sailboard 1 and right sailboard 2 are used to represent the positions of the key points of the components in an ISAR image, so as to obtain a large number of images with category labels and key point labels, and form a training sample set.

[0078] S3: The segmentation network model is trained by using the training sample set, and a trained segmentation network model is obtained.

[0079] Specifically, the images with category labels and key point labels in the training sample set are input into the segmentation network model, and the network parameters in the image feature extraction module and the feature fusion module are continuously updated by using different loss functions, and when a preset number of update iterations is reached, the trained segmentation network model is obtained.

[0080] In the embodiment, the target edge feature extraction unit uses a cross-entropy loss function to calculate edge feature loss for model training, and the loss function expression is:

[0081]

[0082] wherein, Ledge represents the loss function of the target edge feature extraction unit, represents the real label of the image edge feature point, the positive sample value is 1, and the negative sample value is 0; represents the probability that the current edge feature point is predicted as a positive sample.

[0083] The loss function of the target semantic feature extraction unit is:

[0084]

[0085] wherein, is a weight coefficient, used to balance the influence of positive and negative samples on network training; represents the real label of the image semantic feature point, the positive sample value is 1, and the negative sample value is 0; represents the probability that the semantic feature point is predicted as a positive sample.

[0086] The target key point feature extraction unit uses a mean square error loss function to calculate key point feature loss for model training, and the expression of the loss function is:

[0087]

[0088] wherein, Lkey represents the loss function of the target key point feature extraction unit, represents the number of key feature points, representing the true label of the first feature point, i.e., the true coordinates of the target key point, representing the corresponding predicted label, i.e., the coordinates predicted by the feature network.

[0089] The loss function of the feature fusion module is:

[0090] ,

[0091] wherein, is a preset coefficient, and in the embodiment, is taken as .

[0092] S4: inputting the original spacecraft ISAR image to be recognized into the trained segmentation network model to obtain a spacecraft component segmentation result.

[0093] The embodiment proposes an on-orbit spacecraft component segmentation method based on an ISAR image, constructs a spacecraft component segmentation network model, and a spacecraft component intelligent instance segmentation network model based on an ISAR image. Compared with semantic segmentation in other related researches, the spacecraft component intelligent instance segmentation network model based on an ISAR image can further classify a spacecraft sailboard and can provide a basis for a spacecraft behavior cognition task.

[0094] Embodiment Two

[0095] Based on the embodiment one, the embodiment further provides an on-orbit spacecraft component segmentation method based on an ISAR image, and the method comprises the following steps:

[0096] S1: constructing a spacecraft ISAR image training sample set and a test sample set.

[0097] Specifically, S1 of the embodiment comprises the following steps:

[0098] S11: based on 3000 Tiangong-1 space station ISAR imaging simulation samples, the embodiment uses target TLE root numbers and ground-based radar imaging equipment simulation, and the sample size is 384*384. First, the ISAR samples are pixel-level labeled according to three categories of main body, left sailboard and right sailboard, to generate corresponding true value labels and files;

[0099] S12: key point labeling is performed on the target structure in the ISAR sample, and six key point positions of target main body 1, main body 2, left sailboard 1, left sailboard 2, right sailboard 1 and right sailboard 2 in the image are marked, to generate corresponding sample key point labels;

[0100] S13: 400 images are randomly extracted from the ISAR image sample set as a test set, and the remaining 2600 images are training sample sets.

[0101] S2: Construct a segmentation network model of the spacecraft component.

[0102] The intelligent segmentation model of the spacecraft important component based on the Tiangong ISAR image mainly includes three parts: an image preprocessing module, an image feature extraction module, and a feature fusion module, and the network is specifically constructed as follows:

[0103] S21: Input image preprocessing: morphological dilation and erosion operations are used on the images in the training sample set, 3 times of dilation and 3 times of erosion, respectively, as the input of the subsequent model.

[0104] S22: Build a target edge feature extraction unit, see Figure 2 and Figure 3 . After the input image is processed by the first three convolutional layers of the model, a 96x96x256 feature map is obtained, and after two residual blocks are processed, a 48x48x256 and a 24x24x256 feature map are obtained, respectively. The 48x48x256 and 24x24x256 feature maps are processed using an upsampling interpolation operation to obtain two groups of 96x96x256 feature maps, which are combined with other 96x96x256 feature maps using feature connection; the combined 96x96x768 feature map is processed using a convolutional layer to obtain a target edge feature of 96x96x512 .

[0105] S23: Build a target key point feature extraction unit. After the input image is processed by the convolutional layer and the residual layer, a 24x24x512 feature map is obtained, and two deconvolution operations are used to obtain a target key point feature of 96x96x512 .

[0106] S24: Build a target semantic feature extraction unit. After the input image is processed by the bottom-up network, 96x96x256, 48x48x256, and 24x24x256 feature maps are obtained, respectively. After the upsampling and lateral connection operations in the top-down network, a 96x96x768 feature is obtained, and a convolutional layer is used to process it to obtain a target semantic feature of 96x96x512 .

[0107] S25: Build a target feature fusion module. The target edge feature, the target key point feature, and the target semantic feature are connected to obtain a 96x96x1536 feature map, which is processed by a convolutional layer with a 1x1x512 convolutional kernel to obtain a 96x96x512 feature map; respectively, and the target semantic feature is transformed into a 9216x512 and a 512x9216 matrix, and a 9216x9216 correlation matrix is obtained by matrix multiplication ; after convolution operation and target semantic feature element-wise sum, obtaining a fusion feature of 96x96x512 .

[0108] S3: training of the segmentation network model of the spacecraft component. The constructed Tiangong ISAR image training sample set is used to train the segmentation model, the iteration number is set to 150 times, and the trained model parameters are saved. The loss function expression is described in Embodiment 1.

[0109] S4: segmentation prediction of the Tiangong ISAR image test sample set using the trained model, and output of the segmentation result, as shown in Figure 5 , wherein the upper left image is the original image, the upper right image is the segmentation prediction result, and the lower left image and the lower right image are the target key point and edge visualization results of the ISAR image, respectively.

[0110] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0111] In addition, each function module in each embodiment of the present application can be integrated in one processing module, or each module can exist physically, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of hardware plus software function module.

[0112] Yet another embodiment of the present application provides a storage medium having stored therein a computer program for performing the steps of the ISAR image based on-orbit spacecraft component segmentation method described in the above embodiments. Still another aspect of the present application provides an electronic device comprising a memory having stored therein a computer program and a processor which, when invoking the computer program stored in the memory, implements the steps of the ISAR image based on-orbit spacecraft component segmentation method described in the above embodiments. Specifically, the integrated modules implemented in the form of software functional modules described above can be stored in a computer readable storage medium. The software functional modules described above are stored in a storage medium, including a number of instructions for causing an electronic device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform part of the steps of the method described in various embodiments of the present application. And the aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0113] The above is a further detailed description of the present application in combination with specific preferred embodiments, and cannot be deemed to limit the specific implementation of the present application to these descriptions. For those of ordinary skill in the art to which the present application belongs, without departing from the concept of the present application, a number of simple deductions or substitutions can also be made, which should all be considered to fall within the protection scope of the present application.

Claims

1. An ISAR image-based on-orbit spacecraft component segmentation method, characterized in that, The method comprises the following steps: S1: constructing a segmentation network model of a spacecraft component, the segmentation network model comprising an image preprocessing module, an image feature extraction module and a feature fusion module, wherein the image preprocessing module is configured to perform morphological preprocessing on an image input into the segmentation network model to obtain a preprocessed image; the image feature extraction module is configured to extract target edge features, target key point features and target semantic features of the input image respectively; and the feature fusion module is configured to fuse the target edge features, the target key point features and the target semantic features and output a component segmentation result; S2: constructing a training sample set composed of ISAR images of a spacecraft; S3: training the segmentation network model using the training sample set to obtain a trained segmentation network model; S4: inputting an original spacecraft ISAR image to be recognized into the trained segmentation network model to obtain a spacecraft component segmentation result; The image feature extraction module comprises a target edge feature extraction unit, a target key point feature extraction unit and a target semantic feature extraction unit, wherein The target edge feature extraction unit comprises a convolution layer, two convolution structures and two residual blocks connected in sequence, each convolution structure comprises a convolution layer, an activation layer and a pooling layer connected in sequence, and each residual block comprises a plurality of residual structures, each residual structure comprising three convolution layers and a residual operation; the feature dimension output by each residual block is unified to the output dimension of the second convolution structure by using an up-sampling and a feature connection operation, and the two residual blocks and the features of the second convolution structure are merged to form a target edge feature set; The target semantic feature extraction unit is a pyramid structure composed of a bottom-up network and a top-down network, the bottom-up network is configured to extract basic features of different scales of an image, and the top-down network is configured to down-sample the output result of the bottom-up network and fuse the basic features of different scales extracted by the bottom-up network upward to obtain features comprising rich semantic information; The target key point feature extraction unit is connected to the output end of the last residual block of the target edge feature extraction unit and comprises a residual module and two deconvolution operations connected in sequence, and is configured to extract target key point features of an input image; The feature fusion module is specifically configured to: The mixed feature after the feature channel connection of the target edge feature, the target key point feature and the target semantic feature Input a first convolutional layer, output a feature matrix A, and input the semantic feature Input a second convolutional layer, output a feature matrix B; perform inner product operation on the feature matrix A and the feature matrix B to obtain a feature matrix S; input the feature matrix S into a third convolutional layer, and perform convolution on the target semantic feature Add by pixels to obtain the final feature : wherein, denotes a target edge feature set, denotes a target semantic feature, denotes a target keypoint feature, denotes a convolution operation, denotes a pixel-wise addition operation, denotes softmax a function.

2. The ISAR image based on-orbit spacecraft component segmentation method according to claim 1, characterized in that, The image preprocessing module comprises a dilation unit and an erosion unit, wherein The dilation unit is configured to perform morphological dilation on an image input into the segmentation network model for multiple times, and the erosion unit is configured to perform erosion operation on the dilated image for multiple times to obtain a preprocessed image.

3. The ISAR image based on-orbit spacecraft component segmentation method according to claim 1, characterized in that, The S2 comprises: S2.1: based on a 3D model of a space target, generating a target on-orbit motion state using orbit elements, and generating an ISAR simulation image sample set of an on-orbit spacecraft by combining an imaging simulation method; S2.2: performing pixel-level labeling on the images in the image sample set, and setting three component class labels of a spacecraft main body, a left sail plate and a right sail plate; S2.3: key point labeling is performed on images in the ISAR simulation image sample set of the on-orbit spacecraft, 6 labels including main body 1, main body 2, left sail plate 1, left sail plate 2, right sail plate 1 and right sail plate 2 are used to represent the positions of component key points in an ISAR image, so as to obtain a large number of images with category labels and key point labels, and form a training sample set.

4. The ISAR image based on-orbit spacecraft component segmentation method according to claim 3, characterized in that, The S3 comprises: The images with the category labels and the key point labels in the training sample set are input into the segmentation network model, network parameters in the image feature extraction module and the feature fusion module are continuously updated by using different loss functions respectively, and when a preset update iteration number is reached, a trained segmentation network model is obtained.

5. The ISAR image based on-orbit spacecraft component segmentation method according to claim 4, characterized in that, The target edge feature extraction unit uses a cross-entropy loss function to calculate edge feature loss for model training, and the loss function expression is: wherein, represents a loss function of the target edge feature extraction unit, represents a real label of an image edge feature point; represents a probability that a current edge feature point is predicted as a positive sample.

6. The ISAR image based on-orbit spacecraft component segmentation method according to claim 5, characterized in that, The loss function of the target semantic feature extraction unit is: wherein, is a weight coefficient; denotes a true label of an image semantic feature point, denotes a probability that a current semantic feature point is predicted as a positive sample.

7. The ISAR image based on-orbit spacecraft component segmentation method according to claim 6, characterized in that, The target key point feature extraction unit uses a mean square error loss function to calculate key point feature loss for model training, and the loss function expression is: in, This represents the loss function of the target key point feature extraction unit. Indicates the number of key points. Indicates the first The true coordinates of each target key point This represents the predicted label of the target key point feature extraction unit.

8. The ISAR image based on-orbit spacecraft component segmentation method according to claim 7, characterized in that, The loss function of the feature fusion module is: ， wherein is a preset coefficient.

Images