A method and device for spacecraft attitude recognition based on convolutional neural network

Abstract

The disclosed embodiment relates to a method and device for aerospace vehicle attitude recognition based on a convolutional neural network, wherein the method includes: obtaining an image to be recognized; processing the image to be recognized based on a pre-trained aerospace vehicle attitude recognition model to obtain a feature map, and deconvolving the feature map to generate a thermal map; classifying each pixel point in the thermal map based on the classification function of the aerospace vehicle attitude recognition model to determine the probability of each structural point in the thermal map; determining the target structural point and the coordinate position of the target structural point based on the probability of each structural point in the thermal map and a preset probability threshold, and performing coordinate compensation on the coordinate position of the target structural point to obtain the target coordinate position corresponding to the target aerospace vehicle. Thus, the precise coordinates of the key external structural points of the target aerospace vehicle in the image can be identified with high precision to solve the problem of high-precision attitude recognition of the target aerospace vehicle in the key control stage.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to a method, device, equipment and medium for aerospace vehicle attitude recognition based on a convolutional neural network. Background Art

[0002] With the rapid development of aerospace technology, the requirements for the control accuracy of spacecraft's flight attitude and spatial position are becoming increasingly higher. Especially during take-off, landing, and docking, due to the complex environment, high operating requirements, and strict position accuracy requirements, position and attitude control is crucial for spacecraft to accurately complete their scheduled tasks.

[0003] Traditional attitude control methods are achieved through gyroscopes and other sensors. On the one hand, there is a bottleneck in control accuracy, making it difficult to meet millimeter-level requirements. On the other hand, the installation locations of sensors are limited, making it difficult to determine the exact position and attitude of all specified external structures of the aircraft. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a method, device, equipment and medium for aerospace vehicle attitude recognition based on convolutional neural network.

[0005] The disclosed embodiment provides a method for spacecraft attitude recognition based on a convolutional neural network, comprising: acquiring an image to be recognized; processing the image to be recognized based on a pre-trained spacecraft attitude recognition model to obtain a feature map, and performing deconvolution on the feature map to generate a heat map; classifying each pixel point in the heat map based on a classification function of the spacecraft attitude recognition model to determine the probability of each structure point in the heat map; determining a target structure point and a coordinate position of the target structure point based on the probability of each structure point in the heat map and a preset probability threshold, and performing coordinate compensation on the coordinate position of the target structure point to obtain a target coordinate position corresponding to the target spacecraft.

[0006] Optionally, the method further includes: photographing target pixel images of the target spacecraft in different motion scenes, and obtaining the coordinates of key external structure points corresponding to each target pixel image, to obtain image samples; performing augmentation processing on the image samples to obtain an image data set; dividing the image data set into an image training set and an image test set; performing supervised learning on a pre-constructed convolutional neural network model using the image training set, and determining a loss value based on a preset loss function, and when the loss value is greater than or equal to a preset threshold, adjusting the model parameters of the convolutional neural network model until the loss value is less than the threshold, to obtain the spacecraft posture recognition model.

[0007] Optionally, the method further includes: inputting the image test set into the aerospace vehicle attitude recognition model for testing to obtain test results, and adjusting model parameters of the aerospace vehicle attitude recognition model based on the test results.

[0008] Optionally, the method also includes: obtaining a ResNet model and deleting the output layer in the ResNet model to obtain a backbone network; adjusting the upsampling step size of the last module of the ResNet model and superimposing a feature pyramid FPN network on the backbone network to obtain the pre-built convolutional neural network model.

[0009] Optionally, determining the target structural point and the coordinate position of the target structural point based on the probability of each structural point in the heat map and a preset probability threshold includes: comparing the probability of each structural point with a preset probability threshold, and obtaining a target area with a probability greater than the probability threshold; taking the structural point corresponding to the highest probability in the target area as the target structural point, and obtaining the coordinate information corresponding to the target structural point as the coordinate position of the target structural point.

[0010] Optionally, the coordinate compensation of the coordinate position of the target structure point to obtain the target coordinate position corresponding to the target spacecraft includes: performing deconvolution on the feature map to obtain the coordinate compensation value of the target structure point corresponding to the original image coordinate; mapping the coordinate position of the target structure point to the original image to obtain the mapping coordinate value; generating the center coordinate of the target structure point based on the mapping coordinate value and the coordinate compensation value; and adding the coordinate compensation value of the edge position of the target structure point generated by the FPN network in the spacecraft posture recognition model to the center coordinate of the target structure point to obtain the target coordinate position corresponding to the target spacecraft.

[0011] Optionally, the method also includes: installing a preset software environment on the target device; uploading a file corresponding to the spacecraft attitude recognition model to the target device; writing an interface program to receive the image to be recognized to call the spacecraft attitude recognition model for attitude recognition, obtain the target coordinate position corresponding to the target spacecraft, and send the target coordinate position corresponding to the target spacecraft to the client or store it in a database.

[0012] The disclosed embodiment also provides a convolutional neural network-based aerospace vehicle attitude recognition device, comprising: an acquisition module for acquiring an image to be recognized; a processing module for processing the image to be recognized based on a pre-trained aerospace vehicle attitude recognition model to obtain a feature map; a generation module for deconvolving the feature map to generate a heat map; a classification module for classifying each pixel point in the heat map based on a classification function of the aerospace vehicle attitude recognition model to determine the probability of each structure point in the heat map; a determination module for determining a target structure point and a coordinate position of the target structure point based on the probability of each structure point in the heat map and a preset probability threshold; and a compensation module for performing coordinate compensation on the coordinate position of the target structure point to obtain a target coordinate position corresponding to a target aerospace vehicle.

[0013] An embodiment of the present disclosure also provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the aerospace vehicle attitude recognition method based on convolutional neural network as provided in the embodiment of the present disclosure.

[0014] An embodiment of the present disclosure also provides a computer-readable storage medium, which stores a computer program, and the computer program is used to execute the aerospace vehicle attitude recognition method based on convolutional neural network as provided in the embodiment of the present disclosure.

[0015] The embodiment of the present disclosure also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the aerospace vehicle attitude recognition method based on convolutional neural network described in the above aspect.

[0016] Compared with the prior art, the technical solution provided by the embodiment of the present disclosure has the following advantages: the aerospace vehicle attitude recognition solution based on convolutional neural network provided by the embodiment of the present disclosure includes: obtaining an image to be recognized; processing the image to be recognized based on a pre-trained aerospace vehicle attitude recognition model to obtain a feature map, and deconvolving the feature map to generate a heat map; classifying each pixel point in the heat map based on the classification function of the aerospace vehicle attitude recognition model to determine the probability of each structural point in the heat map; determining the target structural point and the coordinate position of the target structural point based on the probability of each structural point in the heat map and a preset probability threshold, and performing coordinate compensation on the coordinate position of the target structural point to obtain the target coordinate position corresponding to the target aerospace vehicle. Thus, the precise coordinates of the key external structural points of the target aerospace vehicle in the image can be identified with high precision to solve the problem of high-precision attitude recognition of the target aerospace vehicle in the key control stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0018] Figure 1 A schematic diagram of a flow chart of a method for aerospace vehicle attitude recognition based on a convolutional neural network provided in an embodiment of the present disclosure;

[0019] Figure 2 A schematic flow chart of another method for aerospace vehicle attitude recognition based on a convolutional neural network provided in an embodiment of the present disclosure;

[0020] Figure 3 A schematic diagram of the structure of a spacecraft attitude recognition device based on a convolutional neural network provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0022] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0023] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0024] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0025] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0026] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0027] At present, the attitude control method of spacecraft is achieved through gyroscopes and other sensors. On the one hand, there is a bottleneck in control accuracy, and it is difficult to meet the millimeter-level requirements. On the other hand, the installation locations of sensors are limited, and it is difficult to determine the exact position and attitude of all specified external structures of the aircraft.

[0028] In view of the above problems, with the rapid development of artificial intelligence and computer vision technology, the posture recognition and image segmentation technology based on convolutional neural networks has been greatly improved, and has achieved good application results in the posture recognition scenes of objects such as human bodies, animals, and aircraft. The present disclosure implements a method for aerospace vehicle posture recognition based on convolutional neural networks, obtains an image to be recognized; processes the image to be recognized based on a pre-trained aerospace vehicle posture recognition model to obtain a feature map, and deconvolves the feature map to generate a heat map; classifies each pixel point in the heat map based on the classification function of the aerospace vehicle posture recognition model to determine the probability of each structural point in the heat map; determines the target structural point and the coordinate position of the target structural point based on the probability of each structural point in the heat map and the preset probability threshold, and coordinates the coordinate position of the target structural point to obtain the target coordinate position corresponding to the target aerospace vehicle. In this way, the precise coordinates of the key external structure points of the target aerospace vehicle in the image can be identified with high precision to solve the problem of high-precision posture recognition of the target aerospace vehicle in the key control stage.

[0029] Figure 1 The present invention provides a flow chart of a method for spacecraft attitude recognition based on a convolutional neural network according to an embodiment of the present invention. The method can be performed by a spacecraft attitude recognition device based on a convolutional neural network, wherein the device can be implemented by software and / or hardware and can generally be integrated in an electronic device. Figure 1 As shown, the method includes:

[0030] Step 101: Obtain an image to be recognized.

[0031] In the embodiment of the present disclosure, the image to be identified is generally an image including a target spacecraft.

[0032] In the embodiments of the present disclosure, there are many ways to obtain the image to be identified. The image to be identified can be received from a terminal device such as a mobile phone based on a pre-written interface program, or the target spacecraft can be directly photographed in real time through a shooting device to obtain the image to be identified, etc. The specific settings are selected according to the actual application needs.

[0033] Step 102: Process the image to be recognized based on the pre-trained aerospace vehicle attitude recognition model to obtain a feature map, and perform deconvolution on the feature map to generate a heat map.

[0034] Step 103: classify each pixel point in the thermal map based on the classification function of the spacecraft attitude recognition model to determine the probability of each structural point in the thermal map.

[0035] In the disclosed embodiment, a spacecraft attitude recognition model is pre-trained. Specifically, a ResNet model is obtained, and the output layer in the ResNet model is deleted to obtain a backbone network. The upsampling step size of the last module of the ResNet-50 model is adjusted, and a feature pyramid FPN (Feature Pyramid Network) network is superimposed on the backbone network to obtain a pre-constructed convolutional neural network model. Target pixel images of the target spacecraft in different motion scenes are photographed, and the coordinates of key external structure points corresponding to each target pixel image are obtained to obtain image samples. The image samples are augmented to obtain image data. The image data set is divided into an image training set and an image test set. The image training set is used to perform supervised learning on the pre-constructed convolutional neural network model, and a loss value is determined based on a preset loss function. When the loss value is greater than or equal to a preset threshold, the model parameters of the convolutional neural network model are adjusted until the loss value is less than the threshold to obtain a spacecraft attitude recognition model.

[0036] It should be noted that one or more convolutional neural networks can be selected according to actual application needs to build a pre-built convolutional neural network model, and supervised learning can be performed based on the image training set to adjust the model parameters of the convolutional neural network model until the loss value is less than the threshold, so as to obtain a spacecraft attitude recognition model.

[0037] In order to further improve the recognition effect of the spacecraft attitude recognition model, the image test set can be input into the spacecraft attitude recognition model for testing, the test results can be obtained, and the model parameters of the spacecraft attitude recognition model can be adjusted based on the test results.

[0038] Therefore, after obtaining the image to be identified, the image to be identified can be directly input into the aerospace vehicle attitude recognition model for processing to finally obtain the target coordinate position corresponding to the target aerospace vehicle.

[0039] Specifically, the aerospace vehicle attitude recognition model processes the image to be recognized, obtains the feature map corresponding to the image to be recognized, deconvolves the feature map to generate a heat map, and classifies each pixel in the heat map through a classification function to determine the probability of each structure point in the heat map. That is to say, by performing a deconvolution operation on the feature map, a heat map of a certain size can be generated, such as a heat map with a size of 1 / 8 of the image to be recognized, and a classification function such as a "sigmoid" function is used to classify each pixel in the heat map to obtain the probability of each structure point; wherein each target aircraft has a total target number, such as 20 structure points, and some or all of the structure points may appear in the image to be recognized, and the probability of a structure point refers to the probability of the pixel in the heat map being a structure point.

[0040] Step 104, determine the target structure point and the coordinate position of the target structure point based on the probability of each structure point in the thermal map and the preset probability threshold, and perform coordinate compensation on the coordinate position of the target structure point to obtain the target coordinate position corresponding to the target spacecraft.

[0041] In the embodiment of the present disclosure, a probability threshold is preset and is selected and set according to an actual application scenario.

[0042] In the embodiments of the present disclosure, there are many ways to determine the target structure point and the coordinate position of the target structure point based on the probability of each structure point in the heat map and a preset probability threshold, such as comparing the probability of each structure point with the preset probability threshold, obtaining the target area with a probability greater than the probability threshold, taking the structure point corresponding to the highest probability in the target area as the target structure point, and obtaining the coordinate information corresponding to the target structure point as the coordinate position of the target structure point; for example, directly taking the structure point in the identified heat map with a probability greater than the preset probability threshold as the target structure point, and obtaining the coordinate position of the target structure point.

[0043] Furthermore, coordinate compensation is performed on the coordinate position of the target structure point to obtain the target coordinate position corresponding to the target spacecraft. For example, deconvolution is performed on the feature map to obtain the coordinate compensation value of the original image coordinate corresponding to the target structure point, and the coordinate position of the target structure point is mapped to the original image to obtain the mapping coordinate value. Based on the mapping coordinate value and the coordinate compensation value, the center coordinate of the target structure point is generated, and the coordinate compensation value of the edge position of the target structure point generated based on the FPN network in the spacecraft attitude recognition model is added to the center coordinate of the target structure point to obtain the target coordinate position corresponding to the target spacecraft. It should be noted that one or more coordinate compensations can be selected according to actual application needs.

[0044] The spacecraft attitude recognition scheme based on convolutional neural network provided by the embodiment of the present disclosure obtains the image to be recognized; processes the image to be recognized based on the pre-trained spacecraft attitude recognition model to obtain a feature map, and deconvolves the feature map to generate a heat map; classifies each pixel point in the heat map based on the classification function of the spacecraft attitude recognition model to determine the probability of each structure point in the heat map; determines the target structure point and the coordinate position of the target structure point based on the probability of each structure point in the heat map and the preset probability threshold, and coordinates the coordinate position of the target structure point to obtain the target coordinate position corresponding to the target spacecraft. In this way, the precise coordinates of the key external structure points of the target spacecraft in the image can be identified with high precision to solve the problem of high-precision attitude recognition of the target spacecraft in the key control stage.

[0045] Based on the description of the above embodiments, the spacecraft posture recognition method based on convolutional neural network of the embodiment of the present disclosure can collect image data containing the target spacecraft, and mark the precise coordinates of the key external structure points of the target number, pre-process the collected image data, including grayscale, normalization and size adjustment operations to meet the input requirements of the convolutional neural network, build a convolutional neural network model, and use the labeled data set for training, optimize the model parameters to improve the accuracy of posture recognition, use an independent verification data set to verify the trained model, evaluate the performance of the model, deploy the trained model to actual applications, receive spacecraft image data in real time, and output posture recognition results.

[0046] Specifically, Figure 2 This is a flow chart of another method for spacecraft attitude recognition based on a convolutional neural network provided by the present disclosure. Based on the above embodiment, this embodiment further optimizes the above method for spacecraft attitude recognition based on a convolutional neural network. Figure 2 As shown, the method includes:

[0047] Step 201, obtain a ResNet model, delete the output layer in the ResNet model, obtain a backbone network, adjust the upsampling step size of the last module of the ResNet model, and superimpose a feature pyramid FPN network on the backbone network to obtain a pre-built convolutional neural network model.

[0048] Specifically, a convolutional neural network model based on ResNet is constructed. For example, ResNet-50 is used as the backbone network for feature extraction, the output layer of ResNet-50 is removed, and the upsampling step size of the last block of ResNet-50 is adjusted to make the size of the output feature map a specific ratio of the original image, such as 1 / 16. The FPN network is superimposed on the backbone network to improve the detection accuracy of the edges of structure points, thereby extracting high-level features of the image and providing a basis for subsequent heat map generation and structure point recognition.

[0049] Step 202: photograph target pixel images of the target spacecraft in different motion scenes, obtain the coordinates of key shape structure points corresponding to each target pixel image, obtain image samples, perform augmentation processing on the image samples, and obtain an image data set.

[0050] Specifically, images of the target spacecraft in different motion scenes are captured, so as to select images in different positions, different postures, and different lighting conditions; more specifically, images containing the target spacecraft are captured using a high-resolution camera, ensuring that the captured images cover a variety of scenes, including different positions, postures, and lighting conditions, and that the size of each image is the target size, such as 1280*1024 pixels. Thus, diversified image data is obtained to ensure that the model can effectively identify the posture of the target spacecraft in different environments.

[0051] Furthermore, the collected images are manually annotated to mark the key structural points of the target spacecraft. Specifically, the aircraft structural points in the target pixel image are annotated using labeling software such as LabelMe software. Usually, each target spacecraft has a certain number of structural points, such as 20, and some or all of the structural points may appear in the image. Thus, accurate annotation data is provided for model training and verification.

[0052] Furthermore, the diversity of the data set is increased by transforming the images. Specifically, the images are rotated, scaled, translated, flipped, and other operations are performed, and different types of noise are inserted into the images, such as Gaussian noise, salt and pepper noise, etc. Through the above methods, for example, the data set is expanded from 4,000 images to 100,000 images. In this way, the generalization ability of the model is enhanced, so that it can maintain a high recognition accuracy under different conditions.

[0053] Step 203: divide the image data set into an image training set and an image test set, use the image training set to perform supervised learning on the pre-built convolutional neural network model, and determine the loss value based on a preset loss function. When the loss value is greater than or equal to a preset threshold, adjust the model parameters of the convolutional neural network model until the loss value is less than the threshold, so as to obtain a spacecraft attitude recognition model.

[0054] In the disclosed embodiment, the image data set is divided into an image training set and an image test set. For example, 30,000 images are randomly selected from 100,000 images as the image test set, and the remaining 70,000 images are used as the image training set. In this way, it is ensured that the model has sufficient data during the training process, and the performance of the model is verified on the test set.

[0055] Further, the model is trained using the image training set. Specifically, the model is supervised by using the labeled image training set, and a suitable loss function is selected, such as the mean square error loss function (MSE) or the cross entropy loss function. The model parameters are optimized using an optimization algorithm (such as Adam, SGD (Stochastic Gradient Descent)), and the appropriate number of training rounds and batch size are set to ensure model convergence and obtain a spacecraft attitude recognition model. Thus, the model is trained to accurately recognize the attitude of the spacecraft.

[0056] Step 204: input the image test set into the aerospace vehicle attitude recognition model for testing, obtain the test results, and adjust the model parameters of the aerospace vehicle attitude recognition model based on the test results.

[0057] In the disclosed embodiment, the trained model is verified using an image test set, the images in the image test set are input into the trained model, the coordinates of the structure points output by the model are compared with the annotation data, the recognition accuracy is calculated, and the model parameters are adjusted according to the verification results to further optimize the model performance. In this way, the performance of the model is evaluated to ensure that it can achieve the expected effect in practical applications.

[0058] Step 205: Install a preset software environment on the target device, upload the corresponding file of the spacecraft attitude recognition model to the target device, and write an interface program to obtain the image to be recognized.

[0059] In the disclosed embodiment, the trained model is deployed to the actual application, the necessary software environment, such as Python, TensorFlow, etc., is installed on the target device such as the server, the trained model file is uploaded to the server, an interface program is written, image data is received and the model is called for gesture recognition, and the recognition result is returned to the client or stored in a database, etc. Thus, the actual application of the model is realized and an efficient gesture recognition service is provided.

[0060] Step 206: Process the image to be recognized based on the pre-trained aerospace vehicle attitude recognition model to obtain a feature map, deconvolve the feature map to generate a heat map, classify each pixel in the heat map based on the classification function of the aerospace vehicle attitude recognition model, and determine the probability of each structural point in the heat map.

[0061] Step 207: compare the probability of each structure point with a preset probability threshold, obtain a target area with a probability greater than the probability threshold, take the structure point corresponding to the highest probability in the target area as the target structure point, and obtain the corresponding coordinate information of the target structure point as the coordinate position of the target structure point.

[0062] In the disclosed embodiment, the feature map is deconvolved and classified to generate a heat map. Specifically, the feature map is deconvolved to generate a heat map with a size of, for example, 1 / 8 of the original image. A classification function, such as a sigmoid function, is used to classify each pixel in the heat map to obtain the probability of each structure point. Thus, a heat map is generated for subsequent structure point recognition.

[0063] Furthermore, the location of the structural point is identified in the heat map. Specifically, in the heat map, the area with a probability greater than the threshold is selected. In these areas, the peak point is taken as the location of the structural point, and other points are suppressed to avoid repeated identification. In this way, the location of the structural point in the heat map is accurately identified, and false detection and missed detection are reduced.

[0064] Specifically, the heat map generation formula is H ij =σ(f(F ij )); where F ij is the feature vector in the i-th row and j-th column of the feature map, f is the deconvolution operation, σ is the sigmoid function, H ij is the pixel value of the i-th row and j-th column in the heat map.

[0065] Among them, Fij represents the feature vector in the feature map, which reflects the characteristics of the local area of ​​the image; f is the deconvolution operation, which is used to enlarge the feature map to the size of the heat map; σ is the sigmoid function, which is used to map the feature value to a probability value between 0 and 1; Hij represents the pixel value in the heat map, which reflects the probability of whether there is a structural point at that position.

[0066] Step 208, deconvolve the feature map to obtain the coordinate compensation value of the target structure point corresponding to the original image coordinate, map the coordinate position of the target structure point to the original image to obtain the mapping coordinate value, and generate the center coordinate of the target structure point based on the mapping coordinate value and the coordinate compensation value.

[0067] Step 209: Add the coordinate compensation value of the edge position of the target structure point generated by the FPN network in the aerospace vehicle attitude recognition model to the center coordinate of the target structure point to obtain the target coordinate position corresponding to the target aerospace vehicle.

[0068] In the disclosed embodiment, coordinate compensation is performed on the structure points identified in the thermal map to improve the recognition accuracy. Specifically, the feature map output by ResNet-50 is deconvolved, and the coordinate compensation value of the structure points identified in the thermal map corresponding to the original image coordinates is predicted. The position of the peak point in the thermal map is mapped to the coordinate value in the original image (the image to be recognized), and the coordinate compensation value of the corresponding position in the feature map is added to generate the center coordinate of the target structure point. The coordinate compensation value of the edge position of the structure point is generated from the FPN network, and the structure point compensation coordinates are further superimposed to generate the final coordinates, that is, the target coordinate position corresponding to the target spacecraft. In this way, the scale difference between the thermal map and the original image is compensated, and the accuracy of structure point recognition is improved.

[0069] Among them, the coordinate compensation formula is (x', y') = (x + △x, y + △y); among them, (x, y) are the coordinates of the peak point in the heat map, (△x, △y) are the coordinate compensation values, and (x', y') are the coordinates of the structure point finally identified; among them, (x, y) are the coordinates of the peak point in the heat map, indicating the position of the structure point in the heat map; (△x, △y) are the coordinate compensation values ​​used to correct the scale difference between the heat map and the original image; (x', y') are the coordinates of the structure point finally identified, indicating the position of the structure point in the original image.

[0070] Therefore, the coordinates of the structural points identified in the thermal map are compensated to improve the recognition accuracy. The FPN network is superimposed on the backbone network and used to generate further coordinate compensation for the structural points to improve the recognition accuracy of the structural point positions. This makes it possible to accurately identify the precise coordinates of the key external structural points of a specific spacecraft in the image, so as to solve the problem of high-precision attitude recognition of the target spacecraft in the critical control stage.

[0071] Figure 3 The present invention provides a schematic diagram of the structure of a spacecraft attitude recognition device based on a convolutional neural network, which can be implemented by software and / or hardware and can generally be integrated into electronic devices. Figure 3 As shown, the device comprises:

[0072] An acquisition module 301 is used to acquire an image to be recognized;

[0073] A processing module 302 is used to process the image to be recognized based on a pre-trained aerospace vehicle attitude recognition model to obtain a feature map;

[0074] A generating module 303, configured to perform deconvolution on the feature map to generate a heat map;

[0075] A classification module 304, for classifying each pixel point in the thermal map based on the classification function of the aerospace vehicle attitude recognition model, and determining the probability of each structural point in the thermal map;

[0076] A determination module 305, configured to determine a target structural point and a coordinate position of the target structural point based on the probability of each structural point in the thermal map and a preset probability threshold;

[0077] The compensation module 306 is used to perform coordinate compensation on the coordinate position of the target structure point to obtain the target coordinate position corresponding to the target spacecraft.

[0078] Optionally, the device also includes: an acquisition module, used to capture target pixel images of the target spacecraft in different motion scenes; a labeling module, used to obtain the coordinates of key external structure points corresponding to each target pixel image, and obtain image samples; an augmentation module, used to perform augmentation processing on the image samples, and obtain an image data set; a division module, used to divide the image data set into an image training set and an image test set; a training module, used to perform supervised learning on a pre-constructed convolutional neural network model using the image training set, and determine a loss value based on a preset loss function, and when the loss value is greater than or equal to a preset threshold, adjust the model parameters of the convolutional neural network model until the loss value is less than the threshold, and obtain the spacecraft posture recognition model.

[0079] Optionally, the device further includes: a testing module, used to input the image test set into the spacecraft attitude recognition model for testing, obtain test results, and adjust model parameters of the spacecraft attitude recognition model based on the test results.

[0080] Optionally, the device also includes: an acquisition and deletion module, used to obtain a ResNet model, and delete the output layer in the ResNet model to obtain a backbone network; an adjustment and superposition module, used to adjust the upsampling step size of the last module of the ResNet50 model, and superimpose a feature pyramid FPN network on the backbone network to obtain the pre-constructed convolutional neural network model.

[0081] Optionally, the determination module 305 is specifically used to: compare the probability of each structure point with a preset probability threshold to obtain a target area with a probability greater than the probability threshold; use the structure point corresponding to the highest probability in the target area as the target structure point, and obtain the corresponding coordinate information of the target structure point as the coordinate position of the target structure point.

[0082] Optionally, the compensation module 306 is specifically used to: perform deconvolution on the feature map to obtain a coordinate compensation value of the target structure point corresponding to the original image coordinates; map the coordinate position of the target structure point to the original image to obtain a mapping coordinate value; generate the center coordinates of the target structure point based on the mapping coordinate value and the coordinate compensation value; and add the coordinate compensation value for the edge position of the target structure point generated by the FPN network in the aerospace vehicle attitude recognition model to the center coordinates of the target structure point to obtain the target coordinate position corresponding to the target aerospace vehicle.

[0083] Optionally, the device also includes: an installation module for installing a preset software environment on the target device; an upload module for uploading the file corresponding to the spacecraft attitude recognition model to the target device; a writing module for writing an interface program to receive the image to be recognized to call the spacecraft attitude recognition model for attitude recognition, obtain the target coordinate position corresponding to the target spacecraft, and send the target coordinate position corresponding to the target spacecraft to the client or store it in a database.

[0084] The aerospace vehicle attitude recognition device based on convolutional neural network provided in the embodiments of the present disclosure can execute the aerospace vehicle attitude recognition method based on convolutional neural network provided in any embodiment of the present disclosure, and has the functional modules and beneficial effects corresponding to the execution method.

[0085] The embodiments of the present disclosure also provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the aerospace vehicle attitude recognition method based on a convolutional neural network provided by any embodiment of the present disclosure.

[0086] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0087] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0088] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0089] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains an image to be identified; processes the image to be identified based on a pre-trained aerospace vehicle attitude recognition model to obtain a feature map, and deconvolves the feature map to generate a thermal map; classifies each pixel point in the thermal map based on the classification function of the aerospace vehicle attitude recognition model, and determines the probability of each structure point in the thermal map; determines the target structure point and the coordinate position of the target structure point based on the probability of each structure point in the thermal map and a preset probability threshold, and coordinate compensates the coordinate position of the target structure point to obtain the target coordinate position corresponding to the target aerospace vehicle.

[0090] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0091] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0092] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0093] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0094] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0095] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:

[0096] processor;

[0097] a memory for storing instructions executable by the processor;

[0098] The processor is used to read the executable instructions from the memory and execute the instructions to implement any aerospace vehicle attitude recognition method based on a convolutional neural network as provided in the present disclosure.

[0099] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute any of the convolutional neural network-based aerospace vehicle attitude recognition methods provided by the present disclosure.

[0100] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0101] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0102] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A method for aerospace vehicle attitude recognition based on convolutional neural network, characterized in that: The method comprises: Obtain an image to be recognized; Processing the image to be recognized based on a pre-trained aerospace vehicle attitude recognition model to obtain a feature map, and performing deconvolution on the feature map to generate a thermal map; Classifying each pixel point in the thermal map based on the classification function of the aerospace vehicle attitude recognition model to determine the probability of each structural point in the thermal map; The target structure point and the coordinate position of the target structure point are determined based on the probability of each structure point in the thermal map and a preset probability threshold, and the coordinate position of the target structure point is coordinate compensated to obtain the target coordinate position corresponding to the target spacecraft.

2. The method for aerospace vehicle attitude recognition based on convolutional neural network according to claim 1, characterized in that: The method further comprises: Taking target pixel images of the target spacecraft in different motion scenes, and obtaining the coordinates of key external structure points corresponding to each target pixel image, to obtain image samples; Performing augmentation processing on the image samples to obtain an image data set; Dividing the image data set into an image training set and an image test set; The image training set is used to perform supervised learning on a pre-built convolutional neural network model, and a loss value is determined based on a preset loss function. When the loss value is greater than or equal to a preset threshold, the model parameters of the convolutional neural network model are adjusted until the loss value is less than the threshold, thereby obtaining the aerospace vehicle attitude recognition model.

3. The method for aerospace vehicle attitude recognition based on convolutional neural network according to claim 2, characterized in that: The method further comprises: The image test set is input into the aerospace vehicle attitude recognition model for testing to obtain a test result, and the model parameters of the aerospace vehicle attitude recognition model are adjusted based on the test result.

4. The method for aerospace vehicle attitude recognition based on convolutional neural network according to claim 2, characterized in that: The method further comprises: Obtain a ResNet model, and delete the output layer in the ResNet model to obtain a backbone network; The upsampling step size of the last module of the ResNet model is adjusted, and a feature pyramid FPN network is superimposed on the backbone network to obtain the pre-built convolutional neural network model.

5. The method for aerospace vehicle attitude recognition based on convolutional neural network according to claim 1, characterized in that: The determining of the target structural point and the coordinate position of the target structural point based on the probability of each structural point in the thermal map and a preset probability threshold comprises: Compare the probability of each structure point with a preset probability threshold, and obtain a target area with a probability greater than the probability threshold; The structure point corresponding to the highest probability in the target area is used as the target structure point, and the coordinate information corresponding to the target structure point is obtained as the coordinate position of the target structure point.

6. The method for aerospace vehicle attitude recognition based on convolutional neural network according to claim 1, characterized in that: The coordinate compensation of the coordinate position of the target structure point to obtain the target coordinate position corresponding to the target aerospace vehicle includes: Deconvolve the feature map to obtain coordinate compensation values ​​of the target structure points corresponding to the original image coordinates; Mapping the coordinate position of the target structure point to the original image to obtain a mapping coordinate value; Generate the center coordinates of the target structure point based on the mapping coordinate value and the coordinate compensation value; The coordinate compensation value of the edge position of the target structure point generated by the FPN network in the aerospace vehicle attitude recognition model is added to the center coordinate of the target structure point to obtain the target coordinate position corresponding to the target aerospace vehicle.

7. The method for aerospace vehicle attitude recognition based on convolutional neural network according to claim 2, characterized in that: The method further comprises: Install the preset software environment on the target device; Uploading the file corresponding to the aerospace vehicle attitude recognition model to the target device; An interface program is written to receive the image to be identified to call the spacecraft attitude recognition model for attitude recognition, obtain the target coordinate position corresponding to the target spacecraft, and send the target coordinate position corresponding to the target spacecraft to the client or store it in a database.

8. A spacecraft attitude recognition device based on convolutional neural network, characterized in that: include: An acquisition module, used for acquiring an image to be recognized; A processing module, used for processing the image to be recognized based on a pre-trained aerospace vehicle attitude recognition model to obtain a feature map; A generation module, used for performing deconvolution on the feature map to generate a heat map; A classification module, used to classify each pixel point in the thermal map based on the classification function of the aerospace vehicle attitude recognition model, and determine the probability of each structural point in the thermal map; A determination module, used to determine a target structural point and a coordinate position of the target structural point based on the probability of each structural point in the thermal map and a preset probability threshold; The compensation module is used to perform coordinate compensation on the coordinate position of the target structure point to obtain the target coordinate position corresponding to the target aerospace vehicle.

9. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the aerospace vehicle attitude recognition method based on convolutional neural network as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the aerospace vehicle attitude recognition method based on convolutional neural network described in claims 1-7.

Images