A method for detecting space debris in large field-of-view images based on on-orbit adaptation

Abstract

The present invention discloses a method for detecting space debris in a large field of view image based on on-orbit adaptation, including: acquiring a large field of view starry sky image to be detected, inputting the large field of view starry sky image into a trained space debris detection model to obtain a target detection result for the large field of view starry sky image; the space debris detection model includes a coupling factor evaluation module, a sparse mixture of experts processing module, a knowledge precise correction module, and a detection result output module; through the collaborative work of multiple modules, the present invention uses various image degradation evaluation indicators at different levels to control expert models in different fields to effectively respond to different factors, and at the same time introduces a knowledge precise correction module to achieve more refined adaptive processing. The present invention significantly improves the target detection performance of large field of view detection images in complex space environments.

Description

Technical Field

[0001] The present invention relates to the fields of space target detection and deep learning, and particularly to a method for detecting space debris in large field-of-view images based on on-orbit adaptation. Background Art

[0002] In recent years, due to the higher information acquisition efficiency of large field-of-view cameras, they have gradually been introduced into the field of space debris monitoring. However, the complexity and diversity of the space environment make large field-of-view cameras with complex hardware designs and stronger sensing capabilities more vulnerable to influence during observation, resulting in different degrees and types of degradation and interference in their data. Traditional single-scene processing models often cannot effectively detect targets in large field-of-view data under different scenarios.

[0003] Reference 1 proposed a method for processing visual tasks based on a mixture of experts model, replacing some dense feedforward layers in the visual self-attention network with sparse mixture of experts layers, and each image patch is only processed by some of the expert layers to adaptively process image regions in different states. This type of method can flexibly make optimal processing decisions for regions in different states of the image by setting multiple "experts" for data in different states, but there are still some drawbacks to this type of method: First, this type of method uses the TOP-k strategy under the control of a single evaluation score for routing, resulting in poor routing accuracy and being unable to handle large field-of-view observation data affected by complex factors; at the same time, the number of "expert" layers in this method is limited, and it cannot obtain optimal detection results under various degradation interferences. Therefore, many visual mixture of experts model methods, including this method, often cannot meet the application requirements for detecting space debris in large field-of-view detection data in complex space environments; Reference 1 is as follows:

[0004] "Riquelme C, Puigcerver J, Mustafa B, et al. Scaling vision with sparse mixture of experts[J]. Advances in Neural Information Processing Systems, 2021, 34: 8583-8595." Summary of the Invention

[0005] The purpose of the present invention is to provide a method for detecting space debris in large field-of-view images based on on-orbit adaptation, and by establishing the selection and parameter correction of a multi-expert detection model guided by image quality indicators, to achieve adaptive object detection of large field-of-view detection data affected by complex factors.

[0006] To achieve the above task, the present invention adopts the following technical solutions:

[0007] A method for detecting space debris in large field-of-view images based on on-orbit adaptation, including:

[0008] Obtain a large field-of-view starry sky image to be detected, input the large field-of-view starry sky image into a trained space debris detection model, and obtain a target detection result for the large field-of-view starry sky image;

[0009] The space debris detection model includes a coupling factor evaluation module, a sparse mixture of experts processing module, a knowledge precise correction module, and a detection result output module, where:

[0010] The coupling factor evaluation module is used to perform preliminary image feature extraction on the large field-of-view starry sky image and calculate different image degradation evaluation indexes;

[0011] The sparse mixture of experts processing module uses the image degradation evaluation indexes to calculate activation weights through a gating network, thereby dynamically activating the corresponding expert models; correct the model parameters of the activated expert models and perform weighted combination, and use the weighted combination of expert models to extract feature maps of the preliminary image features;

[0012] The knowledge precise correction module is based on the image degradation evaluation indexes and uses a hypernetwork to output correction amounts of the model parameters of different expert models, thereby adjusting the model parameters of the expert models;

[0013] The detection result output module is used to perform target detection on the feature maps extracted by the weighted combination of expert models and output the target detection result.

[0014] Further, the different image degradation evaluation indexes include background noise, energy concentration, optical distortion degree, target density, target motion characteristics, and detection ability; where:

[0015] Background noise is used to measure the intensity and distribution of background noise in the large field-of-view starry sky image, and is obtained by calculating the standard deviation of the background noise in the large field-of-view starry sky image;

[0016] Target density is used to describe the distribution of targets in the large field-of-view starry sky image, and is obtained by calculating the number and coverage area of targets in the large field-of-view starry sky image;

[0017] The optical distortion degree is used to describe the actual distortion degree in the large field-of-view starry sky image, and is obtained by calculating the radial distance of pixels in the large field-of-view starry sky image and their distorted radial distances, and the radial distance of the pixel under ideal conditions;

[0018] The detection ability is the apparent magnitude of the lowest mean signal-to-noise ratio target that can be detected in the large field-of-view starry sky image;

[0019] The energy concentration is the average value of the energy concentrations of all light spots in the large field-of-view starry sky image;

[0020] The target motion feature is the target pull line length in the large field of view starry sky image.

[0021] Further, the coupling factor evaluation module includes a feature extraction encoder; the feature extraction encoder includes a convolutional layer, a max pooling layer, and a combination of a series of identity residual blocks and downsampling residual blocks;

[0022] When performing preliminary image feature extraction on the large field of view starry sky image, first use the convolutional layer to extract the feature map of the large field of view starry sky image and extract the number of channels, then reduce the resolution of the obtained feature map to half of the original through the max pooling layer, and then perform further feature extraction through the deep network composed of the combination of identity residual blocks and downsampling residual blocks. Among them, the number of output channels and the resolution of the output features of the identity residual block do not change relative to the number of input channels and the resolution of the input features, and the number of output channels of the downsampling residual block is twice the number of input channels, and the resolution of the output features is half of the resolution of the input features; the fusion of the features extracted by the identity residual block and the downsampling residual block is realized through three adjacent layer feature fusion modules to obtain the preliminary image features.

[0023] Further, the feature extraction encoder sequentially includes a convolutional layer, a max pooling layer, a first identity residual block, a second identity residual block, a first downsampling residual block, a third identity residual block, a second downsampling residual block, a fourth identity residual block, a third downsampling residual block, and a fifth identity residual block from front to back;

[0024] The structures of the three adjacent layer feature fusion modules are the same, and are respectively AFFM1, AFFM2, and AFFM3; where:

[0025] The inputs of AFFM1 are the feature F2_2 output by the first identity residual block, the feature F2_3 output by the second identity residual block, the feature F3_1 output by the first downsampling residual block, and the feature F3_2 output by the third identity residual block. Its specific processing process is: after the features F2_2 and F2_3 are fused, they pass through a convolutional unit, BN+ReLU processing, and then are fused with the fusion result of the features F3_1 and F3_2 again. After the fused result passes through a convolutional unit, BN+ReLU, and a pooling unit for processing, the output feature is ;

[0026] The inputs of AFFM2 are the feature F3_1 output by the first downsampling residual block, the feature F3_2 output by the third identity residual block, the feature F4_1 output by the second downsampling residual block, and the feature F4_2 output by the fourth identity residual block; the output feature of AFFM2 is ;

[0027] The inputs of AFFM3 are the feature F4_1 output by the second downsampling residual block, the feature F4_2 output by the fourth identity residual block, the feature F5_1 output by the third downsampling residual block, and the feature F5_2 output by the fifth identity residual block; the output feature of AFFM2 is ;

[0028] After obtaining the features , and , a concatenation fusion operation is performed with the feature F5_2 output by the fifth identity residual block, and finally the preliminary image feature f is output.

[0029] Furthermore, the sparse mixture of experts processing module uses the image degradation evaluation metric to calculate the activation weights through a gating network, including:

[0030] The gating network is a multi-layer perceptron with an input layer, two hidden layers, and an output layer;

[0031] The input layer of the gating network inputs the metric vector composed of all image degradation evaluation metrics, and this metric vector is passed to the subsequent hidden layer; after the two hidden layers sequentially perform feature extraction and activation function processing on the metric vector, the activation weights of the expert model are finally generated in the fully connected layer serving as the output layer.

[0032] Furthermore, the model parameters of the activated expert model are corrected and weighted combined, and the weighted combined expert model is used to extract the feature map of the preliminary image feature, including:

[0033] The structure of each expert model is the same, which is a two-layer convolutional network, but the model parameters of different expert models are different, and the model parameters are the combination of the weights and biases of all layers of the expert model; the first layer of each expert model is a convolutional layer, and the second layer is a global pooling layer;

[0034] According to the activation weights of the expert model output by the gating network, the top preset number of activation weights are selected after sorting all the activation weights;

[0035] According to whether the activation weights are selected, the activation indicator variables corresponding to each expert model are updated;

[0036] The weighted combination of the expert models is performed using the updated activation indicator variables, the features extracted by each expert model from the preliminary image feature, and the activation weights, and the feature map extracted from the preliminary image feature is output using the weighted combined expert model.

[0037] Furthermore, the knowledge precise correction module is based on the image degradation evaluation metric, and uses a hypernetwork to output the correction amounts of the model parameters of different expert models, thereby adjusting the model parameters of the expert model, including:

[0038] The super network is a multi-layer perceptron with two fully connected layers. Its input is an image degradation evaluation metric. The image degradation evaluation metric is processed by the two fully connected layers of the super network to obtain a feature representation of a fixed size. Subsequently, after layer normalization, a shared feature is obtained. After restricting the output range through an activation function, the output is restricted to an amplitude range using the model parameters of the expert model and the shared feature, thereby obtaining a correction amount with the same dimension as the model parameters of the expert model. The output correction amount will be added to the original model parameters of the expert model, thereby realizing the process of correcting the model parameters of the expert model.

[0039] Furthermore, the detection result output module adopts the detection head of YOLOX, takes the feature map extracted by the expert model after weighted combination output by the sparse mixture of experts processing module as the input, and uses the detection head to output the object detection results, including object classification results, bounding box regression, and object confidence prediction results.

[0040] Furthermore, the overall loss function of the space debris detection model during training is:

[0041] ;

[0042] where is the loss of various detection performances after the feature map extracted by the expert model is input into the detection head, including the classification loss, bounding box regression loss, and confidence loss of object detection; is a learnable parameter, is the correction loss of the expert model, expressed as:

[0043] ;

[0044] where, is the weight coefficient of the regularization term, represents the model parameters of the k-th expert model, is the model parameters of the corrected expert model.

[0045] A large field-of-view image space debris detection device based on on-orbit adaptation, comprising:

[0046] An image acquisition unit for acquiring a large field-of-view starry sky image to be detected;

[0047] An image space debris detection unit for receiving the large field-of-view starry sky image and inputting the large field-of-view starry sky image into the trained space debris detection model to obtain an object detection result for the large field-of-view starry sky image;

[0048] The space debris detection model includes a coupling factor evaluation module, a sparse mixture of experts processing module, a knowledge precise correction module, and a detection result output module, where:

[0049] The coupling factor evaluation module is used to perform preliminary image feature extraction on the large field of view starry sky image and calculate different image degradation evaluation metrics;

[0050] The sparse mixture of experts processing module uses the image degradation evaluation metrics to calculate activation weights through a gating network, thereby dynamically activating the corresponding expert models; correct the model parameters of the activated expert models and perform weighted combination, and use the weighted combined expert models to extract feature maps of the preliminary image features;

[0051] The knowledge precise correction module, based on the image degradation evaluation metrics, uses a hypernetwork to output the correction amounts of the model parameters of different expert models, thereby adjusting the model parameters of the expert models;

[0052] The detection result output module is used to perform object detection on the feature maps extracted by the weighted combined expert models and output the object detection results.

[0053] A terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method for detecting space debris in large field of view images based on on-orbit adaptation.

[0054] A computer-readable storage medium stores a computer program; when the computer program is executed by a processor, it implements the method for detecting space debris in large field of view images based on on-orbit adaptation.

[0055] Compared with the prior art, the present invention has the following technical features:

[0056] The present invention proposes a method for detecting space debris in large field-of-view images based on on-orbit adaptation, which uses a variety of image degradation evaluation indicators at different levels to control expert models in different fields to effectively respond to different factors. At the same time, a knowledge precise correction module is introduced to achieve more refined adaptive processing. A fusion feature extraction model is designed, which combines multiple known image evaluation indicators to obtain the macro-state features of image data containing semantic information. A multi-level and different-degree sparse expert model is designed, and each expert model processes different levels of degradation interference. The gating network dynamically selects a suitable expert model for processing according to the input evaluation indicators, thereby reducing the computational cost and improving the detection accuracy. A super network is designed, which uses multiple evaluation indicators generated by the macro feature extraction encoder to automatically adjust the parameters of the expert model. The super network dynamically generates a correction amount according to these indicators to correct the weights of the expert model, making it more accurately adapt to the features and quality of the current image. Finally, a detection head extracts target information from the feature map corrected by the expert model and performs target detection in the detection image. Through efficient classification, regression, and confidence prediction, the detection head can achieve high-precision detection in complex backgrounds and achieve on-orbit adaptation. Through the collaborative work of the above-mentioned multiple modules, the present invention significantly improves the target detection performance of large field-of-view detection images in complex space environments, including the target detection rate and the target detection positioning accuracy. Description of the Drawings

[0057] Figure 1 It is a schematic structural diagram of the space debris detection model in the present invention;

[0058] Figure 2 It is a schematic structural diagram of the feature extraction encoder in an embodiment of the present invention;

[0059] Figure 3 It is a schematic structural diagram of the downsampling residual block in an embodiment of the present invention;

[0060] Figure 4 It is a schematic structural diagram of the identity residual block in an embodiment of the present invention;

[0061] Figure 5 It is a schematic diagram of the AFFM1 processing process in an embodiment of the present invention;

[0062] Figure 6 It is a schematic diagram of the AFFM2 processing process in an embodiment of the present invention;

[0063] Figure 7 It is a schematic diagram of the AFFM3 processing process in an embodiment of the present invention;

[0064] Figure 8 It is the detection result of a large field-of-view starry sky image in an embodiment of the present invention. Detailed Embodiments

[0065] When observing objects such as space debris using a large field-of-view camera, it is often affected by internal factors of the camera such as optical distortion and dark current, as well as external factors such as high-energy particles and atmospheric refraction, resulting in varying degrees of degradation of the observed data, such as different degrees of background noise, optical distortion, etc. At the same time, due to the differences in the working modes of the camera turntable, there are also significant differences in the number distribution and motion characteristics of background targets. Multiple factors with different degrees of influence in different scenarios interact with each other, forming different levels of degradation and interference to the observed data, making it difficult for the limited "expert" layer under a single index to fully adapt, and thus unable to achieve the optimal detection effect.

[0066] The present invention provides a method for detecting space debris in large field-of-view images based on on-orbit adaptation, which uses diversified evaluation indicators to control expert modules in different fields to effectively respond to different factors, and at the same time introduces expert knowledge correction to achieve more refined adaptive processing.

[0067] See Appendix Figure 1 , a method for detecting space debris in large field-of-view images based on on-orbit adaptation provided by the present invention, includes:

[0068] Obtain a large field-of-view starry sky image to be detected, and input the large field-of-view starry sky image into a trained space debris detection model to obtain a target detection result for the large field-of-view starry sky image;

[0069] The space debris detection model includes a coupling factor evaluation module, a sparse mixture of experts processing module, a knowledge precise correction module, and a detection result output module, where:

[0070] The coupling factor evaluation module is used to perform preliminary image feature extraction on the large field-of-view starry sky image and calculate different image degradation evaluation indicators;

[0071] The sparse mixture of experts processing module uses the image degradation evaluation indicators to calculate activation weights through a gating network, thereby dynamically activating the corresponding expert models; correct the model parameters of the activated expert models and perform weighted combination, and use the weighted combined expert models to extract feature maps of the preliminary image features;

[0072] The knowledge precise correction module is based on the image degradation evaluation indicators and uses a hypernetwork to output correction amounts of the model parameters of different expert models, thereby adjusting the model parameters of the expert models;

[0073] The detection result output module is used to perform target detection on the feature maps extracted by the weighted combined expert models and output the target detection results.

[0074] The present invention constructs a space debris detection model. By introducing diverse image quality evaluation metrics, it more comprehensively describes the current data state, providing more accurate guidance for the selection of expert models. At the same time, by introducing expert knowledge correction, it achieves more accurate feature extraction and target detection with a limited number of models.

[0075] Based on the training process of the space debris detection model, the specific design of the space debris detection model will be further described in detail below.

[0076] 1. Coupling factor evaluation module.

[0077] In the training stage, the training samples of the coupling factor evaluation module are the large field of view starry sky image dataset; the samples in the large field of view starry sky image dataset are large field of view starry sky images affected by stray light, thermal pixels, and high-energy particle noise.

[0078] For the large field of view starry sky image samples in the large field of view starry sky image dataset , the fusion feature extraction model is used to perform preliminary image feature extraction on the samples to obtain the macroscopic state features of the image data containing semantic information ; at the same time, the image degradation evaluation metrics are calculated as the label information of the samples .

[0079] (1.1) Image degradation evaluation metrics.

[0080] There are six image degradation evaluation metrics in this solution, namely: background noise , energy concentration , optical distortion degree , target density , target motion feature (target pull line length), detection ability ; the representation method of each image degradation evaluation metric is as follows:

[0081] First, the spot and background noise parts of the samples can be preliminarily segmented by a simple binary threshold segmentation method; background noise measures the background noise intensity and distribution in the samples , and is obtained by calculating the standard deviation of the background noise in the samples :

[0082] (1);

[0083] Among them, is the intensity of the pixel at the position in the background area of the sample , is the sample The mean gray value of the background part in is the number of pixels in the background part; the greater the background noise, the greater the value of the background noise distribution.

[0084] Target density is used to describe the distribution of targets in the sample and is obtained by calculating the number of targets and the covered area in the sample as follows:

[0085] (2);

[0086] wherein, is the number of targets detected in the sample and is the total area of the sample image (unit: number of pixels).

[0087] Optical distortion degree describes the actual distortion degree in the sample as follows:

[0088] (3);

[0089] wherein, is the number of pixels of the sample image, is the radial distance after distortion of the th pixel in the image, and

[0090]

[0091] Detection ability is the apparent magnitude (apparent magnitude refers to the brightness of a star as seen by an observer with the naked eye) of the faintest (lowest mean signal-to-noise ratio) target that the current sample can detect, and the detection ability and the target motion characteristics are both known attributes of the sample or the large field of view starry sky image to be detected.

[0092] Note: For the th spot, the calculation method of its mean signal-to-noise ratio is as follows:

[0093] (4);

[0094] Among them, is the average gray value of the light spot.

[0095] Take the above six image degradation evaluation indicators as samples of the label information.

[0096] (1.2) Fusion feature extraction model.

[0097] The fusion feature extraction model includes a feature extraction encoder , through the feature extraction encoder to obtain a sample The preliminary image feature f containing the image semantic information; the feature extraction encoder is based on a residual network and realizes the full fusion of the extracted features through three adjacent layer feature fusion modules (AFFM); as Figure 2 shown, where:

[0098] The feature extraction encoder includes a convolutional layer, a max pooling layer, and a combination of a series of identity residual blocks (IR Block) and downsampling residual blocks (DR Block);

[0099] When performing preliminary image feature extraction on the sample , first use the convolutional layer to extract the feature map of the sample and extract the number of channels, then reduce the resolution of the obtained feature map to half of the original through the max pooling layer, and then perform further feature extraction through a deep network composed of a combination of identity residual blocks and downsampling residual blocks, where the number of output channels and the resolution of the output features of the identity residual block do not change relative to the number of input channels and the resolution of the input features, and the number of output channels of the downsampling residual block is twice the number of input channels, and the resolution of the output features is half of the resolution of the input features; through three adjacent layer feature fusion modules (AFFM), the full fusion of the features extracted by the identity residual block and the downsampling residual block is realized, and the preliminary image feature f is obtained.

[0100] As Figure 2 shown, in an embodiment of the present invention, the specific structure of the feature extraction encoder is:

[0101] Table 1: Structure of the feature extraction encoder Structure.

[0102]

[0103] Note: In the above table, Conv1…Conv20 represent different convolutional units, and the parameters of the convolutional units are in parentheses. See the appendix Figure 3 and Figure 4 。

[0104] In this embodiment, in order to make full use of the features at each level and better obtain semantic information at different levels, an adjacent layer feature fusion module (AFFM) is designed in the basic residual network; the structures of the three adjacent layer feature fusion modules are the same, namely AFFM1, AFFM2, and AFFM3; among them:

[0105] The inputs of AFFM1 are the feature F2_2 output by the first identity residual block, the feature F2_3 output by the second identity residual block, the feature F3_1 output by the first downsampling residual block, and the feature F3_2 output by the third identity residual block. Its specific processing process is as follows: As Figure 5 shown, after the feature F2_2 and the feature F2_3 are fused, they are processed by a convolutional unit and BN+ReLU, and then fused with the fusion result of the feature F3_1 and the feature F3_2 again. After the fused result is processed by a convolutional unit, BN+ReLU, and a pooling unit, the output feature is 。

[0106] The inputs of AFFM2 are the feature F3_1 output by the first downsampling residual block, the feature F3_2 output by the third identity residual block, the feature F4_1 output by the second downsampling residual block, and the feature F4_2 output by the fourth identity residual block. Its specific processing process is similar to that of AFFM1. See the appendix Figure 6 ,and will not be elaborated here; the output feature of AFFM2 is 。

[0107] The inputs of AFFM3 are the feature F4_1 output by the second downsampling residual block, the feature F4_2 output by the fourth identity residual block, the feature F5_1 output by the third downsampling residual block, and the feature F5_2 output by the fifth identity residual block. Its specific processing process is similar to that of AFFM1. See the appendix Figure 7 ,and will not be elaborated here; the output feature of AFFM2 is 。

[0108] After obtaining the features 、 and ,a concatenation fusion operation is performed with the feature F5_2 output by the fifth identity residual block, and finally the preliminary image feature f is output.

[0109] 2. Sparse mixture of experts processing module.

[0110] The coupling factor evaluation module outputs six different image degradation evaluation indicators, and the indicator vector formed by them is denoted as ; The core task of the sparse mixture of experts processing module is to select an appropriate expert model based on the gated network and the image degradation evaluation metrics from the coupling factor evaluation module, and to further precisely process the initial image features f after the parameter correction of the expert model by the knowledge precise correction module.

[0111] (2.1) Gated network.

[0112] The gated network is a multi-layer perceptron with an input layer, two hidden layers, and an output layer.

[0113] First is the input layer. The input layer of the gated network inputs an index vector composed of six image degradation evaluation metrics which is passed to the subsequent hidden layer.

[0114] Secondly is the hidden layer, with a total of two layers, implemented as follows:

[0115] (5);

[0116] Among them, is the output feature of the first hidden layer, is the activation function, , , is the weight matrix of the first hidden layer, is the bias term, is the number of hidden layer units in the gated network, is the real number space.

[0117] (6);

[0118] Among them, , , is the weight matrix of the second fully connected layer, is the bias term.

[0119] Finally is the output layer, which is implemented through a fully connected layer and is responsible for generating the activation weights of the expert model; it outputs a -dimensional expert activation vector , represents the total number of expert models, represents the activation probability of each expert model; the specific implementation is as follows:

[0120] (7);

[0121] Among them, is the activation function, , , is the weight matrix of the output layer, and is the bias term of the output layer; Moreover, each value represents the activation weight (activation probability) of the -th expert model.

[0122] (2.2) Expert models.

[0123] Each expert model has the same structure, which is a two-layer convolutional network. However, the model parameters of different expert models are different, manifested as the weights and biases of each network layer being different. Therefore, the features extracted by each expert model are different; the first layer of each expert model is a convolutional layer, and the second layer is a global pooling layer. It should be noted that when each expert model processes the preliminary image features f, the model parameters used are the results after being corrected by the knowledge precise correction module.

[0124] The convolutional layer of the first layer of each expert model performs convolutional processing on the preliminary image features f to learn local features:

[0125] (8);

[0126] Among them, represents two-dimensional convolution, is the convolution kernel of the convolutional layer, is the output of the convolutional layer, is the bias term.

[0127] Subsequently, the ReLU activation function is used to increase non-linearity:

[0128] (9);

[0129] Among them, is the result after the output of the convolutional layer passes through the activation function .

[0130] The second layer is a global pooling layer, and the features extracted by the k-th expert model are the results after global pooling processing, expressed as:

[0131] (10);

[0132] (2.3) Dynamic activation of expert models.

[0133] According to the expert activation vector output by the gating network, select the expert models that need to be activated for subsequent feature extraction; this part uses the Top-K strategy to select from Select the top expert models corresponding to the largest activation weights for activation; The Top-K selection steps are as follows:

[0134] First, sort the expert activation vectors in descending order to obtain the sorted index list That is:

[0135] (11);

[0136] Among them, is the sorted index of the activation weight .

[0137] Subsequently, select the top K activation weights from the activation weights ; At this time, update all activation indicator variables corresponding to the expert models :

[0138] (12);

[0139] Among them, is the index corresponding to the activation weight .

[0140] According to the indication information of , the corresponding expert model can be activated; that is, when the value of is 1, the corresponding k-th expert model will be activated.

[0141] (2.4) Expert models after weighted combination.

[0142] Finally, the K activated expert models are weighted combined, and the sparse mixture of experts uses the output of the expert models after weighted combination to extract the feature map of the preliminary image features which is expressed as follows:

[0143] (13).

[0144] 3. Knowledge precise correction module.

[0145] The knowledge precise correction module introduces an expert knowledge correction strategy to correct the model parameters of the selected expert models through a hypernetwork, so as to adjust the model parameters of different expert models, so that each activated expert model can more precisely adapt to the specific features of the current feature map and better cope with different degradation conditions.

[0146] The model parameters of each expert model are the combination of the weights and biases of all its layers; the hypernetwork is a multi-layer perceptron with two fully connected layers, and its input is the image degradation evaluation index , and the image degradation evaluation index is processed by the two fully connected layers of the hypernetwork to obtain a feature representation of a fixed size. Subsequently, after layer normalization, the shared feature is obtained. After restricting the output range through the activation function, the output is restricted to an amplitude range using the model parameters of the expert model and the shared feature, thereby outputting a correction amount with the same dimension as the model parameters of the expert model ; the output correction amount will be added to the original model parameters of the expert model, thereby realizing the process of correcting the model parameters of the expert model; the specific description is as follows:

[0147] The index vector composed of the image degradation evaluation index extracts the shared feature through the two fully connected layers of the hypernetwork, which is expressed as follows:

[0148] (14);

[0149] (15);

[0150] Among them, are the output features of the first and second fully connected layers of the hypernetwork respectively, are the weights and biases of the first fully connected layer in the hypernetwork respectively, are the weights and biases of the second fully connected layer respectively.

[0151] Using layer normalization processing d to obtain the shared feature :

[0152] (16);

[0153] Finally, the output range is restricted by the activation function tanh, and then the output is scaled to a suitable amplitude range (multiplied by a preset small coefficient ), to prevent the parameters from changing too much, and the specific implementation is as follows:

[0154] Then the output correction amount is:

[0155] (17);

[0156] Among them, is a learnable scaling factor, and its initial value is 0.01; = represents the model parameters of the k-th expert model, and is the weight combination and bias combination of all network layers of the k-th expert model; represents the correction amount corresponding to the k-th expert model.

[0157] Finally, the model parameters of the corrected expert model are expressed as follows:

[0158] (18);

[0159] Using the model parameters to correct the original model parameters of the expert model and then performing feature extraction of the preliminary image features f by the expert model.

[0160] 4. Detection result output module.

[0161] The detection result output module uses the detection head of YOLOX, and takes the feature map extracted by the weighted combined expert model output by the sparse mixture of experts processing module as the input, and uses the detection head to output the object detection results, including the object classification result, bounding box regression, and object confidence prediction result.

[0162] The network structure of the detection head is two convolutional layers and an output layer, and the specific design is as follows:

[0163] First is the first convolutional layer, and the output result is:

[0164] (19);

[0165] Among them, is the activation function, represents two-dimensional convolution, 、 are the convolution kernel and bias of the first convolutional layer.

[0166] Secondly is the second convolutional layer, and the output result is:

[0167] (20);

[0168] Among them, 、 are the convolution kernel and bias of the second convolutional layer.

[0169] Finally, the output layer of YOLOX will generate the relevant parameters of each detection box based on the convolution operation, including the object classification result (category score, determining whether it is an object or background), bounding box regression, and object confidence prediction result.

[0170] 5. Loss function for training the space debris detection model.

[0171] Since the output of the super network in the knowledge precise correction module affects the input features of the detection head, the overall loss function of the space debris detection model during training includes the correction loss of the expert model and the object detection loss of the detection head.

[0172] (5.1) Object detection loss of the detection head.

[0173] This part of the loss is the various detection performance losses after inputting the feature map extracted by the expert model into the detection head:

[0174] (21);

[0175] Among them is the object detection loss of the detection head.

[0176] is the classification loss of object detection, that is, using cross-entropy loss to calculate the difference between the class prediction and the true class of

[0177] detection boxes:

[0178] In the above formula, represents the true class of the object corresponding to the i-th detection box, is the predicted result of the object confidence by the detection head.

[0179] is the bounding box regression loss, that is, using the SmoothL1Loss loss function to regress the ( is the coordinate of the center point of the detection box, is the width and height of the detection box) parameters to ensure that the detection box is close to the true bounding box:

[0180] (23);

[0181] Among them, represents the i-th detection box predicted by the detection head, is the true i-th bounding box.

[0182] is the confidence loss, that is, using binary cross-entropy loss to calculate the error of the object confidence prediction result:

[0183] (24);

[0184] Among them, Indicates the presence marker of the target within the i-th detection box. Is the predicted result of the target confidence level output by the detection head.

[0185] (5.2) The correction loss of the expert model.

[0186] To control the scale of the expert model correction, an L2 regularization loss is added to the loss function to prevent overcorrection:

[0187] (25);

[0188] Among them, Is the correction loss of the expert model, Is the weight coefficient of the regularization term, used to control the scale of the correction, Represents the model parameters of the k-th expert model.

[0189] Therefore, the overall loss function of the space debris detection model during training Is expressed as:

[0190] (26);

[0191] Among them, Are learnable parameters.

[0192] Use the samples in the large field of view starry sky image dataset to train the space debris detection model until the overall loss function Converges or reaches the maximum number of training times, then the training is completed, and the trained space debris detection model is saved.

[0193] In actual application, for a large field of view starry sky image with unknown target categories, input it into the trained space debris detection model to obtain the target detection result. In an embodiment of the present invention, the detection result of the star target in a certain large field of view starry sky image is as Figure 8 Shown.

[0194] The method proposed by the present invention realizes more comprehensive and accurate data processing on the premise of effectively limiting the network scale, and achieves excellent target detection results in the large field of view detection dataset in a complex space environment.

[0195] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for detecting space debris based on large field of view images based on on-orbit adaptation, characterized in that: include: Obtain a large-field-of-view starry sky image to be detected, input the large-field-of-view starry sky image into a trained space debris detection model, and obtain a target detection result for the large-field-of-view starry sky image; The space debris detection model includes a coupling factor evaluation module, a sparse mixed expert processing module, a knowledge precision correction module and a detection result output module, wherein: The coupling factor evaluation module is used to perform preliminary image feature extraction and calculation of different image degradation evaluation indicators for large-field starry sky images; The sparse hybrid expert processing module uses the image degradation assessment index to calculate the activation weight through the gated network, thereby dynamically activating the corresponding expert model; the model parameters of the activated expert model are modified and weighted combined, and the weighted combined expert model is used to extract the feature map of the preliminary image features; The knowledge accurate correction module uses a hypernetwork to output correction amounts of model parameters of different expert models based on the image degradation assessment index, thereby adjusting the model parameters of the expert model; The detection result output module is used to perform target detection on the feature map extracted by the weighted combined expert model and output the target detection result.

2. The method for detecting space debris based on large field of view images based on on-orbit adaptation according to claim 1, characterized in that: The different image degradation assessment indicators include background noise, energy concentration, optical distortion, target density, target motion characteristics and detection capability; wherein: Background noise is used to measure the intensity and distribution of background noise in large-field starry sky images, and is obtained by calculating the standard deviation of background noise in large-field starry sky images; Target density is used to describe the distribution of targets in a large-field-of-view starry sky image. It is obtained by calculating the number and coverage area of ​​targets in the large-field-of-view starry sky image. The degree of optical distortion is used to describe the actual degree of distortion in a wide-field star image. It is obtained by calculating the radial distance of pixels in a wide-field star image and their distortion, and the radial distance of the pixel under ideal conditions. The detection capability is the apparent magnitude of the target with the lowest mean signal-to-noise ratio that can be detected in a wide-field star image; The energy concentration is the average value of the energy concentration of all light spots in the large field of view starry sky image; The target motion feature is the target pull line length in the wide field of view starry sky image.

3. The method for detecting space debris based on large field of view images based on on-orbit adaptation according to claim 1, characterized in that: The coupling factor assessment module includes a feature extraction encoder; The feature extraction encoder includes a convolutional layer, a maximum pooling layer, and a series of identical residual blocks and down-sampling residual blocks; When performing preliminary image feature extraction on a wide-field starry sky image, a convolutional layer is first used to extract a feature map of the wide-field starry sky image and the number of channels, and then the resolution of the feature map is reduced to half of the original through a maximum pooling layer, and then further feature extraction is performed through a deep network composed of an identical residual block and a down-sampling residual block, wherein the number of output channels and the resolution of the output features of the identical residual block are unchanged relative to the number of input channels and the resolution of the input features, the number of output channels of the down-sampling residual block is twice the number of input channels, and the resolution of the output features is half the resolution of the input features; three adjacent layer feature fusion modules are used to realize the fusion of features extracted by the identical residual block and the down-sampling residual block to obtain the preliminary image features.

4. The method for detecting space debris based on large field of view images based on on-orbit adaptation according to claim 1, characterized in that: The sparse hybrid expert processing module uses the image degradation assessment index to calculate the activation weight through the gated network, including: The gating network is a multi-layer perceptron having an input layer, two hidden layers, and an output layer; The input layer of the gating network inputs an indicator vector consisting of all image degradation evaluation indicators, which is passed to the next hidden layer; the two hidden layers extract features and perform activation function processing on the indicator vector in turn, and finally generate the activation weights of the expert model in the fully connected layer as the output layer.

5. The method for detecting space debris based on large field of view images based on on-orbit adaptation according to claim 1, characterized in that: Modifying and weighting the model parameters of the activated expert model, and extracting the feature map of the preliminary image features using the expert model after weighting combination, including: The structure of each expert model is the same, which is a two-layer convolutional network, but the model parameters of different expert models are different. The model parameters are the combination of weights and biases of all layers of the expert model. The first layer of each expert model is a convolutional layer, and the second layer is a global pooling layer. According to the activation weights of the expert model output by the gating network, all activation weights are sorted and then a preset number of activation weights are selected; According to whether the activation weight is selected, the activation indicator variable corresponding to each expert model is updated; The updated activation indicator variables, the features extracted by each expert model from the preliminary image features, and the activation weights are used to perform a weighted combination of the expert models, and the expert model after the weighted combination is used to output a feature map extracted from the preliminary image features.

6. The method for detecting space debris based on large field of view images based on on-orbit adaptation according to claim 1, characterized in that: The knowledge accurate correction module uses the hypernetwork to output the correction amount of the model parameters of different expert models based on the image degradation evaluation index, thereby adjusting the model parameters of the expert model, including: The hypernetwork is a multilayer perceptron with two fully connected layers, and its input is an image degradation assessment index. The image degradation assessment index is processed by two fully connected layers of the hypernetwork to obtain a feature representation of a fixed size, and then a shared feature is obtained after layer normalization processing. After limiting the output range through an activation function, the output is limited to an amplitude range using the model parameters and shared features of the expert model, thereby obtaining a correction amount with the same dimension as the model parameters of the expert model; the output correction amount will be added to the original model parameters of the expert model, thereby realizing the model parameter correction process of the expert model.

7. The method for detecting space debris based on large field of view images based on on-orbit adaptation according to claim 1, characterized in that: The detection result output module adopts the detection head of YOLOX, takes the feature map extracted by the weighted combined expert model output by the sparse hybrid expert processing module as input, and uses the detection head to output the target detection results, including target classification results, bounding box regression, and target confidence prediction results.

8. The method for detecting space debris using large field of view images based on on-orbit adaptation according to claim 1, characterized in that: The overall loss function of the space debris detection model during training is for: ； in Various detection performance losses after the feature maps extracted by the expert model are input into the detection head, including classification loss, bounding box regression loss, and confidence loss of target detection; is a learnable parameter, is the corrected loss of the expert model, expressed as: ； in, is the weight coefficient of the regularization term, represents the model parameters of the k-th expert model, are the model parameters of the modified expert model.

9. A large field of view image space debris detection device based on on-orbit adaptation, characterized in that: include: An image acquisition unit, used to acquire a large-field starry sky image to be detected; An image space debris detection unit is used to receive a large-field-of-view starry sky image and input the large-field-of-view starry sky image into a trained space debris detection model to obtain a target detection result for the large-field-of-view starry sky image; The space debris detection model includes a coupling factor evaluation module, a sparse mixed expert processing module, a knowledge precision correction module and a detection result output module, wherein: The coupling factor evaluation module is used to perform preliminary image feature extraction and calculation of different image degradation evaluation indicators for large-field starry sky images; The sparse hybrid expert processing module uses the image degradation assessment index to calculate the activation weight through the gated network, thereby dynamically activating the corresponding expert model; the model parameters of the activated expert model are modified and weighted combined, and the weighted combined expert model is used to extract the feature map of the preliminary image features; The knowledge accurate correction module uses a hypernetwork to output correction amounts of model parameters of different expert models based on the image degradation assessment index, thereby adjusting the model parameters of the expert model; The detection result output module is used to perform target detection on the feature map extracted by the weighted combined expert model and output the target detection result.

10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor; characterized in that: When the processor executes the computer program, the method for detecting space debris with a large field of view image based on on-orbit adaptation according to any one of claims 1 to 8 is implemented.

Images