Large-field-of-view image space debris detection method based on in-orbit adaptation

Abstract

The invention discloses a large-view-field image space debris detection method based on in-orbit adaptation, and the method comprises the steps: obtaining a to-be-detected large-view-field starry sky image, inputting the large-view-field starry sky image into a trained space debris detection model, and obtaining a target detection result for the large-view-field starry sky image; the space debris detection model comprises a coupling factor evaluation module, a sparse hybrid expert processing module, a knowledge accurate correction module and a detection result output module; according to the invention, through cooperative work of a plurality of modules, expert models in different fields are controlled by using a plurality of different levels of image degradation evaluation indexes so as to realize effective response to different factors, and meanwhile, a knowledge accurate correction module is introduced so as to realize finer adaptive processing. According to the method, the target detection performance of the large-view-field detection image in a complex space environment is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the fields of space target detection and deep learning, and in particular to a method for detecting space debris using large field-of-view images based on on-orbit adaptation. Background Art

[0002] In recent years, large-field-of-view cameras have been gradually introduced into the field of space debris monitoring due to their higher information acquisition efficiency. However, the complexity and diversity of the space environment make large-field-of-view cameras with complex hardware design and stronger perception capabilities more susceptible to influence during observation, resulting in different degrees and types of degradation and interference in their data. Traditional single-scene processing modes often cannot achieve effective target detection for large-field-of-view data in different scenes.

[0003] Reference 1 proposes a visual task processing method based on a hybrid expert model, which replaces some dense feedforward layers in the visual self-attention network with sparse hybrid expert layers. Each image block is processed by only some expert layers, so as to adaptively process image regions with different states. This type of method flexibly makes optimal processing decisions for regions with different states of the image by setting multiple "experts" for different state data, but this type of method still has some disadvantages: First, this type of method uses the TOP-k strategy under the control of a single evaluation score for routing, resulting in poor routing accuracy and inability to cope with large field of view observation data affected by complex factors; at the same time, the number of "expert" layers in this method is limited, and it is impossible to obtain the best detection results under various degradation interferences. Therefore, many visual hybrid expert model methods, including this method, often cannot cope with the application requirements of spatial fragmentation detection of large field of view detection data in complex spatial environments; Reference 1 is as follows:

[0004] "Riquelme C, Puigcerver J, Mustafa B, et al. Scaling vision with sparse mixture of experts[J]. Advances in Neural Information ProcessingSystems, 2021, 34: 8583-8595." Summary of the invention

[0005] The purpose of the present invention is to provide a method for detecting spatial debris of large field of view images based on on-orbit adaptation, and to achieve adaptive object detection of large field of view detection data under the influence of complex factors by establishing a multi-expert detection model selection and parameter correction guided by image quality indicators.

[0006] In order to achieve the above tasks, the present invention adopts the following technical solutions:

[0007] A method for detecting space debris based on large field of view images based on on-orbit adaptation, comprising:

[0008] Obtain a large-field-of-view starry sky image to be detected, input the large-field-of-view starry sky image into a trained space debris detection model, and obtain a target detection result for the large-field-of-view starry sky image;

[0009] The space debris detection model includes a coupling factor evaluation module, a sparse mixed expert processing module, a knowledge precision correction module and a detection result output module, wherein:

[0010] The coupling factor evaluation module is used to perform preliminary image feature extraction and calculation of different image degradation evaluation indicators for large-field starry sky images;

[0011] The sparse hybrid expert processing module uses the image degradation assessment index to calculate the activation weight through the gated network, thereby dynamically activating the corresponding expert model; the model parameters of the activated expert model are modified and weighted combined, and the weighted combined expert model is used to extract the feature map of the preliminary image features;

[0012] The knowledge accurate correction module uses a hypernetwork to output correction amounts of model parameters of different expert models based on the image degradation assessment index, thereby adjusting the model parameters of the expert model;

[0013] The detection result output module is used to perform target detection on the feature map extracted by the weighted combined expert model and output the target detection result.

[0014] Furthermore, the different image degradation assessment indicators include background noise, energy concentration, optical distortion, target density, target motion characteristics and detection capability; wherein:

[0015] Background noise is used to measure the intensity and distribution of background noise in large-field starry sky images, and is obtained by calculating the standard deviation of background noise in large-field starry sky images;

[0016] Target density is used to describe the distribution of targets in a large-field-of-view starry sky image. It is obtained by calculating the number and coverage area of ​​targets in the large-field-of-view starry sky image.

[0017] The degree of optical distortion is used to describe the actual degree of distortion in a wide-field star image. It is obtained by calculating the radial distance of pixels in a wide-field star image and their distortion, and the radial distance of the pixel under ideal conditions.

[0018] The detection capability is the apparent magnitude of the target with the lowest mean signal-to-noise ratio that can be detected in a wide-field star image;

[0019] The energy concentration is the average value of the energy concentration of all light spots in the large field of view starry sky image;

[0020] The target motion feature is the target pull line length in the wide field of view starry sky image.

[0021] Furthermore, the coupling factor evaluation module includes a feature extraction encoder; the feature extraction encoder includes a convolution layer, a maximum pooling layer, and a series of combinations of an identical residual block and a down-sampling residual block;

[0022] When performing preliminary image feature extraction on a wide-field starry sky image, a convolutional layer is first used to extract a feature map of the wide-field starry sky image and the number of channels, and then the resolution of the feature map is reduced to half of the original through a maximum pooling layer, and then further feature extraction is performed through a deep network composed of an identical residual block and a down-sampling residual block, wherein the number of output channels and the resolution of the output features of the identical residual block are unchanged relative to the number of input channels and the resolution of the input features, the number of output channels of the down-sampling residual block is twice the number of input channels, and the resolution of the output features is half the resolution of the input features; three adjacent layer feature fusion modules are used to realize the fusion of features extracted by the identical residual block and the down-sampling residual block to obtain the preliminary image features.

[0023] Further, the feature extraction encoder includes, from front to back, a convolution layer, a maximum pooling layer, a first constant residual block, a second constant residual block, a first down-sampled residual block, a third constant residual block, a second down-sampled residual block, a fourth constant residual block, a third down-sampled residual block, and a fifth constant residual block;

[0024] The structures of the three adjacent layer feature fusion modules are the same, namely AFFM1, AFFM2 and AFFM3; wherein:

[0025] The input of AFFM1 is feature F2_2 output by the first identical residual block, feature F2_3 output by the second identical residual block, feature F3_1 output by the first down-sampling residual block, and feature F3_2 output by the third identical residual block. The specific processing process is as follows: feature F2_2 is fused with feature F2_3, and then processed by convolution unit and BN+ReLU, and then fused with the fusion result of feature F3_1 and feature F3_2 again. The fused result is processed by convolution unit, BN+ReLU and pooling unit, and the output feature is ;

[0026] The input of AFFM2 is the feature F3_1 output by the first down-sampled residual block, the feature F3_2 output by the third identical residual block, the feature F4_1 output by the second down-sampled residual block, and the feature F4_2 output by the fourth identical residual block; the output feature of AFFM2 is ;

[0027] The input of AFFM3 is the feature F4_1 output by the second down-sampled residual block, the feature F4_2 output by the fourth identical residual block, the feature F5_1 output by the third down-sampled residual block, and the feature F5_2 output by the fifth identical residual block; the output feature of AFFM2 is ;

[0028] In getting features , and Afterwards, a series fusion operation is performed with the feature F5_2 output by the fifth identical residual block, and finally the preliminary image feature f is output.

[0029] Furthermore, the sparse hybrid expert processing module uses the image degradation assessment index to calculate activation weights through a gated network, including:

[0030] The gating network is a multi-layer perceptron having an input layer, two hidden layers, and an output layer;

[0031] The input layer of the gating network inputs an indicator vector consisting of all image degradation evaluation indicators, which is passed to the next hidden layer; the two hidden layers extract features and perform activation function processing on the indicator vector in turn, and finally generate the activation weights of the expert model in the fully connected layer as the output layer.

[0032] Furthermore, the activated expert model is subjected to model parameter correction and weighted combination, and the expert model after weighted combination is used to extract a feature map of the preliminary image features, including:

[0033] The structure of each expert model is the same, which is a two-layer convolutional network, but the model parameters of different expert models are different. The model parameters are the combination of weights and biases of all layers of the expert model. The first layer of each expert model is a convolutional layer, and the second layer is a global pooling layer.

[0034] According to the activation weights of the expert model output by the gating network, all activation weights are sorted and then a preset number of activation weights are selected;

[0035] According to whether the activation weight is selected, the activation indicator variable corresponding to each expert model is updated;

[0036] The updated activation indicator variables, the features extracted by each expert model from the preliminary image features, and the activation weights are used to perform a weighted combination of the expert models, and the expert model after the weighted combination is used to output a feature map extracted from the preliminary image features.

[0037] Furthermore, the knowledge accurate correction module uses the hypernetwork to output correction amounts of model parameters of different expert models based on the image degradation evaluation index, thereby adjusting the model parameters of the expert model, including:

[0038] The hypernetwork is a multilayer perceptron with two fully connected layers, and its input is an image degradation assessment index. The image degradation assessment index is processed by two fully connected layers of the hypernetwork to obtain a feature representation of a fixed size, and then a shared feature is obtained after layer normalization processing. After limiting the output range through an activation function, the output is limited to an amplitude range using the model parameters and shared features of the expert model, thereby obtaining a correction amount with the same dimension as the model parameters of the expert model; the output correction amount will be added to the original model parameters of the expert model, thereby realizing the model parameter correction process of the expert model.

[0039] Furthermore, the detection result output module adopts the detection head of YOLOX, takes the feature map extracted by the weighted combined expert model output by the sparse hybrid expert processing module as input, and uses the detection head to output the target detection results, including target classification results, bounding box regression, and target confidence prediction results.

[0040] Furthermore, the overall loss function of the space debris detection model during training is for:

[0041] ;

[0042] in Various detection performance losses after the feature maps extracted by the expert model are input into the detection head, including classification loss, bounding box regression loss, and confidence loss of target detection; is a learnable parameter, is the corrected loss of the expert model, expressed as:

[0043] ;

[0044] in, is the weight coefficient of the regularization term, represents the model parameters of the k-th expert model, are the model parameters of the modified expert model.

[0045] A large-field-of-view image space debris detection device based on on-orbit adaptation, comprising:

[0046] An image acquisition unit, used to acquire a large-field starry sky image to be detected;

[0047] An image space debris detection unit is used to receive a large-field-of-view starry sky image and input the large-field-of-view starry sky image into a trained space debris detection model to obtain a target detection result for the large-field-of-view starry sky image;

[0048] The space debris detection model includes a coupling factor evaluation module, a sparse mixed expert processing module, a knowledge precision correction module and a detection result output module, wherein:

[0049] The coupling factor evaluation module is used to perform preliminary image feature extraction and calculation of different image degradation evaluation indicators for large-field starry sky images;

[0050] The sparse hybrid expert processing module uses the image degradation assessment index to calculate the activation weight through the gated network, thereby dynamically activating the corresponding expert model; the model parameters of the activated expert model are modified and weighted combined, and the weighted combined expert model is used to extract the feature map of the preliminary image features;

[0051] The knowledge accurate correction module uses a hypernetwork to output correction amounts of model parameters of different expert models based on the image degradation assessment index, thereby adjusting the model parameters of the expert model;

[0052] The detection result output module is used to perform target detection on the feature map extracted by the weighted combined expert model and output the target detection result.

[0053] A terminal device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for detecting spatial debris of a large field of view image based on on-orbit adaptation is implemented.

[0054] A computer-readable storage medium stores a computer program; when the computer program is executed by a processor, the method for detecting spatial debris of a large field of view image based on on-orbit adaptation is implemented.

[0055] Compared with the prior art, the present invention has the following technical features:

[0056] The present invention proposes a method for detecting spatial debris of large field of view images based on on-orbit adaptation. It uses multiple image degradation evaluation indicators at different levels to control expert models in different fields to achieve effective response to different factors. At the same time, it introduces a knowledge precision correction module to achieve more sophisticated adaptive processing. A fusion feature extraction model is designed to obtain the macroscopic state characteristics of image data containing semantic information by combining multiple known image evaluation indicators; multi-level sparse expert models of different degrees are designed, and each expert model processes degradation interference at different levels. The gated network dynamically selects a suitable expert model for processing according to the input evaluation indicators, thereby reducing the computational cost and improving the detection accuracy; a hypernetwork is designed to automatically adjust the parameters of the expert model using multiple evaluation indicators generated by the macro feature extraction encoder. The hypernetwork dynamically generates correction amounts based on these indicators, corrects the weights of the expert model, and makes it more accurately adapt to the characteristics and quality of the current image; finally, a detection head is used to extract target information from the feature map corrected by the expert model and perform target detection in the detection image. The detection head can achieve high-precision detection and on-orbit adaptation in complex backgrounds through efficient classification, regression and confidence prediction. The present invention significantly improves the target detection performance of large-field-of-view detection images in complex space environments, including target detection rate and target detection positioning accuracy, through the collaborative work of the above-mentioned multiple modules. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a structural schematic diagram of the space debris detection model in the present invention;

[0058] Figure 2 A schematic diagram of the structure of a feature extraction encoder in one embodiment of the present invention;

[0059] Figure 3 A schematic diagram of the structure of a down-sampling residual block in one embodiment of the present invention;

[0060] Figure 4 Schematic diagram of the structure of an identical residual block in one embodiment of the present invention;

[0061] Figure 5 This is a schematic diagram of the AFFM1 processing process in one embodiment of the present invention;

[0062] Figure 6 A schematic diagram of the AFFM2 processing process in one embodiment of the present invention;

[0063] Figure 7 A schematic diagram of the AFFM3 processing process in one embodiment of the present invention;

[0064] Figure 8 The figure is a detection result of a large-field starry sky image in one embodiment of the present invention. DETAILED DESCRIPTION

[0065] When using a wide-field camera to observe objects such as space debris, it is often affected by internal camera factors such as optical distortion and dark current, as well as external factors such as high-energy particles and atmospheric refraction, causing the observed data to degrade to varying degrees, such as varying degrees of background noise and optical distortion. At the same time, due to differences in the working modes of the camera turntable, the number distribution and motion characteristics of background targets also vary greatly. In different scenarios, multiple factors with different degrees of influence are coupled to each other, resulting in different levels of degradation and interference to the observed data, making it difficult for the limited "expert" layer under a single indicator to fully adapt, and thus unable to achieve the best detection effect.

[0066] The present invention provides a method for detecting space debris of large-field-of-view images based on on-orbit adaptation, which uses diversified evaluation indicators to control expert modules in different fields to achieve effective response to different factors, and introduces expert knowledge correction to achieve more sophisticated adaptive processing.

[0067] See attached Figure 1 The present invention provides a method for detecting space debris using a large field of view image based on on-orbit adaptation, comprising:

[0068] Obtain a large-field-of-view starry sky image to be detected, input the large-field-of-view starry sky image into a trained space debris detection model, and obtain a target detection result for the large-field-of-view starry sky image;

[0069] The space debris detection model includes a coupling factor evaluation module, a sparse mixed expert processing module, a knowledge precision correction module and a detection result output module, wherein:

[0070] The coupling factor evaluation module is used to perform preliminary image feature extraction and calculation of different image degradation evaluation indicators for large-field starry sky images;

[0071] The sparse hybrid expert processing module uses the image degradation assessment index to calculate the activation weight through the gated network, thereby dynamically activating the corresponding expert model; the model parameters of the activated expert model are modified and weighted combined, and the weighted combined expert model is used to extract the feature map of the preliminary image features;

[0072] The knowledge accurate correction module uses a hypernetwork to output correction amounts of model parameters of different expert models based on the image degradation assessment index, thereby adjusting the model parameters of the expert model;

[0073] The detection result output module is used to perform target detection on the feature map extracted by the weighted combined expert model and output the target detection result.

[0074] The present invention constructs a space debris detection model, which introduces diversified image quality evaluation indicators to more comprehensively describe the current data status, thereby providing more precise guidance for the selection of expert models. At the same time, it introduces expert knowledge correction to achieve more accurate feature extraction and target detection with a limited number of models.

[0075] The specific design of the space debris detection model is further explained in detail below based on the training process of the space debris detection model.

[0076] 1. Coupling factor evaluation module.

[0077] During the training stage, the training samples of the coupling factor evaluation model are large-field-of-view starry sky image datasets; the samples in the large-field-of-view starry sky image datasets are large-field-of-view starry sky images that are affected by stray light, hot pixels, and high-energy particle noise.

[0078] For the large field of view starry sky image samples in the large field of view starry sky image dataset , using the fusion feature extraction model to extract samples Perform preliminary image feature extraction to obtain macroscopic state features of image data containing semantic information ; At the same time, calculate the image degradation evaluation index as a sample Label information.

[0079] (1.1) Image degradation assessment indicators.

[0080] There are six image degradation evaluation indicators in this scheme, namely: background noise , energy concentration , the degree of optical distortion , target density , target motion characteristics (target wire length), detection capability ; Each image degradation evaluation index is expressed as follows:

[0081] First, a simple binary threshold segmentation method can be used to preliminarily segment the samples. The light spot and background noise part; background noise Measuring samples The intensity and distribution of background noise in the sample are calculated by The standard deviation of the background noise in is obtained as:

[0082] (1);

[0083] in, It is a sample Pixels in the background area The strength of the position, It is a sample The mean gray value of the background part, is the number of pixels in the background part; the larger the background noise, the larger the value of the background noise distribution.

[0084] Target density Used to describe samples The distribution of the target in the sample is calculated by The number of targets and the area covered are obtained as follows:

[0085] (2);

[0086] in, It is a sample The number of targets detected in It is a sample The total area of ​​the image (in pixels).

[0087] Optical distortion Describes the sample The actual degree of distortion in:

[0088] (3);

[0089] in, It is a sample The number of pixels in the image, The first The distorted radial distance of pixels, is the radial distance of the pixel in the ideal case.

[0090] The samples can be roughly extracted using a simple threshold segmentation method. Each spot in the image and determine the location and area of ​​the spot, energy concentration For sample The energy concentration of each extracted light spot is the average value of the energy concentration of all light spots in the image. The energy concentration of each extracted light spot is the energy proportion of 80% of its gray value.

[0091] Detection capability For the current sample The apparent magnitude of the faintest (lowest average signal-to-noise ratio) target can be detected (the apparent magnitude refers to the brightness of the star as seen by the observer with the naked eye). And target motion characteristics All samples Or known properties of a large field-of-view starry sky image to be detected.

[0092] Note: For The average signal-to-noise ratio The calculation method is:

[0093] (4);

[0094] in, is the grayscale mean of the light spot.

[0095] The above six image degradation evaluation indicators are used as samples Label information.

[0096] (1.2) Fusion feature extraction model.

[0097] The fusion feature extraction model includes a feature extraction encoder , through feature extraction encoder Get the sample Preliminary image features f containing image semantic information; feature extraction encoder Based on the residual network, the full fusion of extracted features is achieved through three adjacent layer feature fusion modules (AFFM); Figure 2 As shown, where:

[0098] Feature Extraction Encoder It includes convolutional layers, maximum pooling layers, and a series of combinations of IR blocks and DR blocks;

[0099] In the sample When performing preliminary image feature extraction, first use the convolutional layer to extract samples The method comprises the following steps: obtaining a feature map and extracting the number of channels, and then reducing the resolution of the feature map to half of the original one through a maximum pooling layer. Then, further feature extraction is performed through a deep network composed of an identical residual block and a down-sampled residual block, wherein the number of output channels and the resolution of the output features of the identical residual block are unchanged relative to the number of input channels and the resolution of the input features, the number of output channels of the down-sampled residual block is twice the number of input channels, and the resolution of the output features is half the resolution of the input features; and fully fusing the features extracted by the identical residual block and the down-sampled residual block is achieved through three adjacent layer feature fusion modules (AFFM) to obtain the preliminary image feature f.

[0100] like Figure 2 As shown, a feature extraction encoder in one embodiment of the present invention The specific structure is:

[0101] Table 1: Feature extraction encoder structure.

[0102]

[0103] Note: In the above table, Conv1…Conv20 represent different convolution units. The parameters in brackets are the parameters of the convolution units. Figure 3 and Figure 4 .

[0104] In this embodiment, in order to make full use of the features of each level and better obtain the semantic information of different levels, a neighboring feature fusion module (AFFM) is designed in the basic residual network; the structures of the three neighboring feature fusion modules are the same, namely AFFM1, AFFM2 and AFFM3; wherein:

[0105] The input of AFFM1 is the feature F2_2 output by the first identical residual block, the feature F2_3 output by the second identical residual block, the feature F3_1 output by the first down-sampled residual block, and the feature F3_2 output by the third identical residual block. The specific processing process is as follows: Figure 5 As shown in the figure, after the fusion of features F2_2 and F2_3, they are processed by convolution unit and BN+ReLU, and then fused with the fusion results of features F3_1 and F3_2. The fusion results are processed by convolution unit, BN+ReLU and pooling unit, and the output features are .

[0106] The input of AFFM2 is the feature F3_1 output by the first down-sampled residual block, the feature F3_2 output by the third identical residual block, the feature F4_1 output by the second down-sampled residual block, and the feature F4_2 output by the fourth identical residual block. The specific processing process is similar to that of AFFM1. Figure 6 , which will not be described here; the output characteristics of AFFM2 are .

[0107] The input of AFFM3 is the feature F4_1 output by the second down-sampled residual block, the feature F4_2 output by the fourth identical residual block, the feature F5_1 output by the third down-sampled residual block, and the feature F5_2 output by the fifth identical residual block. The specific processing process is similar to that of AFFM1. Figure 7 , which will not be described here; the output characteristics of AFFM2 are .

[0108] In getting features , and Afterwards, a series fusion operation is performed with the feature F5_2 output by the fifth identical residual block, and finally the preliminary image feature f is output.

[0109] 2. Sparse mixture expert processing module.

[0110] The coupling factor evaluation module outputs six different image degradation evaluation indicators, and the indicator vector composed of them is recorded as The core task of the sparse hybrid expert processing module is to select an appropriate expert model based on the gating network according to the image degradation evaluation index from the coupling factor evaluation module, and to perform further precise processing on the preliminary image features f after the parameters of the expert model are corrected in the knowledge precision correction module.

[0111] (2.1) Gated network.

[0112] The gating network is a multi-layer perceptron with an input layer, two hidden layers, and an output layer.

[0113] First, the input layer of the gating network inputs the index vector consisting of six image degradation evaluation indicators , the indicator vector Passed to the next hidden layer.

[0114] The second is the hidden layer, which has two layers in total, and is implemented as follows:

[0115] (5);

[0116] in, is the output feature of the first hidden layer, is the activation function, , , is the weight matrix of the first hidden layer, is the bias term, is the number of hidden layer units in the gating network, is the real number space.

[0117] (6);

[0118] in, , , is the weight matrix of the second fully connected layer, is the bias term.

[0119] The last layer is the output layer, which is implemented through a fully connected layer and is responsible for generating the activation weights of the expert model; outputting a dimensional expert activation vector , represents the total number of expert models, Represents the activation probability of each expert model; the specific implementation is as follows:

[0120] (7);

[0121] in, is the activation function, , , is the weight matrix of the output layer, is the bias term of the output layer; And each value Representative The activation weight (activation probability) of an expert model.

[0122] (2.2) Expert model.

[0123] Each expert model has the same structure, which is a two-layer convolutional network, but the model parameters of different expert models are different, which is reflected in the different weights and biases of each network layer. Therefore, the features extracted by each expert model are different; the first layer of each expert model is a convolutional layer, and the second layer is a global pooling layer. It should be noted that when each expert model processes the preliminary image feature f, the model parameters used are the results corrected by the knowledge precision correction module.

[0124] The convolutional layer of the first layer of each expert model performs convolution processing on the preliminary image features f to learn local features:

[0125] (8);

[0126] in, represents a two-dimensional convolution, is the convolution kernel of the convolution layer, is the output of the convolutional layer, is the bias term.

[0127] Then, nonlinearity is added via the ReLU activation function:

[0128] (9);

[0129] in, The output of the convolutional layer passes through the activation function The result after.

[0130] The second layer is the global pooling layer, and the features extracted by the k-th expert model After global pooling The processed result is expressed as:

[0131] (10);

[0132] (2.3) Dynamic activation of expert model.

[0133] Expert activation vector output by the gating network , select the expert model that needs to be activated for subsequent feature extraction; this part uses the Top-K strategy from Select the first The expert model corresponding to the maximum activation weight is activated; the Top-K selection steps are as follows:

[0134] First, the expert activation vector Sort in descending order to get the sorted index list Right now:

[0135] (11);

[0136] in, is the activation weight The sorted index.

[0137] Then, from the activation weights Select the first K activation weights ; At this time, all The activation indicator variable corresponding to the expert model Update:

[0138] (12);

[0139] in, is the activation weight The corresponding index.

[0140] according to The instruction information can activate the corresponding expert model; that is, When the value of is 1, the corresponding k-th expert model will be activated.

[0141] (2.4) Expert model after weighted combination.

[0142] Finally, the K activated expert models are weighted combined, and the sparse hybrid expert uses the weighted combined expert model output to extract the feature map of the preliminary image features. It is expressed as follows:

[0143] (13).

[0144] 3. Knowledge precision correction module.

[0145] The knowledge precision correction module introduces the expert knowledge correction strategy, and corrects the model parameters of the selected expert model through the hypernetwork to adjust the model parameters of different expert models, so that each activated expert model can more accurately adapt to the specific characteristics of the current feature map and better cope with different degradation conditions.

[0146] The model parameters of each expert model are the combination of weights and biases of all its layers; the hypernetwork is a multilayer perceptron with two fully connected layers, and its input is the image degradation evaluation index. The image degradation evaluation index uses the two fully connected layers of the hypernetwork to obtain a fixed-size feature representation, and then uses layer normalization to obtain shared features. After limiting the output range through the activation function, the model parameters and shared features of the expert model are used to limit the output to a range of amplitudes, thereby outputting a correction value with the same dimension as the model parameters of the expert model. ; Output correction amount It will be added to the original model parameters of the expert model, thereby realizing the model parameter correction process of the expert model; the specific instructions are as follows:

[0147] Index vector composed of image degradation evaluation indicators Shared feature extraction is performed through two fully connected layers of the hypernetwork, as shown below:

[0148] (14);

[0149] (15);

[0150] in, are the output features of the first and second fully connected layers of the hypernetwork, respectively. are the weights and biases of the first fully connected layer in the hypernetwork, are the weights and biases of the second fully connected layer, respectively.

[0151] Using layer normalization dGet shared features :

[0152] (16);

[0153] Finally, the output range is limited by the activation function tanh, and then the output is scaled to a suitable amplitude range (multiplied by a preset small coefficient ) to prevent the parameters from changing too much. The specific implementation is as follows:

[0154] The output correction amount for:

[0155] (17);

[0156] in, is a learnable scaling factor, whose initial value is 0.01; = [ ] represents the model parameters of the kth expert model, and is the weight combination and bias combination of all network layers of the k-th expert model; Represents the correction amount corresponding to the kth expert model.

[0157] Finally, the model parameters of the modified expert model are It is expressed as follows:

[0158] (18);

[0159] Using model parameters The original model parameters of the expert model After the correction, the expert model is used to extract the preliminary image features f.

[0160] 4. Detection result output module.

[0161] The detection result output module uses the YOLOX detection head to extract the feature map of the expert model after the weighted combination output by the sparse mixed expert processing module. As input, the detection head is used to output target detection results, including target classification results, bounding box regression, and target confidence prediction results.

[0162] The network structure of the detection head consists of two convolutional layers and an output layer. The specific design is as follows:

[0163] First, the first convolutional layer outputs for:

[0164] (19);

[0165] in, is the activation function, represents a two-dimensional convolution, , is the convolution kernel and bias of the first convolutional layer.

[0166] Next is the second convolutional layer, and the output is:

[0167] (20);

[0168] in, , is the convolution kernel and bias of the second convolution layer.

[0169] Finally, the output layer of YOLOX generates relevant parameters for each detection box based on the convolution operation, including target classification results (category score, judging whether it is a target or background), bounding box regression, and target confidence prediction results.

[0170] 5. Loss function for space debris detection model training.

[0171] Since the output of the hypernetwork in the knowledge precision correction module affects the input features of the detection head, the overall loss function of the space debris detection model during training includes the correction loss of the expert model and the target detection loss of the detection head.

[0172] (5.1) Object detection loss of the detection head.

[0173] This part of the loss is the feature map extracted by the expert model Various types of detection performance loss after input to the detection head:

[0174] (twenty one);

[0175] in is the object detection loss of the detection head.

[0176] is the classification loss for target detection, which is calculated using cross entropy loss The difference between the category prediction and the true category of the detection box:

[0177] (twenty two);

[0178] In the above formula, Indicates the true category of the target corresponding to the i-th detection box, It is the target confidence prediction result output by the detection head.

[0179] is the bounding box regression loss, that is, the SmoothL1Loss loss function is used to regress each detection box ( is the coordinate of the center point of the detection box, The width and height of the detection box) parameters ensure that the detection box is close to the real bounding box:

[0180] (twenty three);

[0181] in, represents the i-th detection box predicted by the detection head, is the true i-th bounding box.

[0182] is the confidence loss, that is, the error of the target confidence prediction result is calculated using the binary cross entropy loss:

[0183] (twenty four);

[0184] in, Indicates the existence mark of the target in the i-th detection box, It is the target confidence prediction result output by the detection head.

[0185] (5.2) Corrected loss of expert model.

[0186] In order to control the scale of expert model correction, L2 regularization loss is added to the loss function to prevent over-correction:

[0187] (25);

[0188] in, is the corrected loss of the expert model, is the weight coefficient of the regularization term, which is used to control the scale of the correction. Represents the model parameters of the kth expert model.

[0189] Therefore, the overall loss function of the space debris detection model during training is It is expressed as:

[0190] (26);

[0191] in, are learnable parameters.

[0192] The space debris detection model is trained using samples from a large field of view starry sky image dataset until the overall loss function The training is completed when convergence or the maximum number of training times is reached, and the trained space debris detection model is saved.

[0193] In practical applications, for a large field of view starry sky image of unknown target category, it is input into a trained space debris detection model to obtain a target detection result. In one embodiment of the present invention, the star target detection result of a large field of view starry sky image is as follows: Figure 8 shown.

[0194] The method proposed in the present invention realizes more comprehensive and accurate data processing under the premise of effectively limiting the network scale, and achieves excellent target detection effect in a large field of view detection data set in a complex spatial environment.

[0195] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for detecting space debris based on large field of view images based on on-orbit adaptation, characterized in that: include: Obtain a large-field-of-view starry sky image to be detected, input the large-field-of-view starry sky image into a trained space debris detection model, and obtain a target detection result for the large-field-of-view starry sky image; The space debris detection model includes a coupling factor evaluation module, a sparse mixed expert processing module, a knowledge precision correction module and a detection result output module, wherein: The coupling factor evaluation module is used to perform preliminary image feature extraction and calculation of different image degradation evaluation indicators for large-field starry sky images; The sparse hybrid expert processing module uses the image degradation assessment index to calculate the activation weight through the gated network, thereby dynamically activating the corresponding expert model; the model parameters of the activated expert model are modified and weighted combined, and the weighted combined expert model is used to extract the feature map of the preliminary image features; The knowledge accurate correction module uses a hypernetwork to output correction amounts of model parameters of different expert models based on the image degradation assessment index, thereby adjusting the model parameters of the expert model; The detection result output module is used to perform target detection on the feature map extracted by the weighted combined expert model and output the target detection result.

2. The method for detecting space debris based on large field of view images based on on-orbit adaptation according to claim 1, characterized in that: The different image degradation assessment indicators include background noise, energy concentration, optical distortion, target density, target motion characteristics and detection capability; wherein: Background noise is used to measure the intensity and distribution of background noise in large-field starry sky images, and is obtained by calculating the standard deviation of background noise in large-field starry sky images; Target density is used to describe the distribution of targets in a large-field-of-view starry sky image. It is obtained by calculating the number and coverage area of ​​targets in the large-field-of-view starry sky image. The degree of optical distortion is used to describe the actual degree of distortion in a wide-field star image. It is obtained by calculating the radial distance of pixels in a wide-field star image and their distortion, and the radial distance of the pixel under ideal conditions. The detection capability is the apparent magnitude of the target with the lowest mean signal-to-noise ratio that can be detected in a wide-field star image; The energy concentration is the average value of the energy concentration of all light spots in the large field of view starry sky image; The target motion feature is the target pull line length in the wide field of view starry sky image.

3. The method for detecting space debris based on large field of view images based on on-orbit adaptation according to claim 1, characterized in that: The coupling factor assessment module includes a feature extraction encoder; The feature extraction encoder includes a convolutional layer, a maximum pooling layer, and a series of identical residual blocks and down-sampling residual blocks; When performing preliminary image feature extraction on a wide-field starry sky image, a convolutional layer is first used to extract a feature map of the wide-field starry sky image and the number of channels, and then the resolution of the feature map is reduced to half of the original through a maximum pooling layer, and then further feature extraction is performed through a deep network composed of an identical residual block and a down-sampling residual block, wherein the number of output channels and the resolution of the output features of the identical residual block are unchanged relative to the number of input channels and the resolution of the input features, the number of output channels of the down-sampling residual block is twice the number of input channels, and the resolution of the output features is half the resolution of the input features; three adjacent layer feature fusion modules are used to realize the fusion of features extracted by the identical residual block and the down-sampling residual block to obtain the preliminary image features.

4. The method for detecting space debris based on large field of view images based on on-orbit adaptation according to claim 1, characterized in that: The sparse hybrid expert processing module uses the image degradation evaluation index to calculate the activation weight through the gated network, including: The gating network is a multi-layer perceptron having an input layer, two hidden layers, and an output layer; The input layer of the gating network inputs an indicator vector consisting of all image degradation evaluation indicators, which is passed to the next hidden layer; the two hidden layers extract features and perform activation function processing on the indicator vector in turn, and finally generate the activation weights of the expert model in the fully connected layer as the output layer.

5. The method for detecting space debris based on large field of view images based on on-orbit adaptation according to claim 1, characterized in that: Modifying and weighting the model parameters of the activated expert model, and extracting the feature map of the preliminary image features using the expert model after weighting combination, including: The structure of each expert model is the same, which is a two-layer convolutional network, but the model parameters of different expert models are different. The model parameters are the combination of weights and biases of all layers of the expert model. The first layer of each expert model is a convolutional layer, and the second layer is a global pooling layer. According to the activation weights of the expert model output by the gating network, all activation weights are sorted and then a preset number of activation weights are selected; According to whether the activation weight is selected, the activation indicator variable corresponding to each expert model is updated; The updated activation indicator variables, the features extracted by each expert model from the preliminary image features, and the activation weights are used to perform a weighted combination of the expert models, and the expert model after the weighted combination is used to output a feature map extracted from the preliminary image features.

6. The method for detecting space debris based on large field of view images based on on-orbit adaptation according to claim 1, characterized in that: The knowledge accurate correction module uses the hypernetwork to output the correction amount of the model parameters of different expert models based on the image degradation evaluation index, thereby adjusting the model parameters of the expert model, including: The hypernetwork is a multilayer perceptron with two fully connected layers, and its input is an image degradation assessment index. The image degradation assessment index is processed by two fully connected layers of the hypernetwork to obtain a feature representation of a fixed size, and then a shared feature is obtained after layer normalization processing. After limiting the output range through an activation function, the output is limited to an amplitude range using the model parameters and shared features of the expert model, thereby obtaining a correction amount with the same dimension as the model parameters of the expert model; the output correction amount will be added to the original model parameters of the expert model, thereby realizing the model parameter correction process of the expert model.

7. The method for detecting space debris based on large field of view images based on on-orbit adaptation according to claim 1, characterized in that: The detection result output module adopts the detection head of YOLOX, takes the feature map extracted by the weighted combined expert model output by the sparse hybrid expert processing module as input, and uses the detection head to output the target detection results, including target classification results, bounding box regression, and target confidence prediction results.

8. The method for detecting space debris using large field of view images based on on-orbit adaptation according to claim 1, characterized in that: The overall loss function of the space debris detection model during training is for: ； in Various detection performance losses after the feature maps extracted by the expert model are input into the detection head, including classification loss, bounding box regression loss, and confidence loss of target detection; is a learnable parameter, is the corrected loss of the expert model, expressed as: ； in, is the weight coefficient of the regularization term, represents the model parameters of the k-th expert model, are the model parameters of the modified expert model.

9. A large field of view image space debris detection device based on on-orbit adaptation, characterized in that: include: An image acquisition unit, used to acquire a large-field starry sky image to be detected; An image space debris detection unit is used to receive a large-field-of-view starry sky image and input the large-field-of-view starry sky image into a trained space debris detection model to obtain a target detection result for the large-field-of-view starry sky image; The space debris detection model includes a coupling factor evaluation module, a sparse mixed expert processing module, a knowledge precision correction module and a detection result output module, wherein: The coupling factor evaluation module is used to perform preliminary image feature extraction and calculation of different image degradation evaluation indicators for large-field starry sky images; The sparse hybrid expert processing module uses the image degradation assessment index to calculate the activation weight through the gated network, thereby dynamically activating the corresponding expert model; the model parameters of the activated expert model are modified and weighted combined, and the weighted combined expert model is used to extract the feature map of the preliminary image features; The knowledge accurate correction module uses a hypernetwork to output correction amounts of model parameters of different expert models based on the image degradation assessment index, thereby adjusting the model parameters of the expert model; The detection result output module is used to perform target detection on the feature map extracted by the weighted combined expert model and output the target detection result.

10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor; characterized in that: When the processor executes the computer program, the method for detecting space debris with a large field of view image based on on-orbit adaptation according to any one of claims 1 to 8 is implemented.

Images