A lightweight target detection model approach for spaceborne edge computing

Abstract

The present invention provides a lightweight target detection model method for onboard edge computing, comprising the following steps: S1. Training an image target detection model capable of identifying feature-missing targets by acquiring a training dataset, performing self-supervised comparative training, and constructing a channel attention mechanism; S2. Applying a graph-structured abstract neural network model to the target detection model, identifying and pruning redundant channels to compress the neural network model, thereby obtaining a compressed model; and S3. Optimizing the performance of the compressed model using a contrastive generative adversarial network and knowledge distillation methods to obtain a lightweight target detection model. This method ensures detection speed and accuracy while enabling lightweight deployment of the compressed model in an onboard edge computing environment, enhancing the ability to identify feature-missing targets.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and more specifically, to a lightweight target detection model method for satellite-borne edge computing. Background Art

[0002] Satellite image target detection uses images captured by low-orbit remote sensing satellites as its data source and utilizes efficient image processing techniques to locate and classify valuable targets within the images. Due to the limited computing power of individual satellites, onboard edge computing (SAT) is widely used in satellite image target detection. However, as satellite payloads continue to increase, the amount of data generated by satellites is also increasing significantly, resulting in a continuous decrease in the efficiency of onboard edge computing. To efficiently identify targets within large satellite image data volumes and achieve timely onboard edge computing, a lightweight target detection model approach for SAT is urgently needed to simplify the current satellite image target detection process.

[0003] In recent years, with the continuous development of deep learning, models represented by convolutional neural networks have demonstrated outstanding performance in image processing, providing an opportunity for accurate and efficient onboard intelligent edge computing. Currently, deep learning-based object detection models can be divided into two categories: one-stage models and two-stage models. Two-stage object detection models first use a selective search algorithm to search for possible object locations, then use a convolutional neural network to extract features from these candidate regions. Finally, the input is fed into a classifier to label the object. One-stage object detection models combine the feature extraction and object detection processes, using a region proposal network to improve the candidate region search process. Region proposal, feature extraction, and bounding box regression are integrated into a unified network framework, achieving end-to-end object detection. To deploy models in computing-constrained onboard edge computing environments, researchers both domestically and internationally are using pruning algorithms to compress models. Model pruning algorithms are categorized as structured pruning and unstructured pruning. Structured pruning compresses the model without changing its structure. It calculates the importance of each neuron in each layer of the neural network and resets the weights of unimportant neurons to 0 based on a threshold, thereby compressing the model. Unstructured pruning methods involve learning channel sparsity in deep neural networks to identify channels that need to be pruned, changing the model's structure and ultimately achieving model compression. To compensate for the loss of accuracy during model compression, existing solutions use knowledge distillation algorithms to transfer knowledge from the pre-pruning complex network to the pruned compressed model, thereby fine-tuning the compressed model's performance.

[0004] Although object detection models based on deep neural network algorithms have significantly improved accuracy compared to traditional detectors, they often involve a significant amount of redundant computation, requiring extensive storage space and high-performance computing units. Given the small size of satellites involved in onboard edge computing, limited computing resources, storage units, and payload, deploying high-precision object detection models onboard is difficult. Therefore, while maintaining detection accuracy, achieving model lightweighting and efficient detection, ensuring stable operation within the resource-constrained environment of onboard edge computing platforms, is crucial for onboard intelligent satellite imagery detection. While model compression schemes based on structured pruning can remove neurons with low contributions from each layer of the neural network, they do not effectively compress the model's redundant structure and thus fail to effectively reduce the computational load. Current model compression schemes based on unstructured pruning analyze neuron contributions using the channel dimension, but they cannot effectively identify redundant channels and neurons, preventing them from effectively compressing the model to improve efficiency. To optimize the performance of the compressed model, current solutions use knowledge distillation algorithms to fine-tune the model. Knowledge distillation algorithms enable compressed models to effectively learn the hard and soft label knowledge required for classification. However, current knowledge distillation solutions struggle to enable unstructured pruned models to learn non-critical features from the original model. This limitation results in the unstructured pruned models being unable to effectively distinguish the categories of objects with incomplete features. In spaceborne computing applications, many objects in images exhibit deformation, displacement, and even incomplete features. Existing recognition solutions based on rule-based training with complete target features struggle to effectively identify objects in such situations. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to propose a lightweight target detection model method for onboard edge computing, solving the problem that the compressed model has difficulty identifying targets with missing features, thereby improving target detection accuracy. The present invention uses a graph structure to represent neurons and constructs a redundant channel pruning model to solve the problem of difficulty removing redundant channels in deep neural network models. The present invention constructs a compression model performance enhancement model based on the comparison of generative adversarial networks and knowledge distillation algorithms, trains the compression model's ability to recognize small sample features, and improves the model's target detection performance.

[0006] The technical means adopted in the present invention are as follows:

[0007] A lightweight target detection model method for spaceborne edge computing includes the following steps:

[0008] S1. Train an image object detection model that can identify feature-missing objects by collecting training datasets, performing self-supervised contrastive training, and building a channel attention mechanism.

[0009] S2. Use a graph structure to abstract the target detection model into a neural network structure graph, identify and prune redundant channels to compress the neural network model, and obtain a compressed model.

[0010] S3. Use contrastive generative adversarial networks and knowledge distillation methods to optimize the performance of the compression model and obtain a lightweight target detection model.

[0011] Furthermore, S1 specifically includes the following steps:

[0012] S11, collecting original images taken by the satellite, screening images with the target to be identified from the original images, cropping the images of the target to be identified, and using the cropped images as a pre-detection data set;

[0013] S12, input the original image into the contrastive learning framework for pre-detection, and output the category of each image;

[0014] S13. Based on the category of each image, an image target detection model is built based on the YOLOX target detection framework. The structure of the image target detection model includes a backbone network, a path aggregation network PAN and a YOLOX decoupling head network;

[0015] The original image is input into the backbone network. The basic structure of the backbone network includes five different stages, C1 to C5. Each stage can extract the features of the input image and output feature maps of different sizes. The feature maps output by C3 to C5 are input into the PAN network.

[0016] The feature map output by C5 is first passed through The convolution adjusts the number of channels to C, and after upsampling, it is spliced with the feature map output by C4 in the channel dimension; the channel segmentation and aggregation module is used to suppress the background part of the image and highlight the foreground part; the channel segmentation and aggregation module first divides the input feature map in the channel dimension to obtain two features of size feature map, one part of the feature map is directly spliced with the other part after two depth-wise convolutions; a channel attention module is introduced between the two depth-wise convolutions; the feature map after splicing according to the channel dimension is passed again Convolution is used to fuse information between different channels. For the bottom-up part of the PAN network, M3 and M4 are downsampled by 2 times and then concatenated with M4 and M5 respectively. After passing through the CSA module again, the outputs P3, P4, and P5 are obtained. The fused feature maps of different sizes are input into the YOLOX decoupling head network to classify and locate the objects contained therein.

[0017] In the channel attention module, the input feature map first aggregates global spatial features through global average pooling, and then completes information interaction between different channels through group convolution. The number of groupings can be freely selected according to the number of input channels. After passing the Sigmoid activation function, it is multiplied with the input.

[0018] The P3, P4, and P5 feature maps are respectively input into the YOLOX decoupling head network. The decoupling head network extracts the target position and category information respectively, learns them separately through different convolutional neural network branches, and finally performs feature fusion; an additional angle parameter is introduced in the regression branch to convert the anchor box coordinates from (center point, width, height) to (center point, width, height, rotation angle) to achieve prediction of rotated targets.

[0019] Furthermore, S12 specifically includes the following steps:

[0020] S121, use data enhancement method to transform the input original image to obtain two similar images And pass through the encoder Encoder to output the feature map , the encoder architecture is consistent and shares parameters;

[0021] S122, feature map Obtained by feature transformation through the mapping head ;

[0022] S123, by predicting the head Make a prediction ,calculate and The cosine similarity of .

[0023] Furthermore, the contrastive learning framework in S12 includes the data augmentation network structure, encoder, projection network structure and prediction network structure;

[0024] The data enhancement network structure Augmentation is used to expand the data set and transform the input image using methods such as random cropping and horizontal flipping to obtain two similar images. And input them into the encoder respectively;

[0025] The encoder uses the ResNet50 network model;

[0026] The Projection network structure consists of a fully connected layer, a BN layer, and a ReLu activation function. It achieves correct image classification by fusing the encoder output features. The two Projection architectures are the same.

[0027] The prediction network structure consists of a fully connected layer, a batch normalization layer, and a ReLu activation function. By further transforming the output of the projection network, the cosine similarity between its output and the output of another projection network is calculated, and this is used as the loss to train the encoder through back propagation.

[0028] Furthermore, S2 specifically includes the following steps:

[0029] S21. Abstracting the neurons of the image target detection model as nodes, abstracting the transmission relationship between neurons as edges, and constructing a neural network structure diagram with several layers of convolutional layers. The neural network structure diagram is stored in the form of a matrix;

[0030] S22. Prune the convolution kernels in the convolution layer of the image object detection model based on the importance and feature maps of the convolution kernels themselves;

[0031] S23. The connections between neurons in different layers are channels. The similarity of the channels is calculated, and the redundant channels in the image target detection model are pruned based on the similarity of the channels.

[0032] Furthermore, S22 specifically includes the following steps:

[0033] S221. From the perspective of the importance of the convolution kernel itself, directly prune the convolution kernel in the convolution layer; for a Layer convolutional layer network, assuming that the first Convolutional layer Pruning is performed, wherein Represents the total number of convolution kernel, that is output channels, and the number of input channels of each convolution kernel is , the size is ;

[0034] In this branch, each convolution kernel in this layer is first calculated The L1 norm of , then sort the norms of these convolution kernels from large to small, and according to the pre-set pruning rate , after removing the soft pruning method The convolution kernels are arranged from large to small from front to back; the soft pruning method is to set the convolution kernel to be pruned to 0;

[0035] S222. From the perspective of feature maps, the network is further pruned by judging the correlation between convolution kernels; for a The network at layer In the layer, Represents the total number of feature maps, Represents the height and width of the feature map, that is, the size of each feature map is ; By calculating the importance of each feature map, sorting them in descending order of importance, and removing the most important ones % of the feature map is cut off, and the corresponding convolution kernel is also cut off. The calculation formula for calculating the importance of each feature map is as follows:

[0036]

[0037] in, Represents the importance of each feature map, That is, convert the three-dimensional feature map into the corresponding feature matrix after two-dimensional, represents the mask matrix, the matrix The first row is 0, and the rest of the rows are 1. It represents the SVD decomposition of the matrix, and its formula is as follows:

[0038]

[0039] in, represents the singular values of the matrix, are the left and right singular value vectors respectively, the matrix The rank of ; Change the eigenvalue Arrange from large to small and remove the ones with the largest change % feature map, and the corresponding filters are also cut off accordingly.

[0040] Furthermore, S23 includes the following steps:

[0041] The structural similarity of the channel is calculated based on the interaction relationship between neurons. The calculation formula for the interaction relationship between neurons is:

[0042]

[0043]

[0044] in, is the node contribution, is the node forward contribution, is the node backward contribution, is the forward node set, is the backward node set;

[0045] Based on the calculated contribution of each node, the similarity between channels is calculated. The formula for calculating the similarity of channels is:

[0046]

[0047]

[0048]

[0049] in, It is a channel With channel The similarity of is the structural similarity, Contribute similarity to the node, and Calculate the channel from the input and output and hierarchical structure With channel The structural similarity of For the Layer The node contribution of each node;

[0050] Arrange the channel overlap from high to low, and put the overlap at the end The channels with only a single branch are identified as redundant channels and are pruned from the network. For those channels that have only a single branch selected for removal, their parameters are restored in the subsequent network iterative update process to participate in the next training process. The joint dynamic channel pruning algorithm is iterative until the pruning rate of the network reaches the target pruning rate. Stop when

[0051] Furthermore, S3 specifically includes the following steps:

[0052] S31. Constructing a contrastive generative adversarial network;

[0053] S32. Generate feature-enhanced datasets using contrastive generative adversarial networks;

[0054] S33. Knowledge Distillation: Fine-tuning compressed models using feature-augmented datasets.

[0055] Furthermore, in S31, the contrastive generative adversarial network uses the target detection model as a discriminator, and the optimization function formula of the contrastive generative adversarial network is:

[0056]

[0057] in, is the distribution of real data, is the noise distribution, and are the outputs of the discriminator and the generator respectively; the loss functions of the generator and the discriminator are:

[0058]

[0059]

[0060] in, is the traditional discriminator loss function, To generate the similarity between the sample and the original sample, For expected similarity, the similarity between the generated sample and the original sample is expected to be , by adjusting The feature richness of generated samples can be changed, and are the generation loss of similar data and the generation loss of heterogeneous data, respectively. For variable parameters.

[0061] Furthermore, S33 specifically includes the following steps:

[0062] The feature-enhanced image is input into the original network teacher net and the pruned lightweight network student net respectively, and the classification probability is output after softmax. , The definition is as follows:

[0063]

[0064] in, Output results for the network model, For each category of output, is the total number of categories; represents the temperature hyperparameter, which is used to control the probability distribution of the model output; The larger the value, the smoother the output probability distribution. The smaller the value, the more concentrated the output probability distribution;

[0065] The output of the original network teacher net is used as the soft target, and the cross entropy loss is calculated by combining the soft target with the output of the student net. At the same time, the real label of the satellite image is used. The cross entropy loss is calculated as the output of the hard target and the student net, and the parameters of the student net are updated through the back propagation algorithm. The calculation formula of the back propagation algorithm is as follows:

[0066]

[0067] in, represents the total loss function of the distillation part, 、 represent hard distillation loss and soft distillation loss respectively, Represents the weight parameter.

[0068] Compared with the prior art, the present invention has the following advantages:

[0069] To address the difficulty models face in identifying features-missing targets, this paper proposes a method for constructing an object detection model based on multi-scale feature fusion. To mitigate the reduced detection accuracy and recall after model compression, this method effectively aggregates multi-scale image features. Building on the feature aggregation network, this approach introduces a channel attention mechanism and proposes a channel information aggregation scheme. This helps the model better aggregate shallow and deep semantic features, preventing background interference in the object recognition task. Considering the large number of rotated objects in satellite imagery, this method adds additional angle prediction parameters to the detection head to predict rotated objects.

[0070] To remove redundant channels during model compression, this paper proposes a neural network model compression method based on channel pruning. This model first abstracts neurons into nodes and uses a graph to represent the model structure, enabling better analysis of neural network models. Next, this method identifies the channels contained in the network model based on the correlation between neurons and their predecessors and successors. Finally, this method calculates channel similarity based on sample pairs, thereby removing redundant channels and compressing complex models.

[0071] In the process of optimizing model performance, this paper proposes a data augmentation-based optimization method to address the difficulty in identifying features in small samples. This method uses a contrastive generative adversarial network to generate a set of augmented samples whose similarity to the original samples falls within a threshold. This set of samples is then used to train knowledge distillation between the pre-compression and post-compression models. The contrastive generative network in this solution uses the pre-compression model as a discriminator to determine the similarity between the generated samples and the original samples. By optimizing model performance through the generated sample set, this method can effectively recover the accuracy loss caused by model compression. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0073] Figure 1 Flow chart of the method of the present invention.

[0074] Figure 2 Schematic diagram of the pre-training framework based on contrastive learning of the present invention.

[0075] Figure 3 This is a schematic diagram of the Projection network structure of the present invention.

[0076] Figure 4 This is a schematic diagram of the Prediction network structure of the present invention.

[0077] Figure 5 This is the target detection framework diagram of the present invention.

[0078] Figure 6 This is the PAN structure diagram of the present invention.

[0079] Figure 7 This is the CSA structure diagram of the present invention.

[0080] Figure 8 This is a structural diagram of the CAM module of the present invention.

[0081] Figure 9 Schematic diagram of the rotation target regression parameters of the present invention.

[0082] Figure 10 Schematic diagram of the joint channel pruning algorithm of the present invention.

[0083] Figure 11 Schematic diagram of the redundant channel pruning method of the present invention.

[0084] Figure 12 Schematic diagram of the model performance enhancement method of the present invention. DETAILED DESCRIPTION

[0085] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0086] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0087] The present invention provides a lightweight target detection model method for satellite-based edge computing. The method trains an image target detection model that can identify target objects with missing features by collecting training data sets, self-supervised comparative training, and building a channel attention mechanism. In the neural network model compression stage, the present invention proposes a neural network model compression method based on channel pruning, such as Figure 1 As shown. This method uses a graph structure to abstract the neural network model, identify and prune redundant channels, and thus generate a more compact model. In the model optimization stage, the present invention proposes a model training method based on data enhancement. This method uses a contrastive generative adversarial network to generate a set of image feature-enhanced samples and uses knowledge distillation to enable the compact model to learn the target recognition capabilities of the pre-compression model. Specifically, it includes the following steps:

[0088] S1. Train an image object detection model that can identify feature-missing objects by collecting training datasets, performing self-supervised contrastive training, and building a channel attention mechanism.

[0089] S2. Use graph structures to abstract the neural network model of the target detection model, identify and prune redundant channels, and thus compress the neural network model to obtain a compressed model;

[0090] S3. Use contrastive generative adversarial networks and knowledge distillation methods to optimize the performance of the compression model and obtain a lightweight target detection model.

[0091] The specific steps of S1 are as follows:

[0092] Collect an image dataset. Filter the original images to identify the target. Crop the resulting images to a size of 224×224. During cropping, be sure to preserve the integrity of the target and avoid damaging it, which could affect model training. Finally, organize and store the dataset to serve as a pre-training dataset for self-supervised contrastive learning.

[0093] Input the collected original image into Figure 2 Pre-training is performed in the contrastive learning framework shown in the figure. First, the contrastive learning framework uses the data augmentation network structure Augmentation to expand the dataset. Augmentation uses methods such as random cropping and horizontal flipping to transform the input image, obtaining two similar images and inputting them into the encoder respectively. The encoder architecture of the image input is consistent and shares parameters. Then, its output feature map is Obtained by feature transformation through the mapping head Finally, by predicting the head pair Make a prediction , calculate its Cosine similarity. The encoder uses the ResNet50 network model. After the image passes through the network to extract features, the final output size is 7×7×2048.

[0094] Projection network structure: Projection consists of a fully connected layer, a BN layer, and a ReLu activation function. Its input is 7×7×2048. By fusing the output features of the encoder, the correct classification of the image can be achieved. Similar to the encoder, the two projection architectures are the same. Its network structure is as follows Figure 3 shown.

[0095] Prediction network structure: The Prediction head is similar to the Projection network structure and is also composed of a fully connected layer, a BN layer, and a ReLu activation function. By further transforming the output of the Projection network, the cosine similarity between its output and the output of another Projection network is calculated, and this is used as the loss to train the encoder through back propagation. Its network structure is as follows Figure 4 shown.

[0096] Construct a target detection model. Normally, the resolution of satellite images is relatively large, and there are a large number of small targets with low resolution and difficult to identify, and the features of some targets are incomplete. In order to accurately identify these targets, the present invention builds the entire detection model based on the YOLOX target detection framework. In order to better identify small target objects in satellite images, the present invention introduces a bottom-up pathway based on the feature aggregation network FPN (Feature pyramid network, FPN) used by YOLOX to form a path aggregation network PAN (Path aggregation network, PAN), so that shallow information and deep semantic information are better integrated. In addition, an additional channel attention mechanism is introduced in the feature aggregation process to further improve the accuracy and robustness of the detection model. Finally, the rotating target in the needle-sensing image is added to the detection head. A detection box for the rotating target is added to the detection head to better identify the rotating target in the image. The target detection model consists of a backbone network, a path aggregation network PAN and a YOLOX decoupled head network. Its overall structure is as follows Figure 5 shown.

[0097] First, the image and the category label output by the self-supervised contrastive learning framework are input into the backbone network. For each batch of input images, they are first scaled to Size. The Backbone network structure includes a downsampling layer and 5 different stages, such as Figure 5 As shown on the left. Each stage can extract the features of the input image and output feature maps of different sizes. Considering that shallow feature maps are more conducive to identifying small targets, and deep feature maps have a better recognition effect for large targets due to their larger receptive field range. Therefore, the present invention inputs the feature maps output by C3 to C5 into the PAN network, and through the fusion of feature information between different layers, helps the model better identify targets of different sizes in satellite images. The basic specific structure diagram of the PAN network is shown as follows Figure 6 shown.

[0098] Specifically, the feature map output by C5 is first passed through The convolution adjusts the number of channels to C, and after upsampling, it is spliced with the feature map output by C4 in the channel dimension. In addition, in view of the large number of targets in satellite images and the small targets that are difficult to identify, the present invention introduces a channel segmentation aggregation (CSA) module to suppress the background part of the image and highlight the foreground part. The CSA model structure diagram is shown in the figure. Figure 7As shown. This module first divides the input feature map into two parts of size The feature map of one part is directly spliced with the other part after two depth-wise convolutions, so as to reduce the amount of calculation and improve the detection speed. At the same time, the present invention introduces a channel attention module (CAM) between the two depth-wise convolutions to make the model pay more attention to the channels containing more important information. Finally, the feature map spliced by channel dimension is passed through the Convolution is used to fuse information between different channels. The information fusion process for C4 and C3 is the same as described above. For the bottom-up portion of the PAN network, the entire process is similar. M3 and M4 are downsampled by a factor of 2 and then concatenated with M4 and M5, respectively. After passing through the CSA module again, the outputs P3, P4, and P5 are obtained. Finally, these fused feature maps of different sizes are input into the subsequent detection head to classify and locate the objects contained therein.

[0099] The CAM module introduced in the present invention is as follows Figure 8 As shown in the figure, the input feature map first aggregates global spatial features through global average pooling. Group convolution is then used to facilitate information exchange between different channels. The number of groupings can be freely selected based on the number of input channels. Compared to using fully connected layers, group convolution can effectively reduce computational complexity while avoiding information loss caused by dimensionality reduction. Finally, the input is multiplied by a sigmoid activation function, allowing the model to focus more on channels containing important information.

[0100] Finally, the P3, P4, and P5 feature maps are input into the YOLOX decoupling head network, whose structure is as follows: Figure 5 As shown on the right. Since YOLOX is an anchor-free framework, there is no need to set the shape of the anchor box in advance. In addition, for rotating targets in satellite images, the present invention introduces additional angle parameters in the regression branch, converting the anchor box coordinates from (center point, width, height) to (center point, width, height, rotation angle) to achieve the prediction of rotating targets, as shown in the schematic diagram. Figure 9 shown.

[0101] The specific steps of S2 are as follows:

[0102] To enable network deployment in a spaceborne environment, this paper proposes a neural network model compression method based on channel pruning to prune the pre-trained network model. By continuously removing "unimportant" convolution kernels during training, the network's parameter count is reduced, making the model suitable for deployment on an in-orbit satellite. This method consists of three phases: constructing a neural network structure diagram, neuron pruning, and channel pruning.

[0103] In the first stage, the present invention abstracts the neurons of the neural network into nodes and the transmission relationship between neurons into edges to construct a neural network structure diagram. The structure diagram is stored in the form of a matrix and is represented as ,in is the depth of the neural network, is the maximum number of neurons in each layer.

[0104] In the second stage, the present invention prunes the convolution kernels in the convolution layer from the perspective of the importance of the convolution kernel itself and the feature map. The method diagram of this stage is shown in Figure 10 shown.

[0105] First, the present invention directly prunes the convolution kernel in the convolution layer from the perspective of the importance of the convolution kernel itself. Layer convolutional layer network, assuming that the first Convolutional layer Pruning is performed, wherein Represents the total number of convolution kernel, that is output channels. And the number of input channels of each convolution kernel is , the size is In this branch, each convolution kernel in this layer is first calculated The L1 norm of After that, the norms of these convolution kernels are sorted from large to small, and the pruning rate is set according to the pre-set , "pruning" those convolution kernels with smaller norm values. Here, the present invention adopts the idea of soft pruning. By setting the convolution kernels to be pruned to 0 instead of actually removing the filter from the network, the entire network model structure remains unchanged, minimizing the impact of different inputs on the pruning results.

[0106] Then, the present invention further prunes the network from the perspective of feature graph by judging the correlation between convolution kernels. The network at layer In the layer, Represents the total number of A feature map. Represents the height and width of the feature map, that is, the size of each feature map is By calculating the importance of each feature map, the relatively unimportant feature maps are cut off, that is, the corresponding convolution kernels are cut off. The calculation formula is as follows:

[0107]

[0108] in Represents the importance of each feature map, That is, the feature matrix corresponding to the three-dimensional feature map is converted into two-dimensional. represents the mask matrix, the matrix row is 0, and the rest of the rows are 1. It represents the SVD decomposition of the matrix, and its formula is as follows:

[0109]

[0110] in represents the singular values of the matrix, are the left and right singular value vectors respectively. From this we can see that the matrix The rank of According to the formula, when the mask matrix Will After the row is set to 0, re- Perform SVD decomposition and get The rank of By observing the change in rank after removing the feature map, we can get Considering that the smaller the rank, the smaller the amount of information contained in the feature map, The larger the value, the more important the feature map is. , the present invention removes those For feature maps with smaller value changes, the corresponding filters are also pruned. Here, the present invention also sets the corresponding filters to 0 instead of actually removing them from the network to ensure the recoverability of the network.

[0111] Finally, the present invention simultaneously selects the channels to be removed and prunes them from the network. For those channels that only have a single branch selected for removal, their parameters are restored in the subsequent network iterative update process to participate in the next training process. It should be noted that the joint dynamic channel pruning algorithm is iterative until the network pruning rate reaches the target pruning rate. Therefore, the network parameter model can be compressed to the target size.

[0112] In the third stage, the present invention further prunes redundant channels by calculating the similarity of the channels. The schematic diagram of the method in this stage is shown in FIG. Figure 11To compare the similarity of channels, the present invention calculates the structural similarity of channels based on the interaction relationship between neurons. The calculation formula for the interaction relationship of neurons is:

[0113]

[0114]

[0115] in, is the node contribution, is the node forward contribution, is the node backward contribution, is the forward node set, is the backward node set.

[0116] Based on the calculated contribution of each node, the similarity between channels can be calculated. The formula for calculating channel similarity is:

[0117]

[0118]

[0119]

[0120] in, It is a channel With channel The similarity of is the structural similarity, Contribute similarity to the node. and Calculate the channel from the input and output and hierarchical structure With channel structural similarity. For the Layer The node contribution of each node.

[0121] Finally, the present invention can effectively identify the overlap degree of each channel based on the calculated similarity of each channel, and remove redundant channels with a high overlap degree.

[0122] The specific steps of S3 are as follows:

[0123] In S2, the heavy target detection model is effectively compressed into a compact model. However, some neurons and channels are pruned, the target detection accuracy of the model decreases, and the model loses the ability to recognize some small sample features. In order to improve the detection accuracy, this paper proposes a model performance enhancement method based on generative adversarial networks and knowledge distillation. The method diagram is shown in the figure. Figure 12This method consists of three stages: building a contrastive generative adversarial network, generating a feature-enhanced dataset, and knowledge distillation.

[0124] This paper proposes a comparative generative adversarial network model. To enable a lightweight model to efficiently learn the ability to recognize small sample features, this model focuses on generating image data that is similar to the original satellite imagery but contains richer features. This model uses the object detection model trained in step 1 as the discriminator of the generative adversarial network. This effectively determines the similarity between the generated image and the original image and can also effectively accelerate model training. The optimization function formula of the generative adversarial network is:

[0125]

[0126] in, is the distribution of real data, is the noise distribution, and are the outputs of the discriminator and the generator respectively. In this network, the loss functions of the generator and the discriminator are:

[0127]

[0128]

[0129] in, is the traditional discriminator loss function, To generate the similarity between the sample and the original sample, is the expected similarity. The similarity between the generated sample and the original sample is expected to be , by adjusting The feature richness of generated samples can be changed. and They are the generation loss of similar data and the generation loss of heterogeneous data, respectively. For variable parameters.

[0130] In the knowledge distillation stage, the present invention uses the feature enhancement dataset to fine-tune the pruned network. First, the feature-enhanced image is input into the original network teacher net and the pruned lightweight network student net respectively, and the classification probability is output after softmax. , which is defined as follows.

[0131]

[0132] in Output results for the network model, For each category of output, is the total number of categories, Represents the temperature hyperparameter, which is used to control the probability distribution of the model output. The larger the value, the smoother the output probability distribution. The smaller the value, the more concentrated the output probability distribution.

[0133] The present invention uses the output of the original network teacher net as a soft target and calculates the cross entropy loss with the output of the student net. At the same time, the real label of the satellite image is used , use it as a hard target and the output of the student net to calculate the cross entropy loss, and update the parameters of the student net through the back propagation algorithm. The calculation formula is as follows:

[0134]

[0135] in, represents the total loss function of the distillation part, 、 represent hard distillation loss and soft distillation loss respectively, Represents the weight parameter.

[0136] Because current satellite imagery object detection models have numerous parameters and are difficult to deploy on resource-constrained satellites for edge computing, this paper proposes a lightweight object detection model approach for onboard edge computing. This method, while using a compressed model to enable lightweight deployment in an onboard edge computing environment, maintains detection speed and accuracy, and enhances the ability to identify objects with missing features.

[0137] During the model construction process, the present invention proposes a method for constructing an object detection model based on multi-scale feature fusion. This method can detect objects in satellite imagery that are deformed, displaced, or have incomplete features, effectively improving the accuracy and recall of object recognition and detection after model compression.

[0138] During the model compression process, this paper proposes a neural network model compression method based on channel pruning. This method identifies and removes redundant channels in the neural network model, reducing the number of model parameters and improving the model compression ratio. By focusing on pruning redundant model channels, this method can achieve a balance between detection accuracy and model parameter count.

[0139] During the model performance optimization process, this paper proposes a data augmentation-based model performance optimization method. This method uses the uncompressed model as the discriminator in a generative adversarial network to generate a feature-enhanced sample set, improving the model's accuracy in detecting feature-missing objects. This method also applies a knowledge distillation algorithm, enabling the compressed model to effectively learn from feature recognition experience. This effectively improves the lightweight model's object detection accuracy, bringing its performance close to that of the uncompressed model.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lightweight target detection model method for space-borne edge computing, characterized in that: The steps include: S1. Train an image object detection model that can identify feature-missing objects by collecting training datasets, performing self-supervised contrastive training, and building a channel attention mechanism. S1 specifically includes the following steps: S11, collecting original images taken by the satellite, screening images with the target to be identified from the original images, cropping the images of the target to be identified, and using the cropped images as a pre-detection data set; S12, input the original image into the contrastive learning framework for pre-detection, and output the category of each image; S13. Based on the category of each image, an image target detection model is built based on the YOLOX target detection framework. The structure of the image target detection model includes a backbone network, a path aggregation network PAN and a YOLOX decoupling head network; The original image is input into the backbone network. The basic structure of the backbone network includes five different stages, C1 to C5. Each stage can extract the features of the input image and output feature maps of different sizes. The feature maps output by C3 to C5 are input into the PAN network. The feature map output by C5 is first passed through The convolution adjusts the number of channels to C, and after upsampling, it is spliced with the feature map output by C4 in the channel dimension; the channel segmentation and aggregation module is used to suppress the background part of the image and highlight the foreground part; the channel segmentation and aggregation module first divides the input feature map in the channel dimension to obtain two features of size feature map, one part of the feature map is directly spliced with the other part after two depth-wise convolutions; a channel attention module is introduced between the two depth-wise convolutions; the feature map after splicing according to the channel dimension is passed again Convolution is used to fuse information between different channels. For the bottom-up part of the PAN network, M3 and M4 are downsampled by 2 times and then concatenated with M4 and M5 respectively. After passing through the CSA module again, the outputs P3, P4, and P5 are obtained. The fused feature maps of different sizes are input into the YOLOX decoupling head network to classify and locate the objects contained therein. In the channel attention module, the input feature map first aggregates global spatial features through global average pooling, and then completes information interaction between different channels through group convolution. The number of groupings can be freely selected according to the number of input channels. After passing the Sigmoid activation function, it is multiplied with the input. The P3, P4, and P5 feature maps are input into the YOLOX decoupling head network respectively. The decoupling head network extracts the target position and category information respectively, and learns them separately through different convolutional neural network branches, and finally performs feature fusion. An additional angle parameter is introduced in the regression branch to convert the anchor box coordinates from (center point, width, height) to (center point, width, height, rotation angle) to achieve prediction of rotated targets. S2. Use a graph structure to abstract the target detection model into a neural network structure graph, identify and prune redundant channels to compress the neural network model, and obtain a compressed model. S3. Use contrastive generative adversarial networks and knowledge distillation methods to optimize the performance of the compression model and obtain a lightweight target detection model.

2. The method for lightweight target detection model for space-based edge computing according to claim 1, characterized in that: S12 specifically includes the following steps: S121, use data enhancement method to transform the input original image to obtain two similar images And pass through the encoder Encoder to output the feature map , the encoder architecture is consistent and shares parameters; S122, feature map Obtained by feature transformation through the mapping head ; S123, by predicting the head Make a prediction ,calculate and The cosine similarity of .

3. The method for lightweight target detection model for space-based edge computing according to claim 1, characterized in that: The contrastive learning framework in S12 includes data augmentation network structure, encoder, projection network structure and prediction network structure; The data enhancement network structure Augmentation is used to expand the data set and transform the input image using random cropping and horizontal flipping methods to obtain two similar images And input them into the encoder respectively; The encoder uses the ResNet50 network model; The Projection network structure consists of a fully connected layer, a BN layer, and a ReLu activation function. It achieves correct image classification by fusing the encoder output features. The two Projection architectures are the same. The prediction network structure consists of a fully connected layer, a batch normalization layer, and a ReLu activation function. By further transforming the output of the projection network, the cosine similarity between its output and the output of another projection network is calculated, and this is used as the loss to train the encoder through back propagation.

4. The method for lightweight target detection model for space-based edge computing according to claim 1, characterized in that: S2 specifically includes the following steps: S21. Abstracting the neurons of the image target detection model as nodes, abstracting the transmission relationship between neurons as edges, and constructing a neural network structure diagram with several layers of convolutional layers. The neural network structure diagram is stored in the form of a matrix; S22. Prune the convolution kernels in the convolution layer of the image object detection model based on the importance and feature maps of the convolution kernels themselves; S23. The connections between neurons in different layers are channels. The similarity of the channels is calculated, and the redundant channels in the image target detection model are pruned based on the similarity of the channels.

5. The method for lightweight target detection model for space-based edge computing according to claim 4, characterized in that: S22 specifically includes the following steps: S221. From the perspective of the importance of the convolution kernel itself, directly prune the convolution kernel in the convolution layer; for a Layer convolutional layer network, assuming that the first Convolutional layer Pruning is performed, wherein Represents the total number of convolution kernel, that is output channels, and the number of input channels of each convolution kernel is , the size is ; First, calculate each convolution kernel in this layer The L1 norm of , then sort the norms of these convolution kernels from large to small, and according to the pre-set pruning rate , after removing the soft pruning method The convolution kernels are arranged from large to small from front to back; the soft pruning method is to set the convolution kernel to be pruned to 0; S222. From the perspective of feature maps, the network is further pruned by judging the correlation between convolution kernels; for a The network at layer In the layer, Represents the total number of feature maps, Represents the height and width of the feature map, that is, the size of each feature map is ; By calculating the importance of each feature map, sorting them in descending order of importance, and removing the most important ones % feature map, and also cut off the corresponding convolution kernel. The calculation formula for calculating the importance of each feature map is as follows: in, Represents the importance of each feature map, That is, convert the three-dimensional feature map into the corresponding feature matrix after two-dimensional, represents the mask matrix, the matrix The first row is 0, and the rest of the rows are 1. It represents the SVD decomposition of the matrix, and its formula is as follows: in, represents the singular values of the matrix, are the left and right singular value vectors respectively, the matrix The rank of ; Change the eigenvalue Arrange from large to small and remove the ones with the largest change % feature map, and the corresponding filters are also cut off accordingly.

6. The method for lightweight target detection model for space-based edge computing according to claim 4, characterized in that: S23 includes the following steps: The structural similarity of the channel is calculated based on the interaction relationship between neurons. The calculation formula for the interaction relationship between neurons is: in, is the node contribution, is the node forward contribution, is the node backward contribution, is the forward node set, is the backward node set; Based on the calculated contribution of each node, the similarity between channels is calculated. The formula for calculating the similarity of channels is: in, It is a channel With channel The similarity of is the structural similarity, Contribute similarity to the node, and Calculate the channel from the input and output and hierarchical structure With channel The structural similarity of For the Layer The node contribution of each node; Arrange the channel overlap from high to low, and put the overlap at the end The channels with only a single branch are identified as redundant channels and are pruned from the network. For those channels that have only a single branch selected for removal, their parameters are restored in the subsequent network iterative update process to participate in the next training process. The joint dynamic channel pruning algorithm is iterative until the pruning rate of the network reaches the target pruning rate. Stop when 7. The method for lightweight target detection model for space-based edge computing according to claim 1, characterized in that: S3 specifically includes the following steps: S31. Constructing a contrastive generative adversarial network; S32. Generate feature-enhanced datasets using contrastive generative adversarial networks; S33. Knowledge Distillation: Fine-tuning compressed models using feature-augmented datasets.

8. The method for lightweight target detection model for space-based edge computing according to claim 7, characterized in that: In S31, the contrastive generative adversarial network uses the target detection model as a discriminator, and the optimization function formula of the contrastive generative adversarial network is: in, is the distribution of real data, is the noise distribution, is the output of the discriminator, is the output of the generator; the loss functions of the generator and the discriminator are: in, is the traditional discriminator loss function, To generate the similarity between the sample and the original sample, For expected similarity, the similarity between the generated sample and the original sample is expected to be , by adjusting The feature richness of generated samples can be changed, is the generation loss of similar data, is the generation loss of heterogeneous data, For variable parameters.

9. The method for lightweight target detection model for space-based edge computing according to claim 7, characterized in that: S33 specifically includes the following steps: The feature-enhanced image is input into the original network teacher net and the pruned lightweight network studentnet respectively, and the classification probability is output after softmax. , The definition is as follows: in, Output results for the network model, For each category of output, is the total number of categories; represents the temperature hyperparameter, which is used to control the probability distribution of the model output; The larger the value, the smoother the output probability distribution. The smaller the value, the more concentrated the output probability distribution; The output of the original network teacher net is used as the soft target, and the cross entropy loss is calculated by combining the soft target with the output of the student net. At the same time, the real label of the satellite image is used. The cross entropy loss is calculated as the output of the hard target and the student net, and the parameters of the student net are updated through the back propagation algorithm. The calculation formula of the back propagation algorithm is as follows: in, represents the total loss function of the distillation part, represents the hard distillation loss, represents the soft distillation loss, Represents the weight parameter.

Images