Spatial constraint-based remote sensing image classification model structure search method

Abstract

The invention provides a spatial constraint-based remote sensing image classification model structure search method, which comprises the following steps of: constructing a search network and initializing network parameters, obtaining a search target function, obtaining a training sample set to pre-train the search network, and constraining the search network by adopting set spatial constraint parameters, and solving an optimal spatial constraint parameter by adopting a stochastic gradient descent algorithm, creating a constraint mask to perform constraint updating on a plurality of sub neural network models, updating a non-zero weight parameter of the search network, performing early stop detection based on an early stop rule, stopping search when the early stop rule is consistent, and updating an architecture parameter for performing comprehensive training and verification on the search model. And returning to the sample set training step for iterative loop until a maximum iterative parameter is reached, obtaining a target architecture parameter configuration search network, and obtaining a target classification model. According to the method, the search range can be effectively narrowed, the computing resource occupation and the computing complexity are reduced, and low-cost deployment is facilitated.

Description

Technical Field

[0001] The invention relates to the technical field of remote sensing image classification, and in particular to a remote sensing image classification model structure search method based on space constraints. Background Art

[0002] With the rapid development of remote sensing technology, remote sensing images can be obtained more conveniently and play an important role in many fields such as land resource management, environmental monitoring, urban planning and military. Remote sensing image classification, as a core link of remote sensing image analysis, is crucial to improving work efficiency and decision-making quality in these fields.

[0003] Deep learning methods have achieved remarkable success in remote sensing image classification tasks. These methods can automatically identify and learn complex patterns in images, greatly improving the accuracy and efficiency of classification. However, remote sensing image classification still faces a series of challenges. The classification task becomes extremely difficult due to the huge differences in size and morphology of high-resolution remote sensing images in complex scenes. These images usually contain a lot of information, from terrain features to different types of surface cover, each with its own unique visual characteristics, requiring the classification model to capture subtle differences.

[0004] In addition, remote sensing equipment carried by satellite and airborne platforms is subject to strict weight and volume restrictions, which limits the ability to carry computing resources. Therefore, although current deep learning models perform well when running on servers, these models often require a large amount of computing resources, making them difficult to apply directly to edge devices, such as remote sensing image processing on satellites or drones. In addition, the high resource consumption during the model search process and the cost of deploying these high-quality models on edge devices have also greatly limited the development of remote sensing image classification technology.

[0005] Therefore, there is an urgent need for a remote sensing image classification method that can narrow the search scope, reduce the model calculation complexity and computing resource occupancy to achieve low-cost deployment. Summary of the invention

[0006] Based on this, it is necessary to provide a remote sensing image classification model structure search method based on spatial constraints to address the above technical problems.

[0007] A remote sensing image classification model structure search method based on spatial constraints includes the following steps: S1, constructing a search network based on a neural network, the search network includes multiple sub-neural network models, initializing the parameters of the search network, and obtaining a search objective function; S2, obtaining a training sample set, pre-training the search network through the training sample set, and constraining the search network with a set spatial constraint parameter; S3, solving the optimal spatial constraint parameter using a random gradient descent algorithm, creating a constraint mask based on the optimal spatial constraint parameter, and performing constraint update on the multiple sub-neural network models through the constraint mask; S4, updating the non-zero weight parameters of the search network using a random gradient descent algorithm, and performing early stopping detection based on a set early stopping rule, and stopping the search process when the early stopping rule is met; S5, updating the architecture parameters of the search network using a random gradient descent algorithm, and using them to perform comprehensive training and verification of the search network; S6, returning to step S2 for an iterative loop until the number of iterations meets the preset maximum iteration parameter, obtaining the target architecture parameter, and obtaining the target classification model based on the target architecture parameter and the search network for remote sensing image classification.

[0008] In one embodiment, step S1 includes: constructing a search network based on a neural network, wherein the search network includes a plurality of sub-neural network models; initializing the architecture parameters, weight parameters, loss function strength parameters, learning rate, and maximum iteration parameters of the search network to obtain a search objective function, which is:

[0009]

[0010] In the formula, θ ＊ (α) is the pre-trained weight parameter, θ is the weight corresponding to the i-th layer network, is the loss function of the weight parameters and architecture parameters in the pre-training stage, α is the architecture parameter, is the constraint mask corresponding to the optimal spatial constraint parameter, m is the constraint mask, N is the constraint mask length, is the loss function of the weight parameters and architecture parameters in the spatial constraint stage, is the loss function of the weight parameters and architecture parameters in the fine-tuning stage, is the weight parameter after fine-tuning.

[0011] In one embodiment, step S2 includes: obtaining a training sample set, wherein the training sample set includes a plurality of remote sensing sample images, preprocessing all remote sensing sample images in the training sample set, and using the preprocessed training sample set to pretrain the plurality of sub-neural network models; initializing the set spatial constraint parameters, wherein the spatial constraint parameters are:

[0012]

[0013] In the formula, is the spatial constraint parameter, θ i is the weight corresponding to the i-th layer in the search network; constrain the search network in training based on the initialized spatial constraint parameters; iteratively use the stochastic gradient descent algorithm to update the weight parameters of the search network, and the formula is:

[0014]

[0015] In the formula, is the weight parameter for the t+1th iteration, is the weight parameter of the tth iteration, is the pre-training loss function of weight parameters and architecture parameters, is the learning rate of the weight parameter, Represents the pre-training loss function About the weight parameter θ ＊ The gradient of t are the architecture parameters of the tth iteration.

[0016] In one embodiment, step S3 includes: using a stochastic gradient descent method to find the optimal spatial constraint parameter, the formula is:

[0017]

[0018] In the formula, S t+1 is the optimal spatial constraint parameter for the t+1th iteration, S t is the spatial constraint parameter of the tth iteration, η s is the learning rate of the spatial constraint parameters, is the loss function of the spatial constraint parameters and the architecture parameters, Represents the loss function Regarding the gradient of the spatial constraint parameter s; creating a constraint mask according to the optimal spatial constraint parameter, and performing constraint updates on the multiple sub-neural network models based on the constraint mask.

[0019] In one embodiment, step S4 includes: updating the non-zero weight parameters of the search network using a stochastic gradient descent algorithm, the formula is:

[0020]

[0021] In the formula, is the non-zero weight parameter of the t+1th iteration, is the non-zero weight parameter of the tth iteration, is the learning rate of non-zero weight parameters, To fine-tune the loss function, Represents the fine-tuning loss function About non-zero weight parameters gradient; performing early stopping detection based on a set early stopping rule, and when it is detected that the number of jump connections in a unit of the search network is greater than or equal to the set number, it is determined that the early stopping rule is met and the search process is stopped.

[0022] In one of the embodiments, the early stopping rule also includes: detecting the degree of change in the ranking of the architecture parameters within a set continuous period. If the degree of change in the ranking of the architecture parameters within the set continuous period is lower than a set degree threshold, it is determined that the search process is saturated and meets the early stopping rule, and the search process is stopped.

[0023] In one embodiment, step S5 includes: using a stochastic gradient descent algorithm to update the search network architecture parameters, the formula is:

[0024]

[0025] In the formula, α t+1 is the architecture parameter of the t+1th iteration, α t is the architecture parameter of the tth iteration, η α is the learning rate of the architecture parameter α, is the validation loss function of the fine-tuned weight parameters and architecture parameters, Represents the validation loss function About the architecture parameter α t ; traverse the search network and set the proportion of corresponding modules according to the updated architecture parameters; iteratively obtain multiple remote sensing sample images based on the training sample set, and the multiple remote sensing sample images include input features and target labels; input the input features to the search network after the proportion setting to obtain the predicted output; calculate the cross entropy loss according to the predicted output and the corresponding target label; perform back propagation according to the cross entropy loss to calculate the gradient of the search network parameters relative to the loss; clip the gradient, update the weights using the stochastic gradient descent method, and adjust the learning rate based on the cosine annealing scheduler.

[0026] In one of the embodiments, after step S6, it also includes: fine-tuning the target classification model and target architecture parameters by combining network pruning and knowledge distillation to obtain an edge network model, and the edge network model can be deployed in an edge device.

[0027] In one of the embodiments, the target classification model and target architecture parameters are fine-tuned by combining network pruning and knowledge distillation to obtain an edge network model, including: obtaining available resources of edge devices and occupied resources of the target classification model; calculating the difference between the occupied resources and the available resources, and when the difference is greater than 0, using the ratio of the difference to the occupied resources as the pruning ratio, and pruning the target classification model by a structured pruning method; based on the knowledge distillation method, using the model before pruning to train the model after pruning to obtain the edge network model.

[0028] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows: a search network including multiple sub-neural networks is constructed through a neural network, the parameters of the search network are initialized, the search objective function is obtained, a training sample set is obtained for pre-training the search network, and the search network is constrained by the set spatial constraint parameters, thereby effectively narrowing the search range; a stochastic gradient descent algorithm is used to solve the optimal spatial constraint parameters, and a constraint mask is created based on the optimal spatial constraint parameters to constrain multiple sub-neural network models; a stochastic gradient descent algorithm is used to update the non-zero weight parameters of the search network, and early stopping detection is performed based on the set early stopping rule. The search process is stopped when the early stopping rule is met to achieve optimization of the evaluation of the search network performance during the training process, saving a lot of computing resources and time; the stochastic gradient descent algorithm is used to update the architecture parameters and is used for comprehensive training and verification of the search network. The search network is iteratively trained multiple times through the above steps until the maximum iteration parameter setting is met to obtain the target architecture parameters. Based on the target architecture parameters and the search network, the target classification model is obtained for remote sensing image classification, thereby achieving further constraints on the search network without losing accuracy, effectively reducing the computational complexity and the amount of model parameters, and facilitating low-cost deployment of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of a flow chart of a remote sensing image classification model structure search method based on spatial constraints in one embodiment;

[0030] Figure 2 FIG. 4 is a schematic diagram of an early stopping operation in one embodiment. DETAILED DESCRIPTION

[0031] Before describing the specific embodiments of the present invention, the overall concept of the present invention is described as follows:

[0032] The present invention is mainly developed for the lightweight process of network models in the field of remote sensing image classification. Currently, model search occupies a large amount of computing resources and has a high deployment cost.

[0033] Therefore, the present invention proposes a remote sensing image classification model structure search method based on spatial constraints. By constraining the search space through spatial constraint parameters, the search range can be effectively narrowed for the depth, width and specific types of connection patterns of the network. By monitoring the performance improvement of the network during training to optimize the evaluation method and terminating the training of network modules with more than two jump connections in advance, a large amount of computing resources and time can be saved. In view of the characteristics of remote sensing images, interval boundary propagation is applied to enhance the robustness of the network, and the disturbance in the hybrid training strategy is increased to enhance the adversarial resistance. Without losing accuracy, further constraints on the search network are completed, which effectively reduces the computational complexity and the amount of model parameters, facilitates the efficient deployment of the model on edge devices, and can reduce deployment costs.

[0034] After introducing the overall concept of the present invention, in order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail by specific implementation methods in combination with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0035] In one embodiment, Figure 1 As shown, a remote sensing image classification model structure search method based on spatial constraints is provided, comprising the following steps:

[0036] Step S101, constructing a search network based on a neural network, the search network includes a plurality of sub-neural network models, initializing the parameters of the search network, and obtaining a search target function.

[0037] Specifically, a search network is constructed, the search network includes a plurality of sub-neural network models, and the parameters of the constructed search network are initialized to obtain a search objective function.

[0038] Wherein, step S101 includes: constructing a search network based on a neural network, the search network includes multiple sub-neural network models; initializing the architecture parameters, weight parameters, loss function strength parameters, learning rate and maximum iteration parameters of the search network, and obtaining the search objective function, which is:

[0039]

[0040] In the formula, θ ＊ (α) is the pre-trained weight parameter, θ is the weight corresponding to the i-th layer network, is the loss function of the weight parameters and architecture parameters in the pre-training stage, β is the architecture parameter, is the constraint mask corresponding to the optimal spatial constraint parameter, m is the constraint mask, N is the constraint mask length, is the loss function of the weight parameters and architecture parameters in the spatial constraint stage, is the loss function of the weight parameters and architecture parameters in the fine-tuning stage, is the weight parameter after fine-tuning.

[0041] Specifically, a search network is constructed based on a neural network deep learning algorithm to obtain a search network including multiple sub-neural network models. The parameters of the search network are initialized, including architecture parameters, weight parameters, strength parameters for controlling a 0-1 loss function, a learning rate, and a maximum iteration parameter, so as to obtain a search objective function corresponding to the search network.

[0042] Since the search network training process in the present application includes four stages: pre-training, constraint, fine-tuning and training, corresponding conditional functions are set for the pre-training, constraint and fine-tuning stages respectively, and the training stage of the search network is implemented by repeatedly iterating pre-training, constraint and fine-tuning to obtain the corresponding search objective function.

[0043] Step S102, obtaining a training sample set, pre-training a search network through the training sample set, and constraining the search network using set spatial constraint parameters.

[0044] Specifically, a training sample set is obtained, a constructed search network is pre-trained through the training sample set, and a spatial constraint parameter is set to constrain the search network in training, thereby effectively reducing the search range and improving the search efficiency.

[0045] Wherein, step S102 includes: obtaining a training sample set, the training sample set includes a plurality of remote sensing sample images, preprocessing all remote sensing sample images in the training sample set, and using the preprocessed training sample set to pretrain a plurality of sub-neural network models; initializing the set spatial constraint parameters, the spatial constraint parameters are:

[0046]

[0047] In the formula, is the spatial constraint parameter, θ i is the weight corresponding to the i-th layer in the search network; constrain the search network in training based on the initialized spatial constraint parameters; iteratively use the stochastic gradient descent algorithm to update the weight parameters of the search network, and the formula is:

[0048]

[0049] In the formula, is the weight parameter for the t+1th iteration, is the weight parameter of the tth iteration, is the pre-training loss function of weight parameters and architecture parameters, is the learning rate of the weight parameter, Represents the pre-training loss function About the weight parameter θ ＊ The gradient of t is the architecture parameter of the tth iteration.

[0050] Specifically, a training sample set including multiple remote sensing sample images is obtained, and all remote sensing sample images are preprocessed, such as image enhancement, image calibration, edge detection, normalization, etc., and the preprocessed training sample set is used to pretrain multiple sub-neural networks of the search network, so that the performance of the search network trained with the image is better. The set spatial constraint parameters are initialized and used to spatially constrain the search network in training. At the same time, the weight parameters in the search network are updated through multiple iterations using the stochastic gradient descent algorithm.

[0051] Among them, the selection definition subnet of the search network is a customized autograd.Function, which is used to select a subset of weights in the forward propagation and allow the gradient to be broadcast live in the back propagation, that is, the gradient is not modified during the back propagation. Propagation method: The calculation logic is to sort the constraint parameters and select the weight corresponding to the highest spatial constraint parameter as the active weight (the rest are set to 0). Backward propagation method: Simply return the incoming gradient without any modification. Therefore, during the back propagation process, all weights will receive gradient information, but only the selected weights will be used in the forward propagation.

[0052] Among them, the cell class in the network is responsible for processing inputs and generating outputs, and is dynamically constructed according to the given genotype parameter. The parameter initialization method is: select different preprocessing methods according to the reduction_prev parameter to process the output of the previous unit. According to the reduction parameter, this unit is determined to be a normal unit or a reduced dimension unit, and the corresponding operations and connections are obtained from the genotype. The Compile method builds the operation module in the current unit by parsing the operation name and index defined in the genotype. According to the forward propagation: preprocess the inputs S0 and S1. Traverse all the steps defined by the current unit, process two inputs through two operations in each step, and add the results. If the upper and lower limits are defined during training, drop path regularization is applied to the outputs of non-identical operations. Finally, according to the _concat index list, merge specific state outputs. The input data first passes through the starting layer and then passes through all units in turn. After passing through a unit, update the state variables S0 and S1. If the auxiliary head is reached and is in training mode, an additional output will be generated from the auxiliary head. Finally, each cell processes alphas through the softmax function to control the selection of operations. The final output is generated through global pooling and the main classifier.

[0053] The parameters of the search network also have different states at different stages of training, such as constraints, pre-training, and fine-tuning, including freezing or unfreezing, to facilitate the implementation of network constraint strategies and effective network training adjustments. In the pre-training stage of the search network, the weights, biases, and architecture parameters are unfrozen for training, while the constraint scores are frozen. This stage may also freeze the parameters of the batch normalization layer, depending on the freeze_bn parameter.

[0054] Step S103, using the stochastic gradient descent method to find the optimal spatial constraint parameters, and creating a constraint mask based on the optimal spatial constraint parameters, and performing constraint update on multiple sub-neural network models through the constraint mask.

[0055] Specifically, the stochastic gradient descent method is used to train the spatial network constraints, find the optimal spatial constraint parameters, and create a constraint mask based on the optimal spatial constraint parameters. The constraint mask is used to update the constraints of multiple sub-neural network models. The mask learning is carried out synchronously with the network training, and the redundant weight parameters in the candidate operations are removed through the learned constraint mask.

[0056] In neural networks, constraint masks are used to control the usage of various parameters in the network. They are usually a binary matrix of the same shape as the weight parameters, where the value of each element is 0 or 1. Elements with a value of 1 indicate that the corresponding weight is active in the network, while elements with a value of 0 indicate that the corresponding weight is masked or ignored in forward and backward propagation.

[0057] After removing the unimportant weights in the training network, in order to continue to obtain the values ​​of the spatial constraint parameters, the formula is solved by the stochastic gradient descent method:

[0058]

[0059] In the formula, θ pre and are the parameters and mathematical expectations of the pre-trained network respectively.

[0060] Wherein, step S103 includes: using the stochastic gradient descent method to find the optimal space constraint parameter, the formula is:

[0061]

[0062] In the formula, s t+1 is the optimal spatial constraint parameter for the t+1th iteration, S t is the spatial constraint parameter of the tth iteration, η s is the learning rate of the spatial constraint parameters, is the loss function of the spatial constraint parameters and the architecture parameters, Represents the loss function The gradient of the spatial constraint parameter s is calculated; a constraint mask is created according to the optimal spatial constraint parameter, and multiple sub-neural network models are constrained and updated based on the constraint mask.

[0063] Specifically, step S103 is the network constraint stage. When performing network training in this stage, the constraint score needs to be updated. Therefore, the constraint score is unfrozen, and the weights, biases and architecture parameters are frozen at the same time. If the parameter freeze_bn is true, the parameters of the batch normalization layer are also frozen.

[0064] Step S104, using a stochastic gradient descent algorithm to update the non-zero weight parameters of the search network, and performing early stopping detection based on a set early stopping rule, and stopping the search process when the early stopping rule is met.

[0065] Specifically, the search network after network constraints needs to be fine-tuned to restore and improve performance. The stochastic gradient descent algorithm is used to update the non-zero weight parameters in the search network to improve the accuracy of the network. The search network in training is tested for early stopping based on the set early stopping rules. When the early stopping rules are met, the search process is stopped and the current candidate module is skipped.

[0066] Wherein, step S104 includes: using a stochastic gradient descent algorithm to update the weight parameters of the search network, the formula is:

[0067]

[0068] In the formula, is the non-zero weight parameter of the t+1th iteration, is the non-zero weight parameter of the tth iteration, is the learning rate of non-zero weight parameters, To fine-tune the loss function, Represents the fine-tuning loss function About non-zero weight parameters The gradient of the search network is determined by the early stopping rule. The search process is stopped when the number of jump connections in a unit of the search network is greater than or equal to the set number.

[0069] Specifically, in the fine-tuning stage, the stochastic gradient descent algorithm is used to update non-zero weights to improve network accuracy, and when it is detected that the number of jump connections in a unit of the search network is equal to or greater than 2, the search process stops and the current candidate module is skipped.

[0070] The preparation of the search network during the fine-tuning phase is similar to training, unfreezing the weights, biases, and architecture parameters, freezing the constraint scores, and optionally freezing the parameters of the batch normalization layer.

[0071] Among them, the early stopping rule also includes: detecting the degree of change in the order of the architecture parameters within a set continuous period. If the degree of change in the order of the architecture parameters within the set continuous period is lower than a set degree threshold, it is determined that the search process is saturated and meets the early stopping rule, and the search process is stopped.

[0072] Specifically, in order to further improve the early stopping rule, a supplementary early stopping rule based on the stability of the architecture parameters is introduced. When the order of the architecture parameters of the learnable operation remains stable within a certain number of training cycles, such as 10 cycles, the search process is considered to be close to saturation. At this time, the search is terminated to prevent further performance degradation.

[0073] In the search phase, strategy configuration is performed through interval-bounded propagation, aiming to improve the overall robustness of the search network without excessively sacrificing the performance of the search network, which helps to improve the performance of remote sensing image classification.

[0074] In terms of performance evaluation, at the initial stage of the search process, the training data of a model will be underfitted, so the architecture parameters and weight parameters, which are learnable parameters of a model, will become better together at the beginning of the search process, that is, the cooperation period.

[0075] It is important to note that the first unit of the entire model has access to accurate data information, while the data passed to the last unit is much noisier.

[0076] If different units are allowed to have different architectures in a single-shot model, the first unit will learn features faster than the last unit. Since the feature representation learned by the last unit is relatively poorer than that learned by the first unit, the last unit is more likely to choose more skip connections to obtain better feature representations directly from the first unit.

[0077] As the number of searches increases, the skip connections in the selected architecture will grow significantly. During this period, the architecture becomes worse, resulting in a competition between the architecture parameters and the weight parameters, causing the search network to collapse.

[0078] Therefore, this application sets an early stopping rule for this situation. When a situation that meets the early stopping rule is detected, the search operation is stopped. The early stopping operation is as follows: Figure 2As shown. When two or more skip connections appear in a unit during the search process, the search network begins to show a tendency to collapse, that is, the performance drops sharply. Set an early stopping criterion, that is, terminate the search process when the above conditions are met. This early stopping rule can not only simplify the implementation of the search network, but also significantly improve the performance of the search network and reduce the time required for the search. In addition, the early stopping operation directly controls the number of skip connections during the search process to more effectively avoid collapse. Define the early stopping criterion based on the number of skip connections, the formula is:

[0079] ∑ i∈Cell l(op i = "skip-connect")≥2,

[0080] where i traverses all operations in a unit, l(·) is an indicator function that is 1 when the operation is “skip_connect” and 0 otherwise, and the search process stops when the number of skip connections in a unit is equal to or greater than 2.

[0081] Furthermore, a supplementary early stopping criterion based on the stability of architectural parameters is introduced, and the formula is:

[0082] S(α i , T) ≥ Threshold,

[0083] where α i is an operation that sorts according to the structural parameter α, T is the number of training cycles examined, and S(·) measures the degree of change of these sortings within T consecutive cycles. When this stability measure exceeds a preset threshold, early stopping is triggered.

[0084] Step S105, using a stochastic gradient descent algorithm to update the architecture parameters of the search network, and to perform comprehensive training and verification of the search network.

[0085] Specifically, in order to comprehensively train and verify the search network, a stochastic gradient descent algorithm is used to update the architecture parameters of the search network, thereby improving the performance of the search network.

[0086] Wherein, step S105 includes: using a stochastic gradient descent algorithm to update the search network architecture parameters, the formula is:

[0087]

[0088] In the formula, α t+1 is the architecture parameter of the t+1th iteration, α t is the architecture parameter of the tth iteration, η α is the learning rate of the architecture parameter α, is the validation loss function of the fine-tuned weight parameters and architecture parameters, Represents the validation loss function About the architecture parameter α t The method comprises the following steps: 1. The method comprises the following steps: 1. traverse the search network and set the proportion of the corresponding modules according to the updated architecture parameters; 2. iteratively obtain multiple remote sensing sample images based on the training sample set, and the multiple remote sensing sample images include input features and target labels; 3. input the input features to the search network after the proportion setting to obtain the predicted output; 4. calculate the cross entropy loss according to the predicted output and the corresponding target label; 5. perform back propagation according to the cross entropy loss and calculate the gradient of the search network parameters relative to the loss; 6. clip the gradient, update the weights using the stochastic gradient descent method, and adjust the learning rate based on the cosine annealing scheduler.

[0089] Specifically, after obtaining the updated non-zero weight parameters in step S104, the stochastic gradient descent algorithm is used to perform the upper optimization problem to update the architecture parameters. The parameters of the search network are all unfrozen in step S105, including the constraint scores, weights, biases and architecture parameters, so as to fully train the search network through the updated architecture parameters.

[0090] The function is set to traverse all modules of the search network, and the corresponding proportions are set according to the architecture parameters and space constraint parameters, and the training and verification cycle is performed in step S105. Iteratively obtain batches of remote sensing sample images from the training sample set, which include input features and target labels; input the input features to the search network, obtain the predicted output through the forward propagation of the search network, and calculate the cross entropy loss based on the predicted data and the corresponding target label; perform back propagation according to the cross entropy loss function, calculate the gradient of the search network parameters relative to the loss, and in order to prevent the gradient from exploding, the gradient can be clipped; use the stochastic gradient descent method to update the weight parameters to reduce the loss; use the cosine annealing scheduler to adjust the learning rate. With the support of multiple strategies, different training strategies are integrated, such as adversarial training, random smoothing, and IBP (IntervalBound Propagation), to improve the robustness and accuracy of the search network, and each strategy is optimized for specific security or performance requirements.

[0091] Step S106, return to step S102 to perform an iterative loop until the number of iterations meets the preset maximum iteration parameter, obtain the target architecture parameters, and obtain the target classification model based on the target architecture parameters and the search network for remote sensing image classification.

[0092] Specifically, return to step S102 to perform an iterative loop until the number of iterations meets the setting of the preset maximum iteration parameter, obtain the target architecture parameters of the search network, discretize the target architecture parameters, and configure the search network parameters according to the discretized target architecture parameters to obtain a target classification model for remote sensing image classification, thereby reducing the computational complexity and computing resource usage, so as to facilitate the deployment of the model to the edge device and reduce the deployment cost.

[0093] In this embodiment, a search network including multiple sub-neural networks is constructed through a neural network, the parameters of the search network are initialized, the search objective function is obtained, and a training sample set is obtained for pre-training the search network, and the search network is constrained by the set spatial constraint parameters, thereby effectively narrowing the search range; the stochastic gradient descent algorithm is used to solve the optimal spatial constraint parameters, and a constraint mask is created based on the optimal spatial constraint parameters to constrain multiple sub-neural network models; the stochastic gradient descent algorithm is used to update the non-zero weight parameters of the search network, and early stopping detection is performed based on the set early stopping rule, and the search process is stopped when the early stopping rule is met, so as to achieve evaluation and optimization of the performance of the search network during the training process, saving a lot of computing resources and time; the stochastic gradient descent algorithm is used to update the architecture parameters, and is used for comprehensive training and verification of the search network. The search network is iteratively trained for multiple times through the above steps until it meets the setting of the maximum iteration parameters, and the target architecture parameters are obtained. The target classification model is obtained based on the target architecture parameters and the search network, and is used for remote sensing image classification, thereby achieving further constraints on the search network without losing accuracy, effectively reducing the computational complexity and the amount of model parameters, and facilitating low-cost deployment of the model.

[0094] In one embodiment, after step S106, the method further includes: fine-tuning the target classification model and the target architecture parameters by combining network pruning and knowledge distillation to obtain an edge network model, and the edge network model can be deployed in an edge device.

[0095] Specifically, the target classification model obtained in step S106 is a high-performance model that can be used in remote sensing image classification scenarios with sufficient resources and relatively high performance requirements. However, in some application scenarios, the edge device has few available resources, and when it is directly deployed to the edge device, there may be a problem of insufficient resources. Therefore, it is also possible to use a combination of network pruning and knowledge distillation to optimize the structure of the acquired target classification model, fine-tune the model structure and weights, and further reduce the model resource occupancy so that it can be deployed to the edge device.

[0096] Among them, the steps of obtaining the edge network model include: obtaining the available resources of the edge device and the occupied resources of the target classification model; calculating the difference between the occupied resources and the available resources, and when the difference is greater than 0, taking the ratio of the difference to the occupied resources as the pruning ratio, and using a structured pruning method to prune the target classification model; based on the knowledge distillation method, using the model before pruning to train the pruned model to obtain the edge network model.

[0097] Specifically, since the obtained target classification model has high performance and occupies relatively large resources, there may be insufficient resources when it is deployed to the edge device. a and the resource occupied by the target classification model R b , and the available resources R of the edge device a and model occupies resources R b To evaluate, when R b -R a >0, (R b -R a ) / R b The target classification model obtained in the previous step is pruned as the pruning ratio. The structured pruning method is used during pruning, so that the pruned model can achieve effective resource compression effect. In order to make the pruned model maintain higher performance, the pruned model is trained with the model before pruning based on the knowledge distillation method. During distillation training, in order to better maintain the knowledge of the original model, a combination of the intermediate layer and the output layer can be used. After completing the knowledge distillation, the edge network model is obtained and can be deployed to the corresponding edge device, which can further reduce resource usage, facilitate model edge deployment, and can also achieve adaptation to edge devices with different available resources.

[0098] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0099] Obviously, those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a computer storage medium (ROM / RAM, magnetic disk, optical disk) and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Therefore, the present invention is not limited to any specific combination of hardware and software.

[0100] The above contents are further detailed descriptions of the present invention in combination with specific implementation methods, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the scope of protection of the present invention.

Claims

1. A method for searching a remote sensing image classification model structure based on spatial constraints, characterized in that: The following steps are involved: S1. constructing a search network based on a neural network, wherein the search network includes a plurality of sub-neural network models, initializing parameters of the search network, and obtaining a search target function; S2, obtaining a training sample set, pre-training the search network through the training sample set, and constraining the search network using set spatial constraint parameters; S3, using a stochastic gradient descent algorithm to solve the optimal spatial constraint parameters, and creating a constraint mask based on the optimal spatial constraint parameters, and performing constraint update on the multiple sub-neural network models through the constraint mask; S4, using a stochastic gradient descent algorithm to update the non-zero weight parameters of the search network, and performing early stopping detection based on a set early stopping rule, and stopping the search process when the early stopping rule is met; S5. Using a stochastic gradient descent algorithm to update the architecture parameters of the search network, and to perform comprehensive training and verification of the search network; S6. Return to step S2 to perform an iterative loop until the number of iterations meets the preset maximum iteration parameter, obtain the target architecture parameters, and obtain the target classification model based on the target architecture parameters and the search network for remote sensing image classification.

2. The method for searching a remote sensing image classification model structure based on spatial constraints according to claim 1, characterized in that: The step S1 comprises: Constructing a search network based on a neural network, wherein the search network includes a plurality of sub-neural network models; Initialize the architecture parameters, weight parameters, loss function strength parameters, learning rate and maximum iteration parameters of the search network to obtain the search objective function, which is: In the formula, θ ＊ (α) is the pre-trained weight parameter, θ is the weight corresponding to the i-th layer network, is the loss function of the weight parameters and architecture parameters in the pre-training stage, α is the architecture parameter, is the constraint mask corresponding to the optimal spatial constraint parameter, m is the constraint mask, N is the constraint mask length, is the loss function of the weight parameters and architecture parameters in the spatial constraint stage, is the loss function of the weight parameters and architecture parameters in the fine-tuning stage, is the weight parameter after fine-tuning.

3. The method for searching a remote sensing image classification model structure based on spatial constraints according to claim 2, characterized in that: The step S2 comprises: Acquire a training sample set, wherein the training sample set includes a plurality of remote sensing sample images, preprocess all the remote sensing sample images in the training sample set, and use the preprocessed training sample set to pretrain the plurality of sub-neural network models; Initialize the set space constraint parameters, the space constraint parameters are: In the formula, is the spatial constraint parameter, θ i is the weight corresponding to the i-th layer in the search network; Constrain the search network in training based on the initialized spatial constraint parameters; The weight parameters of the search network are updated iteratively using the stochastic gradient descent algorithm, and the formula is: In the formula, is the weight parameter for the t+1th iteration, is the weight parameter of the tth iteration, is the pre-training loss function of weight parameters and architecture parameters, is the learning rate of the weight parameter, Represents the pre-training loss function About the weight parameter θ * The gradient of t is the architecture parameter of the tth iteration.

4. The method for searching a remote sensing image classification model structure based on spatial constraints according to claim 3, characterized in that: The step S3 comprises: The stochastic gradient descent method is used to find the optimal spatial constraint parameters, and the formula is: In the formula, S t+1 is the optimal spatial constraint parameter for the t+1th iteration, S t is the spatial constraint parameter of the tth iteration, η s is the learning rate of the spatial constraint parameters, is the loss function of the spatial constraint parameters and the architecture parameters, Represents the loss function The gradient with respect to the spatial constraint parameter s; A constraint mask is created according to the optimal spatial constraint parameters, and constraint updates are performed on the multiple sub-neural network models based on the constraint mask.

5. The method for searching a remote sensing image classification model structure based on spatial constraints according to claim 4, characterized in that: The step S4 comprises: The non-zero weight parameters of the search network are updated using the stochastic gradient descent algorithm, and the formula is: In the formula, is the non-zero weight parameter of the t+1th iteration, is the non-zero weight parameter of the tth iteration, is the learning rate of non-zero weight parameters, To fine-tune the loss function, represents the fine-tuning loss function About non-zero weight parameters The gradient of An early stopping detection is performed based on a set early stopping rule. When it is detected that the number of jump connections in a unit of the search network is greater than or equal to the set number, it is determined that the early stopping rule is met and the search process is stopped.

6. The method for searching a remote sensing image classification model structure based on spatial constraints according to claim 5, characterized in that: The early stopping rules also include: The degree of change in the order of the architecture parameters within a set continuous period is detected. If the degree of change in the order of the architecture parameters within the set continuous period is lower than a set degree threshold, it is determined that the search process is saturated and meets the early stopping rule, and the search process is stopped.

7. The method for searching a remote sensing image classification model structure based on spatial constraints according to claim 5, characterized in that: The step S5 comprises: The stochastic gradient descent algorithm is used to update the search network architecture parameters, and the formula is: In the formula, α t+1 is the architecture parameter of the t+1th iteration, α t is the architecture parameter of the tth iteration, η α is the learning rate of the architecture parameter α, is the validation loss function of the fine-tuned weight parameters and architecture parameters, Represents the validation loss function About the architecture parameter α t The gradient of Traversing the search network, and setting the proportions of corresponding modules according to the updated architecture parameters; Iteratively acquiring a plurality of remote sensing sample images based on the training sample set, wherein the plurality of remote sensing sample images include input features and target labels; Inputting the input features into the scaled search network to obtain a prediction output; Calculating a cross entropy loss based on the predicted output and the corresponding target label; Perform back propagation based on the cross entropy loss to calculate the gradient of the search network parameters with respect to the loss; The gradients are clipped, the weights are updated using stochastic gradient descent, and the learning rate is adjusted based on a cosine annealing scheduler.

8. The method for searching a remote sensing image classification model structure based on spatial constraints according to claim 1, characterized in that: After step S6, the method further includes: The target classification model and target architecture parameters are fine-tuned by combining network pruning and knowledge distillation to obtain an edge network model, which can be deployed in an edge device.

9. The method for searching a remote sensing image classification model structure based on spatial constraints according to claim 8, characterized in that: The method of combining network pruning and knowledge distillation to fine-tune the target classification model and target architecture parameters to obtain an edge network model includes: Obtaining available resources of the edge device and occupied resources of the target classification model; Calculating the difference between the occupied resources and the available resources, and when the difference is greater than 0, taking the ratio of the difference to the occupied resources as the pruning ratio, and pruning the target classification model in a structured pruning manner; Based on the knowledge distillation method, the model before pruning is used to train the model after pruning to obtain the edge network model.

Images