Remote sensing image target detection method adopting two-stage microarchitecture search
Through the dual-stage microarchitecture search algorithm, combined with feature pyramid network and self-distillation strategy, the problems of jump connection enrichment and depth mismatch in traditional methods are solved, and better remote sensing image object detection performance and small object detection accuracy are achieved.
Patent Information
- Application Number
- CN202510082430.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-08-17
- Filing Date
- 2025-01-20
- Publication Date
- 2025-04-25
AI Technical Summary
Traditional microarchitecture searches have problems in remote sensing image object detection, such as jump connection enrichment, search network and evaluation network depth mismatch, resulting in poor network performance, especially in small object detection.
A two-stage microarchitecture search algorithm is proposed. By combining the separate learning stage and the joint learning stage, a feature pyramid network and self-distillation strategy are introduced, and an αSoftmax regularization strategy is adopted to ensure the fairness of candidate operation selection and alleviate the problem of depth mismatch.
It realizes better remote sensing image object detection performance, can obtain better target detection results on different remote sensing image data sets, and improves network stability and small target detection accuracy.
Smart Images
Figure CN119888194A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of target detection, and more specifically to a method of using a neural network architecture search to perform remote sensing image target detection tasks. Background Art
[0002] Optical remote sensing image target detection is the classification and positioning of objects of interest in a given satellite or aerial image, and has achieved good application results in many fields such as urban planning, environmental monitoring, and military reconnaissance. With the development of remote sensing technology, the spatial resolution of remote sensing images has been continuously improved. Research on spatial information and local detail information in high-resolution remote sensing images is of great significance for improving the accuracy and effect of target detection at this stage.
[0003] The method based on convolutional neural network is one of the popular methods in the field of remote sensing image target detection. It provides strong technical support for high-resolution remote sensing image target detection. Researchers design network models and perform iterative training to obtain better target detection results. At present, in order to meet performance requirements, the design of convolutional neural networks has gradually become more complex, resulting in the design of network models requiring a lot of manpower and computing resources, which has seriously affected the development of remote sensing image target detection.
[0004] In order to reduce the burden on researchers when designing networks, the neural network architecture search method has been proposed for the automatic construction of network models. Compared with manually designed networks, this method reduces the consumption of manpower and computing resources and reduces the influence of human subjective factors. At present, one of the mainstream methods of neural network architecture search is differentiable architecture search, which is widely used in target detection in natural scenes, but the effect of target detection in remote sensing images is not ideal. The main reasons are: ① The performance of the searched network is poor. At present, the search network and evaluation network of differentiable architecture search are deeply separated, resulting in the network model obtained in the network search stage not being optimal in the evaluation stage. At the same time, differentiable architecture search has the phenomenon of unfair selection of candidate operations. As the number of network search rounds increases, jump connections are selected too much in the search process, resulting in a decrease in network stability and the model is prone to performance collapse. ② Small target detection is difficult. Since the pixel area occupied by small targets in remote sensing image target detection datasets is small, it is difficult for the searched network model to extract enough information for small target detection, resulting in the network model being unable to achieve good detection accuracy in remote sensing image datasets with many small targets. Summary of the invention
[0005] The present invention proposes a two-stage differentiable architecture search algorithm for target detection in high-resolution remote sensing images, which solves the problems of skip connection enrichment and mismatch between the search network and the evaluation network depth in traditional differentiable architecture search, and can better detect target objects based on the characteristics of remote sensing images. This method inherits the advantages of traditional neural network architecture search and can automatically build a network model when the designer lacks expert experience. The network model obtained by the search can achieve better target detection results on different remote sensing image data sets.
[0006] In order to achieve the above objectives, the present invention adopts the following technical solutions.
[0007] 1. A remote sensing image target detection method using a two-stage differentiable architecture search, including a separate learning stage and a joint learning stage.
[0008] The steps for the individual learning phase are as follows:
[0009] S1: build feature pyramid network;
[0010] S2: build detection head;
[0011] S3: Build the backbone network for neural network architecture search;
[0012] S4: Pre-train the search network and evaluation network in the backbone network respectively, and use the weights obtained in this stage to initialize the joint learning stage.
[0013] The steps of the joint learning phase are as follows:
[0014] S5: Based on dual optimization, a self-distillation strategy is combined to guide the search network for architecture search.
[0015] S6: Introduce a regularization strategy during the network search process to address the problem of skip connection enrichment and ensure fairness in candidate operation selection.
[0016] S7: Implement the network search process, optimize the search network and the evaluation network mutually, and iterate the network search and network evaluation to ensure the consistency of depth.
[0017] S8: End.
[0018] 2. According to claim 1, a remote sensing image target detection method using a two-stage differentiable architecture search is characterized in that a feature pyramid network is introduced in S1 for feature fusion, so that the model can better handle multi-scale targets in remote sensing images.
[0019] 3. The remote sensing image target detection method using a two-stage differentiable architecture search according to claim 1 is characterized in that a detection head is used in S2 to classify the target category and regress the bounding box.
[0020] 4. According to the remote sensing image target detection method using two-stage differentiable architecture search as described in claim 1, it is characterized in that the backbone network is automatically constructed in S3 using a method based on differentiable architecture search.
[0021] 5. According to claim 1, a remote sensing image target detection method using a two-stage differentiable architecture search is characterized in that a separate learning stage is performed in S4, which involves randomly initializing network architecture parameters by pre-training the search network and the evaluation network in a training set and a validation set, respectively.
[0022] 6. A remote sensing image target detection method using a two-stage differentiable architecture search according to claim 1, characterized in that in S5, joint learning is performed to combine shallow and deep features of the evaluation network through a self-distillation strategy as feedback to the search network, and guide the student model to search the network without additionally training the teacher model.
[0023] 7. A remote sensing image target detection method using a two-stage differentiable architecture search according to claim 1, characterized in that α is introduced in S6 Softmax Regularization strategy: This regularization strategy no longer applies traditional L1 or L2 regularization to the network architecture parameter α, but instead performs continuous processing on the network architecture parameters after Softmax continuous processing, thereby ensuring the fairness of candidate operation selection.
[0024] 8. According to claim 1, a remote sensing image target detection method using a two-stage differentiable architecture search is characterized in that a self-distillation strategy is used in S7 to iteratively perform network search and network evaluation, so that the entire process is dynamically updated with the network search, rather than learning a fixed teacher model, so that the search network can obtain feedback information of the evaluation network from the current search state, alleviating the depth mismatch problem between the search network and the evaluation network. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only schematic diagrams of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0026] Figure 1Abstract flow chart of the present invention
[0027] Figure 2 This is the target detection network architecture diagram
[0028] Figure 3 Comparison between this algorithm and traditional differentiable architecture search algorithms
[0029] Figure 4 Regularization process for network architecture parameters DETAILED DESCRIPTION
[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0031] The embodiment of the present invention discloses a remote sensing image target detection method using a two-stage differentiable architecture search, including two parts: a separate learning stage and a joint learning stage:
[0032] The steps for the individual learning phase are as follows:
[0033] S1: build feature pyramid network;
[0034] S2: build detection head;
[0035] S3: Build the backbone network for neural network architecture search;
[0036] S4: Pre-train the search network and evaluation network in the backbone network respectively, and use the weights obtained in this stage to initialize the joint learning stage.
[0037] The steps of the joint learning phase are as follows:
[0038] S5: Based on dual optimization, a self-distillation strategy is combined to guide the search network for architecture search.
[0039] S6: Introduce a regularization strategy during the network search process to address the problem of skip connection enrichment and ensure fairness in candidate operation selection.
[0040] S7: Implement the network search process, optimize the search network and the evaluation network mutually, and iterate the network search and network evaluation to ensure the consistency of depth.
[0041] S8: End.
[0042] In order to further optimize the above technical solution. First, according to steps S1-S3, the backbone network is searched using a method based on a differentiable architecture search, and a feature pyramid network is introduced for feature fusion, and finally the detection head is used to detect the target in the remote sensing image.
[0043] Secondly, a separate learning phase is performed according to step S4, which involves randomly initializing the network architecture parameters by pre-training the search network and the evaluation network in the training set and the validation set respectively.
[0044] Next, the joint learning phase is performed according to step S5, and the shallow and deep features of the evaluation network are combined through the self-distillation strategy as feedback to the search network. The student model is guided to search the network without additional training of the teacher model.
[0045] Then, according to step S6, α is introduced Softmax Regularization strategy: This regularization strategy no longer applies traditional L1 or L2 regularization to the network architecture parameter α, but instead performs continuous processing on the network architecture parameters after Softmax continuous processing, thereby ensuring the fairness of candidate operation selection.
[0046] Finally, according to step S7, the self-distillation strategy is used to iteratively perform network search and network evaluation, so that the entire process is dynamically updated with the network search, rather than learning a fixed teacher model, so that the search network can obtain feedback information from the evaluation network from the current search state, alleviating the depth mismatch problem between the search network and the evaluation network.
[0047] The method involved in the present invention is further described below.
[0048] In order to improve the accuracy and efficiency of remote sensing image target detection, this paper proposes a two-stage differentiable architecture search algorithm. Figure 2 As shown in the figure. First, the search part of the backbone network uses a two-stage differentiable architecture search method in the constructed search space. By establishing a joint learning mechanism between the search network and the evaluation network, a backbone network with better performance is searched. Then, the backbone network obtained by the search is used for feature extraction, and feature fusion is performed through the Feature Pyramid Networks (FPN). Finally, the detection head (Detection of the Head) is used to detect the ground objects in the image.
[0049] Aiming at the problem that the network model obtained by traditional microarchitecture search has poor performance, the present invention redesigns the microarchitecture search method, such as Figure 3 As shown, Figure 3(a) is the traditional DARTS algorithm, Figure 3 (b) is the present algorithm. Limited by GPU memory, usually the differentiable architecture search searches the network in a shallow network (8 cells), and stacks the network units obtained from the search to form a deep network (20 cells) for network evaluation. Although the differentiable architecture search can efficiently perform network search, the search network and the evaluation network of this method are deeply separated, and the optimal unit obtained from the search network is usually more suitable for shallow architectures, and may not be optimal in the evaluation network of a deep architecture. Therefore, the size of the network architecture parameter α obtained from the search network cannot reflect the actual performance of each candidate operation, resulting in the network model obtained from the search not being the best model in terms of performance. At the same time, as the differentiable architecture search proceeds, jump connections are more often chosen by network models due to their faster convergence characteristics, but too many jump connections do not improve the performance of the network, but instead make the network less stable. Therefore, the present invention is designed Figure 3 The method shown in (b) performs a two-stage search on the network.
[0050] Different from the traditional differentiable architecture search method, by adding the self-distillation strategy on the basis of the double-layer optimization of the network architecture parameter α and the network weight w, the search process is divided into a separate learning phase and a joint learning phase to search the network. The network search and network evaluation are continuously iterated until the network model converges.
[0051]
[0052] Formula (1) represents the process of a single learning phase, w t and w v Represent the weights of the search network and the evaluation network respectively, and They represent the weights of the search network and the evaluation network after pre-training. In the separate learning phase, the network model pre-trains the search network and the evaluation network on the training set and the validation set, respectively. Specifically, when pre-training the search network, the network architecture parameter α is randomly initialized, and the network is trained on the training set to obtain the weight that minimizes the loss of the training set. At the same time, the evaluation network is pre-trained, and the network is trained on the validation set according to the network architecture parameter α to obtain the weight that minimizes the loss of the validation set. In the joint learning phase, the network model uses the weights trained in the separate learning phase as initialization weights and uses the self-distillation strategy to update the network architecture parameters α.
[0053]
[0054] Formula (2) represents the process of joint learning. Indicates that according to the initialization weight Search to obtain the network architecture parameter α that minimizes the loss of the training set. Indicates that the weight w that minimizes the loss of the validation set is obtained by training according to the network architecture parameter α v , represents the self-distillation part between the evaluation network and the search network, α * and Represents the final network architecture parameters and network weights after iteration. The entire search-evaluation process passes the optimal unit structure obtained by the search network to the evaluation network, and then passes the feedback information back to the search network through self-distillation. By establishing an iterative relationship between the search network and the evaluation network, the problem of deep separation between the search network and the evaluation network is well solved.
[0055] Since remote sensing images have a large number of small targets compared to images in natural scenes, traditional differentiable architecture search methods cannot be well applied to the field of remote sensing image target detection. To solve this problem, the model no longer uses only the deepest layer features of the evaluation network as feedback information to the search network, but combines the shallow part of the evaluation network with the deep part as feedback information to the search network. Usually, the features of the shallow part of the network have high resolution but lack semantic information, while the features of the deep part of the network have low resolution but rich semantic information. Therefore, the algorithm combines the features of the shallow network and the deep network through the embedding block to guide the update of the search network architecture parameter α. Compared with using only the deepest layer features of the evaluation network as feedback information, the network searched by combining features of different scales as feedback information can better detect small targets in remote sensing images.
[0056]
[0057] Formula (3) is the specific calculation process of self-distillation, where N is the number of training samples, τ is the temperature coefficient in the cross entropy loss, and p(w v ,α) are the multi-scale features of the search network and the evaluation network after Softmax continuous processing.
[0058]
[0059] Among them, f i v and f i tare the features of the search network and the evaluation network output respectively. Formulas (4) and (5) are the Softmax continuous processing process. The features after Softmax continuous processing are used as feedback information to guide network search.
[0060] The self-distillation method used in the present invention can guide the student model to search for the network architecture parameter α without additional training of the teacher model, thereby reducing the calculation of the model. Secondly, the teacher model used in the self-distillation method is dynamically updated as the network searches, rather than learning a fixed teacher model, so that the search network can obtain feedback information of the evaluation network from the current search state. By adding the network evaluation stage to the network search, the problem that the differentiable architecture search method cannot search for the optimal network is avoided.
[0061] At present, there is a phenomenon of skip connection enrichment in the search method of microarchitecture. Since the skip connection converges quickly, the searched network model has overfitting. In order to solve the problem of performance degradation caused by overfitting of the network model, this paper introduces the idea of regularization to improve the problem of unfair selection of candidate operations. Figure 4 As shown in Figure 1, this regularization method is different from the traditional L1 and L2 regularization. It no longer imposes a regularization penalty term on the network architecture parameter α, but instead imposes a regularization penalty term on the network architecture parameter α after Softmax processing. Softmax Regularization is performed. The core of this regularization method is to constrain the network architecture parameter α Softmax The value of is set to limit the variance of the parameter, so that each candidate operation in the search space is selected with equal weight, achieving the purpose of fair search in the search space.
[0062] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0063] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A remote sensing image target detection method using a two-stage differentiable architecture search, including a separate learning stage and a joint learning stage: The steps for the individual learning phase are as follows: S1: build feature pyramid network; S2: build detection head; S3: Build the backbone network for neural network architecture search; S4: Pre-train the search network and evaluation network in the backbone network respectively, and use the weights obtained in this stage to initialize the joint learning stage. The steps of the joint learning phase are as follows: S5: Based on dual optimization, a self-distillation strategy is combined to guide the search network to perform architecture search. S6: Introduce a regularization strategy during the network search process to address the problem of skip connection enrichment and ensure fairness in candidate operation selection. S7: Implement the network search process, optimize the search network and the evaluation network mutually, and iterate the network search and network evaluation to ensure the consistency of depth. S8: End.
2. A remote sensing image target detection method using a two-stage differentiable architecture search according to claim 1, characterized in that: A feature pyramid network is introduced in S1 for feature fusion, enabling the model to better target multi-scale targets in remote sensing images.
3. The remote sensing image target detection method using a two-stage differentiable architecture search according to claim 1, characterized in that: In S2, the detection head is used to perform object classification and bounding box regression respectively.
4. The remote sensing image target detection method using a two-stage differentiable architecture search according to claim 1, characterized in that: In S3, the backbone network is automatically constructed using a method based on microarchitecture search.
5. The remote sensing image target detection method using a two-stage differentiable architecture search according to claim 1, characterized in that: A separate learning phase is performed in S4, which involves randomly initializing the network architecture parameters by pre-training the search network and the evaluation network on the training set and the validation set, respectively.
6. The remote sensing image target detection method using a two-stage differentiable architecture search according to claim 1, characterized in that: In S5, a joint learning phase is performed to combine the shallow and deep features of the evaluation network through a self-distillation strategy as feedback to the search network, guiding the student model to search the network without additionally training the teacher model.
7. The remote sensing image target detection method using a two-stage differentiable architecture search according to claim 1, characterized in that: Introducing α in S6 Softmax Regularization strategy: This regularization strategy no longer applies traditional L1 or L2 regularization to the network architecture parameter α, but instead performs continuous processing on the network architecture parameters after Softmax continuous processing, thereby ensuring the fairness of candidate operation selection.
8. The remote sensing image target detection method using a two-stage differentiable architecture search according to claim 1, characterized in that: In S7, the self-distillation strategy is used to iteratively perform network search and network evaluation, so that the entire process is dynamically updated with the network search, rather than learning a fixed teacher model. This enables the search network to obtain feedback information from the evaluation network from the current search status, alleviating the depth mismatch problem between the search network and the evaluation network.