A remote sensing image lightweight network rapid detection method, device and medium

Abstract

The application discloses a kind of remote sensing image light weight network fast detection method, device and medium, the method includes obtaining remote sensing image;Input remote sensing image to pre-trained light weight target detection network, obtain detection result;Wherein: light weight target detection network is equipped with Mobilenetv3tiny network and improved enhanced feature extraction network PAFPN, improved enhanced feature extraction network PAFPN is equipped with Ghost module, the application designs a kind of light weight Mobilenetv3tiny network, can reduce model parameter quantity, improve detection speed, by introducing Ghost module in improved enhanced feature extraction network PAFPN, reduce the complexity of network under the premise of guaranteeing detection accuracy, easy to deploy to mobile device.

Description

TECHNICAL FIELD

[0001] The present application relates to a remote sensing image lightweight network fast detection method, device and medium, belonging to the image processing technical field. BACKGROUND

[0002] With the development of remote sensing technology, remote sensing image recognition plays an important role in precision agriculture, marine monitoring, urban layout, military defense and other fields. For example, remote sensing image recognition can provide technical support for geological research and geological exploration, and help to carry out geological survey, geological environment evaluation and basic geological research. However, in terms of real-time monitoring of ships at sea, road traffic analysis, disaster detection and other aspects, the algorithm is required to be efficient and real-time to monitor the target, so how to balance the detection accuracy and speed is still a great challenge.

[0003] Early remote sensing image target detection mainly uses artificial feature modeling combined with a classifier to classify and determine the target category. With the development of the times, the early detection method is difficult to meet the actual application requirements in terms of accuracy and speed. In recent years, with the continuous development of deep learning, many mature detection algorithms have been formed, such as the two-stage algorithm represented by Faster RCNN, Mask RCNN and the single-stage algorithm represented by SSD, YOLO, which makes it possible to quickly and accurately identify targets and objects. Among them, the two-stage Faster RCNN detection method first extracts the candidate region, and then classifies and regresses the candidate region, so the detection accuracy is high, but the detection speed is slow, which cannot meet the real-time detection requirement. The single-stage YOLO series detection method directly regresses the target from the input image, so the detection speed is fast, but the overall detection accuracy is lower than the current advanced network. Therefore, there is an urgent need for a target detection method to effectively balance the detection accuracy and speed, and facilitate device deployment. SUMMARY

[0004] The purpose of the present application is to overcome the shortcomings of the prior art, provide a remote sensing image lightweight network fast detection method, device and medium, improve the balance of remote sensing image target detection speed and accuracy, and be easy to deploy to mobile devices.

[0005] To achieve the above purpose, the present application adopts the following technical scheme:

[0006] In a first aspect, the present application provides a remote sensing image lightweight network fast detection method, comprising:

[0007] obtaining a remote sensing image;

[0008] inputting the remote sensing image into a pre-trained lightweight target detection network to obtain a detection result;

[0009] The lightweight target detection network is provided with a Mobilenetv3tiny network and an improved enhanced feature extraction network PAFPN.

[0010] Further, the Mobilenetv3tiny network comprises a Focus focusing layer and a Dark feature extraction layer, the Dark feature extraction layer is provided with a bottleneck based on a residual structure for feature extraction, and the Focus focusing layer is used for slice operation on the input picture.

[0011] Further, the improved enhanced feature extraction network PAFPN further comprises a feature layer P and a splicing layer.

[0012] Further, after obtaining the remote sensing image, data enhancement is performed by using Mosaic and Mixup.

[0013] Further, the pre-training method of the lightweight target detection network comprises:

[0014] The lightweight target detection network is trained by using a COCO image recognition data set;

[0015] The accuracy of the lightweight target detection network in identifying the verification set image during the training process is recorded, and whether the training process converges is judged according to the accuracy, and the weight of the network after convergence is recorded as the weight of the lightweight target detection network.

[0016] Further, the training method of the lightweight target detection network comprises:

[0017] The lightweight target detection network is trained by using a remote sensing image NWPU VHR-10 data set;

[0018] The precision of the lightweight target detection network in detecting the verification set target during the training process is recorded, and whether the training process converges is judged according to the precision, and the weight of the network after convergence is recorded as the weight of the initial lightweight target detection network;

[0019] The training is repeated to make the detection network converge, and the obtained network weight is the final lightweight target detection network weight.

[0020] Further, in the training process of the lightweight target detection network, the loss function is optimized to balance the identification and detection processes, wherein the loss function used in the training process is , wherein represents a generalized intersection over union loss function, represents a category loss, represents a confidence loss.

[0021] Further, the repeated training makes the detection network converge, comprising:

[0022] During the recording training process, the accuracy of the network for target detection of the verification set is recorded, and when the accuracy fluctuation change does not exceed the preset threshold in the set number of rounds of training, it is considered that the network has converged.

[0023] In a second aspect, the present application provides a remote sensing image lightweight network rapid detection device, comprising:

[0024] An image acquisition module is configured to acquire a remote sensing image.

[0025] An image detection module is configured to input the remote sensing image into a pre-trained lightweight target detection network to obtain a detection result.

[0026] In a third aspect, the present application provides an electronic device comprising a processor and a storage medium.

[0027] The storage medium is configured to store instructions.

[0028] The processor is configured to operate according to the instructions to perform the steps of the method according to any one of the preceding aspects.

[0029] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method according to any one of the preceding aspects.

[0030] Compared with the prior art, the present application has the following beneficial effects:

[0031] The present application provides a remote sensing image lightweight network rapid detection method, which designs a lightweight Mobilenetv3tiny network, which can reduce the model parameter quantity and improve the detection speed.

[0032] The application builds a lightweight target detection network, pre-trains using a COCO image recognition data set, then overall trains using a remote sensing image target detection data set, modifies a loss function to adjust the network, accelerates the model convergence speed, repeatedly trains to make the detection network converge, and obtains the final required multi-scale target detection network weight; the application optimizes the training method of the target detection network, realizes further stable adjustment training of the detection network, and improves the performance of the network for rapid detection of remote sensing image targets; the network provided by the application effectively balances the detection speed and accuracy, is easy to deploy, and is suitable for the needs of real-time detection of remote sensing image targets. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is a flowchart of a remote sensing image lightweight network rapid detection method provided by an embodiment of the application;

[0034] Figure 2 is a whole network architecture diagram provided by an embodiment of the application;

[0035] Figure 3 is a structure diagram of a backbone network provided by an embodiment of the application;

[0036] Figure 4 is a structure diagram of a reinforced feature extraction network provided by an embodiment of the application. DETAILED DESCRIPTION

[0037] The application will be further described below with reference to the drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the application, and cannot be used to limit the protection scope of the application.

[0038] Embodiment 1, this embodiment introduces a remote sensing image lightweight network rapid detection method, comprising:

[0039] obtaining a remote sensing image;

[0040] inputting the remote sensing image into a pre-trained lightweight target detection network to obtain a detection result;

[0041] Among them: the lightweight target detection network is provided with a Mobilenetv3tiny network and an improved reinforced feature extraction network PAFPN, and the improved reinforced feature extraction network PAFPN is provided with a Ghost module.

[0042] The application process of the remote sensing image lightweight network rapid detection method provided by this embodiment specifically involves the following steps:

[0043] Step 1: build a lightweight target detection network of deep learning;

[0044] The multi-scale target detection network building process is as follows:

[0045] Step 1.1: According to the requirements, a lightweight Mobilenetv3tiny network is designed to build a new backbone, which can reduce the model parameter amount and improve the detection speed;

[0046] Step 1.2: Build a reinforced feature extraction module, introduce a Ghost module in the network model, reduce the complexity of the network under the premise of ensuring detection accuracy.

[0047] The Mobilenetv3tiny network and the new backbone network structure are as shown in Figure 3 The Focus focusing layer and the Dark feature extraction layer are mainly included, the bottleneck based on the residual structure is used for feature extraction, and a good balance can be kept in the parameter amount and detection accuracy. The Focus focusing layer mainly performs slicing operation on the input picture, and gets a value every other pixel in a picture, so as to get four pictures, the four pictures are complementary, and information loss is avoided. The bottleneck structure of the Dark feature extraction layer retains the linear bottleneck, the reverse residual structure and the h-swish activation function in the Mobilenetv3, so as to reduce the calculation amount and improve the detection performance. According to the step length Stride, the bottleneck structure is selected, if Stride=1, a branch structure is added, if Stride=2, no branch structure is added. According to Use_se, the attention mechanism is selected, and according to Use_hs, the Relu, h-swish activation function type is selected. The Relu activation function provides more sensitive activation and input similar to linear relationship, and avoids saturation. The h-swish activation function retains the characteristics of the Relu function, and reduces the calculation overhead. In view of the problem of low detection accuracy, the weight of each channel is adjusted, the position of 5x5 and 3x3 deep convolution is redesigned, and the model feature extraction capability is strengthened. At the same time, in order to avoid dimension reduction, the efficient channel attention ECA module is introduced, and the detection performance of the deep convolution network is improved.

[0048] The Ghost module and the reinforced feature extraction network PAFPN structure are as shown in Figure 4As shown, this paper introduces a Ghost module in the PAFPN structure to obtain more feature information with less parameter amount. The improved PAFPN structure includes feature layer P, splicing and Ghost convolution. Convolutional neural networks produce redundant feature maps during operation, which contain a large number of network parameters and consume more computing resources. However, these redundant information ensures a comprehensive understanding of the input image, which affects the model detection accuracy, so these redundant feature maps cannot be directly removed. Specifically, the Ghost module divides the ordinary convolutional layer in the deep neural network into two parts. The first part involves ordinary convolution, but their total number will be strictly controlled. On the basis of the inherent feature mapping of the first part, more feature mappings are generated through a series of simple linear operations. Compared with the ordinary convolutional neural network, this Ghost module reduces the total number of required parameters and computational complexity without changing the size of the output feature mapping, so that redundant features can be obtained with low overhead, reducing the parameter amount and operation amount while ensuring the feature extraction capability, and improving the detection speed.

[0049] The deep learning-based remote sensing image lightweight network structure built as shown in Figure 2 As shown, the improved target detection network in this paper uses Mosaic and Mixup for data enhancement at the input end, uses Mobilenetv3tiny as the backbone network after preprocessing, and introduces PAFPN multi-scale feature fusion to improve the target detection accuracy, and uses three decoupling heads for prediction at the output end.

[0050] Step 2: The detection network is pre-trained using the COCO image recognition dataset;

[0051] The detection network pre-training process is as follows:

[0052] Step 2.1: Train the detection network using the COCO image recognition dataset;

[0053] Step 2.2: Record the accuracy of the network in identifying the validation set images during the training process, and determine whether the training process has converged according to the accuracy. Record the network weight after convergence as the weight of the target detection network;

[0054] Step 3: The detection network is trained using the remote sensing image NWPU VHR-10 target detection dataset;

[0055] The detection network overall training process is as follows:

[0056] Step 3.1: Train the network using the remote sensing image NWPU VHR-10 dataset;

[0057] Step 3.2: record the accuracy of the network in the training process for target detection of the verification set, and determine whether the training process converges according to the accuracy, and record the weight of the network after convergence as the weight of the detection network.

[0058] Step 4: optimize the loss function, balance the identification and detection processes, and accelerate the convergence speed of the model.

[0059] The optimized loss function is a loss function used in the training process , wherein represents a generalized intersection over union loss function, represents a category loss, and represents a confidence loss.

[0060] Step 5: repeat the training to make the detection network converge, and the obtained network weight is the final required multi-scale target detection network weight.

[0061] The repeated training and the method for determining the convergence of the target detection network are as follows: record the accuracy of the network in the training process for target detection of the verification set, and when the accuracy fluctuates by no more than 0.05 mAP in 10 training rounds, it is considered that the network has converged.

[0062] In this embodiment, the rapid target detection network obtained after training for n iterations is used for 800 images of ten different targets in the remote sensing image NWPU VHR-10 data set, and finally the target recognition accuracy is 0.9299 mAP and the detection speed is 57.49 FPS. It is not difficult to see that the present application effectively balances the detection speed and accuracy, is easy to deploy, and is suitable for the needs of real-time detection of remote sensing image targets, and successfully realizes the lightweight remote sensing image rapid detection capability in an innovative way.

[0063] This embodiment builds a lightweight target detection network, pre-trains it using the COCO image recognition data set, then trains the whole network using the remote sensing image target detection data set, modifies the loss function to adjust the network, accelerates the convergence speed of the model, repeats the training to make the detection network converge, and obtains the final required multi-scale target detection network weight. The present application optimizes the training method of the target detection network, realizes further stable adjustment and training of the detection network, and improves the performance of the network for rapid detection of remote sensing image targets. The network proposed in the present application effectively balances the detection speed and accuracy, is easy to deploy, and is suitable for the needs of real-time detection of remote sensing image targets.

[0064] Embodiment 2 provides a lightweight network rapid detection device for remote sensing images, comprising:

[0065] An image acquisition module is configured to acquire remote sensing images.

[0066] The image detection module is used for inputting a remote sensing image into a pre-trained lightweight target detection network to obtain a detection result; wherein, the lightweight target detection network is provided with a Mobilenetv3tiny network and an improved enhanced feature extraction network PAFPN, and the improved enhanced feature extraction network PAFPN is provided with a Ghost module.

[0067] Embodiment 3 provides an electronic device, comprising a processor and a storage medium;

[0068] The storage medium is used for storing instructions;

[0069] The processor is used for operating according to the instructions to perform the steps of the method according to any one of the preceding embodiments.

[0070] Embodiment 4 provides a computer-readable storage medium, which stores a computer program, and the program is executed by a processor to implement the steps of the method according to any one of the preceding embodiments.

[0071] The above only describes the preferred embodiments of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the technical principles of the present application, a number of improvements and modifications can be made, and these improvements and modifications should be considered as the protection scope of the present application.

Claims

1. A lightweight network-based fast detection method for remote sensing images, characterized in that, include: Acquire remote sensing images; Input remote sensing images into a pre-trained lightweight target detection network to obtain detection results; Wherein: the lightweight target detection network includes a Mobilenetv3tiny network and an improved enhanced feature extraction network PAFPN, and the improved enhanced feature extraction network PAFPN includes a Ghost module; The Mobilenetv3tiny network includes a Focus layer and a Dark feature extraction layer. The Dark feature extraction layer has a bottleneck based on a residual structure for feature extraction. The Focus layer is used to slice the input image. The improved enhanced feature extraction network PAFPN also includes a feature layer P and a splicing layer; After acquiring remote sensing images, data augmentation was performed using Mosaic and Mixup; The pre-training method for the lightweight object detection network includes: A lightweight object detection network was trained using the COCO image recognition dataset. Record the accuracy of the lightweight object detection network in recognizing images on the validation set during the training process. Based on this, determine whether the training process has converged. Record the weights of the network after convergence as the weights of the lightweight object detection network. The training method for the lightweight object detection network includes: A lightweight object detection network was trained using the NWPU VHR-10 dataset of remotely sensed images. Record the accuracy of the lightweight object detection network in detecting objects on the validation set during the training process, and use this to determine whether the training process has converged. Record the weights of the network after convergence as the initial weights of the lightweight object detection network. Repeated training causes the detection network to converge, and the resulting network weights are the final lightweight target detection network weights.

2. The lightweight network-based rapid detection method for remote sensing images according to claim 1, characterized in that, During the training process of the lightweight object detection network, the loss function is optimized to balance the recognition and detection processes. The loss function used during training is: ,in This represents the generalized intersection-union loss function. Indicates category loss. This represents the confidence loss.

3. The lightweight network-based rapid detection method for remote sensing images according to claim 1, characterized in that, The repeated training causes the detection network to converge, including: Record the accuracy of the network in detecting targets on the validation set during the training process. If the accuracy fluctuation does not exceed a preset threshold in a set number of training rounds, the network is considered to have converged.

4. A lightweight network rapid detection device for remote sensing images, used to implement the lightweight network rapid detection method for remote sensing images according to any one of claims 1-3, characterized in that, include: Image acquisition module, used to acquire remote sensing images; An image detection module is used to input remote sensing images into a pre-trained lightweight target detection network to obtain detection results; wherein: the lightweight target detection network includes a Mobilenetv3tiny network and an improved enhanced feature extraction network PAFPN, and the improved enhanced feature extraction network PAFPN includes a Ghost module.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When executed by a processor, the program implements the steps of the method according to any one of claims 1 to 3.

Images