A remote sensing image classification method and system based on improved MobileNet v2

Abstract

The present invention provides a remote sensing image classification method and system based on improved MobileNet v2, wherein the method comprises: step 1: obtaining a remote sensing image classification data set; step 2: preprocessing the remote sensing image classification data set; step 3: obtaining a first output feature according to the processed data; step 4: obtaining a second output feature by multi-scale mixed convolution of the first output feature; step 5: cross-fusion of the second output feature to obtain a third output feature; step 6: determining useful information according to the third output feature; step 7: performing classification prediction according to the useful information; step 8: iterating according to the classification prediction result, determining the target network model, and determining the classification result based on the remote sensing image classification task. The present invention provides a remote sensing image classification method and system based on improved MobileNet v2, cross-fusion of feature information of different scales, higher classification accuracy, introduction of attention mechanism, and higher recognition efficiency.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a remote sensing image classification method and system based on improved MobileNet v2. Background Art

[0002] Classifying remote sensing images is a critical task in geographic information systems (GIS), environmental monitoring, urban planning, agricultural management, disaster assessment, and many other fields. Remote sensing image classification can help identify and monitor different types of land use and cover, such as forests, farmlands, urban areas, and water bodies. Remote sensing classification can also assist urban planners in understanding urban expansion patterns, identifying illegal construction, and planning future infrastructure development. Farmers and agricultural experts can use remote sensing images to monitor crop health, assess crop yields, and manage irrigation and fertilization. In addition, remote sensing image classification can quickly assess the impact of natural disasters (such as floods, fires, and earthquakes) on the surface, thereby helping rescue teams locate disaster-affected areas and develop rescue plans.

[0003] MobileNet v2 is a lightweight and highly accurate network based on the MobileNet v1 network. It retains its simplicity and does not require any special operators, while significantly improving its accuracy. It is widely used in various application scenarios in the field of computer vision.

[0004] The invention patent with application number: CN202210749334.3 discloses a remote sensing image mining method that couples knowledge graph and deep neural network, wherein the method includes: step 1, constructing a remote sensing image knowledge expression model that takes into account the imaging mechanism of ground objects and the characteristics of remote sensing images; step 2, constructing a remote sensing image knowledge graph based on multi-source data; step 3, using knowledge representation learning methods to mine entity and relationship knowledge in the remote sensing image knowledge graph, and converting it into a low-dimensional dense vector representation in the knowledge space; step 4, using a deep convolutional neural network to extract typical visual features of remote sensing images, and learn to map visual space features to knowledge space features; step 5, calculating the similarity between the mapping vector and the remote sensing image knowledge representation learning vector; step 6: for the remote sensing image to be mined, the typical visual feature extraction and feature mapping learning methods of steps 4 and 5 are used to mine remote sensing image knowledge. The above invention has further improved the interpretation and mining effects of remote sensing images.

[0005] However, the above-mentioned existing technology uses the MobileNet-V2 model to extract visual features of remote sensing images. The MobileNet-V2 model lacks the function of multi-scale learning features, and does not fully utilize the information between each feature for deep fusion and interaction. When the scale of the target object varies greatly, the accuracy will be slightly lower. In addition, when the target information is relatively complex, MobileNet v2 lacks an attention mechanism to make it pay more attention to the key parts of the image, which reduces the efficiency of subsequent image recognition.

[0006] In view of this, there is an urgent need for a remote sensing image classification method and system based on an improved MobileNet v2 to at least solve the above-mentioned shortcomings. Summary of the invention

[0007] One of the purposes of the present invention is to provide a remote sensing image classification method and system based on an improved MobileNet v2, preprocess the acquired remote sensing image, and input the processed data obtained by the preprocessing into the MobileNet v2 network to obtain the first output feature, and perform multi-scale mixed convolution, feature cross fusion and weight calculation on the first output feature in sequence to determine useful information. The useful information is input into the fully connected layer for classification prediction, and multiple iterations are performed based on the difference between the predicted result and the actual result to determine the weight file configuration target network model that meets the iteration requirements, the target network model performs the remote sensing image classification task, determines the classification result, extracts feature information of multiple different scales, uses the cross fusion method, improves the classification accuracy, introduces the attention mechanism to ignore redundant information, and improves the recognition efficiency.

[0008] An embodiment of the present invention provides a remote sensing image classification method based on an improved MobileNet v2, comprising:

[0009] Step 1: Obtain a remote sensing image classification dataset and divide it into a training set and a test set;

[0010] Step 2: Preprocess the remote sensing image classification data set to obtain processed data;

[0011] Step 3: Input the processed data into the MobileNet v2 network to obtain the first output feature;

[0012] Step 4: Perform multi-scale mixed convolution on the first output feature to obtain the second output feature;

[0013] Step 5: Perform feature cross fusion on the second output feature to obtain the third output feature;

[0014] Step 6: Calculate the weight of the third output feature to determine useful information;

[0015] Step 7: Input useful information into the fully connected layer for classification prediction;

[0016] Step 8: Perform multiple iterations based on the classification prediction results. When the requirements for stopping iterations are met, save the weight file, obtain the target network model, and determine the classification results based on the remote sensing image classification task of the target network model.

[0017] Preferably, step 2: preprocessing the remote sensing image classification data set to obtain processed data includes:

[0018] The remote sensing images in the remote sensing image classification dataset are aligned into RGB images of size 256*256 to obtain processed data.

[0019] Preferably, step 4: performing multi-scale mixed convolution on the first output feature to obtain the second output feature includes:

[0020] Configure the multi-scale mixed convolution module;

[0021] Inputting the first output feature into a multi-scale mixed convolution module to obtain a second output feature;

[0022] Among them, a multi-scale mixed convolution module is configured, including:

[0023] Set 1*1 normal convolution, 3*3 normal convolution, 3*3 dilated convolution with a dilation rate of 6, and 3*3 dilated convolution with a dilation rate of 12;

[0024] Set the BN layer and ReLU6 activation layer.

[0025] An embodiment of the present invention provides a remote sensing image classification method based on improved MobileNet v2, further comprising:

[0026] Get the required receptive field;

[0027] Calculate the current receptive field;

[0028] Adjust the dilation rate of the dilated convolution according to the required receptive field and the current receptive field;

[0029] Among them, calculating the current receptive field includes:

[0030] RF=k+(k-1)(r-1)

[0031] Among them, RF is the current receptive field, k is the size of the dilated convolution kernel, and r is the dilated rate of the dilated convolution.

[0032] Preferably, obtaining the required receptive field includes:

[0033] Obtain historical remote sensing image classification tasks, and extract the first task features of the historical remote sensing image classification tasks;

[0034] Determine the classification accuracy of historical remote sensing image classification tasks;

[0035] Construct a training sample vector based on the first task features and classification accuracy;

[0036] According to the training sample vector, a receptive field determination model is constructed;

[0037] Analyze the remote sensing image classification task to obtain the second task features and the required classification accuracy;

[0038] Construct an input vector according to the second task characteristics and the required classification accuracy;

[0039] The input vector is input into the receptive field determination model to obtain the desired receptive field.

[0040] Preferably, step 6: performing weight calculation on the third output feature to determine useful information includes:

[0041] Construct channel attention block;

[0042] Building a spatial attention block;

[0043] Input the third output feature into the channel attention block to obtain the channel attention feature output by the channel attention block;

[0044] Perform multiplication operation on the channel attention feature and the third output feature to obtain the input feature map;

[0045] Input the input feature map into the spatial attention block to obtain the spatial attention feature;

[0046] Multiply the input feature map and the spatial attention feature to obtain useful information;

[0047] Among them, constructing the channel attention block includes:

[0048] Setting the first adaptive global maximum pooling layer;

[0049] Setting the first adaptive global average pooling layer;

[0050] Set up a two-layer neural network. The first layer of the two-layer neural network is a 1*1 convolution and a Mi sh activation function, and the second layer of the two-layer neural network is a 1*1 convolution;

[0051] Set the first Mi sh activation function layer;

[0052] Among them, the spatial attention block is constructed, including:

[0053] Setting a second adaptive global maximum pooling layer;

[0054] Set the second adaptive global average pooling layer;

[0055] Set 7*7 convolution;

[0056] Set the second Mish activation function layer.

[0057] Preferably, the Mish activation function includes:

[0058] Mish = x*tanh(ln(1+e x )

[0059] Among them, Mish is the Mish activation function, and x is the feature map to be input into the activation function.

[0060] Preferably, step 1: obtaining a remote sensing image classification data set includes:

[0061] Extracting task features of remote sensing image classification tasks;

[0062] Build an acquisition index based on task characteristics;

[0063] According to the acquisition index, a first remote sensing image classification data subset is acquired through a local node;

[0064] According to the acquisition index, a second remote sensing image classification data subset is acquired through a big data node;

[0065] The first remote sensing image classification data subset and the second remote sensing image classification data subset are collectively used as a remote sensing image classification data set.

[0066] Preferably, according to the acquisition index, obtaining the second remote sensing image classification data subset through the big data node includes:

[0067] Get the node label of the big data node;

[0068] Determine a first trustworthy value of a big data node according to the node label;

[0069] Filtering a large data node whose first credibility value is greater than or equal to a preset first target threshold value and using it as the first target node;

[0070] The remaining big data nodes except the first target node in the big data nodes are used as second target nodes;

[0071] Acquiring the feedback characteristics of the data provided by the second target node;

[0072] According to the feedback characteristics, fitting the feedback characteristic curve;

[0073] Derivative calculation is performed on the feedback characteristic curve to obtain the feedback increase rate curve;

[0074] If the curve characteristics of the feedback increase rate curve meet the concave curve characteristics, the second target node is used as the third target node;

[0075] Determine the upward adjustment value of the feedback increase rate curve according to the concave curve characteristics of the third target node and the preset upward adjustment value determination library;

[0076] Determining a second credible value of the third target node according to the upward-adjusted value and the first credible value of the third target node;

[0077] If the second credible value is greater than or equal to the preset second target threshold, the corresponding third target node is used as the fourth target node;

[0078] According to the acquisition index, a second remote sensing image classification data subset is acquired through the first target node and the fourth target node.

[0079] An embodiment of the present invention provides a remote sensing image classification method based on improved MobileNet v2, further comprising:

[0080] Step 9: Obtain the warning rules for the remote sensing image classification task, and issue corresponding warnings based on the classification results and warning rules;

[0081] Among them, step 9: obtain the warning rules of the remote sensing image classification task, and make corresponding warnings according to the classification results and warning rules, including:

[0082] According to the warning rules, determine the warning classification characteristics;

[0083] According to the classification results, determine the actual classification features;

[0084] Match the actual classification features with the warning classification features. If there is a match, trigger the warning response measures of the corresponding warning classification features.

[0085] Get the response path set;

[0086] Parse the response path set and determine the response path characteristics, which include: measure matching degree, path response time, and path emergency dispatch time;

[0087] Determine the first response path in the response path set based on the early warning response measures;

[0088] Obtaining path confirmation information for each first response path;

[0089] Parse the path confirmation information, and if there is an unconfirmed first response path, determine the third response path according to the early warning response measures, the path characteristics of the confirmed first response path, and the path characteristics of the second response path in the response path set except the first response path;

[0090] Send early warning response measures to the third response path.

[0091] An embodiment of the present invention provides a remote sensing image classification system based on an improved MobileNet v2, comprising:

[0092] The data set acquisition subsystem is used to acquire the remote sensing image classification data set and divide the remote sensing image classification data set into a training set and a test set;

[0093] The preprocessing subsystem is used to preprocess the remote sensing image classification data set to obtain processed data;

[0094] A first output feature acquisition subsystem, used to input the processed data into the MobileNet v2 network to obtain a first output feature;

[0095] A multi-scale mixed convolution subsystem, used for performing multi-scale mixed convolution on the first output feature to obtain a second output feature;

[0096] A feature cross fusion subsystem, used for performing feature cross fusion on the second output feature to obtain a third output feature;

[0097] A parallel attention subsystem is used to calculate the weight of the third output feature and determine the useful information;

[0098] The classification prediction subsystem is used to input useful information into the fully connected layer for classification prediction;

[0099] The classification subsystem is used to perform multiple iterations based on the classification prediction results. When the requirements for stopping iterations are met, the weight file is saved, the target network model is obtained, and the classification results are determined based on the remote sensing image classification task of the target network model.

[0100] The beneficial effects of the present invention are:

[0101] The present invention preprocesses the acquired remote sensing image, and inputs the processed data obtained by the preprocessing into the MobileNet v2 network to obtain the first output feature, and sequentially performs multi-scale mixed convolution, feature cross fusion and weight calculation on the first output feature to determine useful information. The useful information is input into the fully connected layer for classification prediction, and multiple iterations are performed based on the difference between the predicted result and the actual result to determine the weight file configuration target network model that meets the iteration requirements, and the target network model performs the remote sensing image classification task to determine the classification result, extracts feature information of multiple different scales, uses the cross fusion method to improve the classification accuracy, introduces the attention mechanism to ignore redundant information, and improves the recognition efficiency.

[0102] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in the present application documents.

[0103] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0105] Figure 1 Schematic diagram of a remote sensing image classification method based on improved MobileNet v2 in an embodiment of the present invention;

[0106] Figure 2 A schematic diagram of visualizing classification results of remote sensing images in an embodiment of the present invention;

[0107] Figure 3 Schematic diagram of a remote sensing image classification system based on improved MobileNet v2 in an embodiment of the present invention. DETAILED DESCRIPTION

[0108] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0109] The embodiment of the present invention provides a remote sensing image classification method based on improved MobileNet v2, such as Figure 1 As shown, including:

[0110] Step 1: Obtain a remote sensing image classification dataset and divide it into a training set and a test set; the remote sensing image classification dataset is: a collection of remote sensing images with labeled classification results; the division rules of the training set and the test set are set manually;

[0111] Step 2: Preprocess the remote sensing image classification data set to obtain processed data; wherein the preprocessing is: processing the remote sensing image into data suitable for model training, such as: adjusting it to a size suitable for model input;

[0112] Step 3: Input the processed data into the MobileNet v2 network to obtain the first output feature; wherein the first output feature is: a feature representation of the output image of the MobileNet v2 network;

[0113] Step 4: Perform multi-scale mixed convolution on the first output feature to obtain a second output feature; wherein the second output feature is: a feature obtained by extracting the first output feature by combining convolution kernels of different scales;

[0114] Step 5: Perform feature cross fusion on the second output features to obtain a third output feature; wherein the third output feature is: a feature obtained by combining the second output features of different levels or different sources;

[0115] Step 6: Calculate the weight of the third output feature to determine useful information; the weight calculation is: adding an attention mechanism can automatically learn the weight of the feature map, pay more attention to the key parts of the image, and ignore redundant information; the useful information is: the result obtained after the third output feature removes redundant information;

[0116] Step 7: Input the useful information into the fully connected layer for classification prediction; wherein the classification prediction is: the result obtained by classifying the remote sensing image according to the useful information;

[0117] Step 8: Perform multiple iterations based on the classification prediction results. When the requirements for stopping iterations are met, save the weight file, obtain the target network model, and determine the classification results based on the remote sensing image classification task of the target network model. Among them, performing multiple iterations based on the classification prediction results means: testing according to the test set, judging the classification accuracy, adjusting the model training parameters in the direction of improving the accuracy, and iterating the model. The requirements for stopping iterations are manually preset, for example: the accuracy reaches 98%; the weight file is: a file containing all weights and bias parameters learned during the model training process; the target network model is: a remote sensing image classification model based on the improved MobileNet v2, which deploys the weight file; the remote sensing image classification task is: input the remote sensing image to be classified and the classification requirements of the target network model; the classification result is: the execution result of the target network model for the remote sensing image classification task, and the visualization diagram of the classification result is as follows Figure 2 shown.

[0118] The working principle and beneficial effects of the above technical solution are:

[0119] This application preprocesses the acquired remote sensing images, and inputs the processed data obtained by the preprocessing into the MobileNet v2 network to obtain the first output feature, and performs multi-scale mixed convolution, feature cross fusion and weight calculation on the first output feature in sequence to determine useful information. The useful information is input into the fully connected layer for classification prediction, and multiple iterations are performed based on the difference between the predicted results and the actual results to determine the weight file configuration target network model that meets the iteration requirements. The target network model performs the remote sensing image classification task, determines the classification result, extracts feature information of multiple different scales, uses the cross fusion method to improve the classification accuracy, introduces the attention mechanism to ignore redundant information, and improves the recognition efficiency.

[0120] On the datasets WHU-RS19, RSSCN7 and SIRI-WHU, the overall classification accuracy (OA) of the present method reached 98.19%, 94.18% and 96.37% respectively. As shown in Tables 1-3: Table 1 is a comparison of the overall classification accuracy of the present invention with other methods on the dataset WHU-RS19, Table 2 is a comparison of the overall classification accuracy of the present invention with other methods on the dataset RSSCN7, and Table 3 is a comparison of the overall classification accuracy of the present invention with other methods on the dataset SIRI-WHU;

[0121]

[0122] Table 1

[0123]

[0124] Table 2

[0125]

[0126] Table 3

[0127] In one embodiment, step 2: preprocessing the remote sensing image classification data set to obtain processed data includes:

[0128] The remote sensing images in the remote sensing image classification dataset are aligned into 256*256 RGB images to obtain processed data. The 256*256 size refers to the image resolution, that is, the width and height of the image are both 256 pixels.

[0129] The working principle and beneficial effects of the above technical solution are:

[0130] This application aligns the remote sensing images in the remote sensing image classification dataset into RGB images of size 256*256, obtains processed data ready to be input into the machine learning model for training, and improves the suitability of subsequent training.

[0131] In one embodiment, step 4: performing multi-scale mixed convolution on the first output feature to obtain the second output feature includes:

[0132] Configure the multi-scale mixed convolution module;

[0133] Inputting the first output feature into a multi-scale mixed convolution module to obtain a second output feature;

[0134] Among them, a multi-scale mixed convolution module is configured, including:

[0135] Set 1*1 normal convolution, 3*3 normal convolution, 3*3 dilated convolution with a dilation rate of 6, and 3*3 dilated convolution with a dilation rate of 12. Among them, dilated convolution increases the receptive field by inserting skipped units in the convolution kernel. The dilation rate indicates the number and distribution of holes in the convolution kernel. For example, a dilation rate of 6 means that there is an actual value in every 6 convolution kernel units, and the rest are holes.

[0136] Set the BN layer and ReLU6 activation layer. The BN layer is: batch normalization layer.

[0137] The working principle and beneficial effects of the above technical solution are:

[0138] In order to solve the problem of large differences in target feature scales in remote sensing images, this application proposes a multi-scale hybrid convolution module to extract target semantic information of different scales. This module consists of four different convolutions, namely 1*1 ordinary convolution, 3*3 ordinary convolution, 3*3 dilated convolution with a void rate of 6, and 3*3 dilated convolution with a void rate of 12. After convolution, the output is obtained through the BN layer and the ReLU6 activation function. The receptive field of branch 1 in the multi-scale hybrid convolution module is 1*1, the receptive field of branch 2 is 3*3, the receptive field of branch 3 is 23*23, and the receptive field of branch 4 is 47*47. The receptive fields of the four branches are different, and feature information of different scales can be extracted to solve the problem of large differences in target feature scales.

[0139] The embodiment of the present invention provides a remote sensing image classification method based on improved MobileNet v2, further comprising:

[0140] Obtain the required receptive field; wherein the required receptive field is: the minimum receptive field size determined by the technician according to the classification task;

[0141] Calculate the current receptive field; where the current receptive field is: the actual size of the input image area covered by the neuron;

[0142] Adjust the dilation rate of the dilated convolution according to the required receptive field and the current receptive field. The dilated convolution can bring different receptive fields by adjusting the dilation rate, and adjusting the dilation rate makes the current receptive field approach the required receptive field.

[0143] Among them, calculating the current receptive field includes:

[0144] RF=k+(k-1)(r-1)

[0145] Among them, RF is the current receptive field, k is the size of the dilated convolution kernel, and r is the dilated rate of the dilated convolution.

[0146] The working principle and beneficial effects of the above technical solution are:

[0147] The present application obtains the required receptive field determined by the technician according to the classification task, and in addition, calculates the current receptive field. The dilation rate of the dilated convolution is adjusted according to the required receptive field and the current receptive field, thereby improving the rationality of the receptive field setting.

[0148] In one embodiment, obtaining a desired receptive field includes:

[0149] Obtain historical remote sensing image classification tasks and extract the first task features of the historical remote sensing image classification tasks; wherein the historical remote sensing image classification tasks are: classification work of remote sensing images that have been completed in the past; the first task features are: the characteristic representation of the historical remote sensing image classification tasks, such as: task type, what kind of neural network to set and how large the receptive field is, etc.;

[0150] Determine the classification accuracy of the historical remote sensing image classification task; where the classification accuracy is: the quantitative result of the accuracy of the classification result compared with the requirement;

[0151] According to the first task feature and the classification accuracy, a training sample vector is constructed; wherein the training sample vector is: a vector composed of the first task feature and the classification accuracy, and the distribution position of the vector elements is manually determined;

[0152] According to the training sample vector, a receptive field determination model is constructed; wherein the receptive field determination model is: an AI model that determines the optimal receptive field size;

[0153] Analyze the remote sensing image classification task to obtain the second task feature and the required classification accuracy; the second task feature is: the feature representation of the remote sensing image classification task; the required classification accuracy is: the classification accuracy expected to be achieved by the remote sensing image classification task;

[0154] According to the second task feature and the required classification accuracy, an input vector is constructed; wherein the distribution positions of the vector elements of the input vector are the same as those of the training sample vector;

[0155] The input vector is input into the receptive field determination model to obtain the desired receptive field.

[0156] The working principle and beneficial effects of the above technical solution are:

[0157] The present application extracts the first task features in the historical remote sensing image classification task, and also determines the classification accuracy of the historical remote sensing image classification task. A training sample vector is constructed based on the first task features and the classification accuracy. A receptive field determination model is introduced, and the training sample vector is input into the receptive field determination model. Based on the same construction rules as the training sample vector, the input vector is determined according to the acquired second task features and the required classification accuracy. The input vector is input into the receptive field determination model to obtain the required receptive field, thereby improving the suitability of the required receptive field determination process.

[0158] In one embodiment, step 6: weighting the third output feature to determine useful information includes:

[0159] Construct channel attention block;

[0160] Building a spatial attention block;

[0161] Input the third output feature into the channel attention block to obtain the channel attention feature output by the channel attention block;

[0162] Perform multiplication operation on the channel attention feature and the third output feature to obtain the input feature map;

[0163] Input the input feature map into the spatial attention block to obtain the spatial attention feature;

[0164] Multiply the input feature map and the spatial attention feature to obtain useful information;

[0165] Among them, constructing the channel attention block includes:

[0166] Setting the first adaptive global maximum pooling layer;

[0167] Setting the first adaptive global average pooling layer;

[0168] Set up a two-layer neural network. The first layer of the two-layer neural network is a 1*1 convolution and a Mi sh activation function, and the second layer of the two-layer neural network is a 1*1 convolution;

[0169] Set the first Mish activation function layer;

[0170] Among them, the spatial attention block is constructed, including:

[0171] Setting a second adaptive global maximum pooling layer;

[0172] Set the second adaptive global average pooling layer;

[0173] Set 7*7 convolution;

[0174] Set the second Mish activation function layer.

[0175] The working principle and beneficial effects of the above technical solution are:

[0176] The channel attention block performs adaptive global maximum pooling and adaptive global average pooling on the input feature map from the height and width dimensions respectively, obtains feature maps of two dimensions, and then feeds them into a two-layer neural network. The first layer is a 1*1 convolution and activation function, and the second layer is a 1*1 convolution. Then the output features are added and the final channel attention features are generated through the activation function operation. Finally, the channel attention features are multiplied with the input features, and the generated feature map will be used as the input feature map of the spatial attention mechanism block.

[0177] The spatial attention mechanism block performs a channel-based adaptive global maximum pooling and an adaptive global average pooling on the input feature map to obtain two feature maps. The two feature maps are then channel-joined, and a 7*7 convolution and activation function are performed to generate the spatial attention feature. Finally, the spatial attention feature is multiplied by the input feature map.

[0178] This application introduces channel attention blocks and spatial attention blocks. The addition of the attention mechanism can automatically learn the weights of the feature maps, making them pay more attention to the key parts of the image and ignore redundant information, thereby improving the classification accuracy under complex backgrounds and solving the problem that the detection network is sensitive to background interference.

[0179] In one embodiment, the Mish activation function includes:

[0180] Mish = x*tanh(ln(1+e x )

[0181] Among them, Mish is the Mish activation function, and x is the feature map to be input into the activation function.

[0182] The working principle and beneficial effects of the above technical solution are:

[0183] This application introduces the Mish activation function to capture the relationship between convolution outputs. The Mish activation function has no upper bound, which can ensure that there is no saturation area, so there will be no gradient vanishing problem during training; the Mish activation function has no lower bound, which helps to achieve regularization effect; the Mish activation function is non-monotonic, and some smaller negative inputs can be retained as negative outputs to improve the interpretability and gradient flow of the network. The Mish activation function is smooth, has good generalization ability and effective optimization ability of the results, and can improve the quality of the results.

[0184] In one embodiment, step 1: obtaining a remote sensing image classification dataset includes:

[0185] Extracting the task features of the remote sensing image classification task; wherein the task features are: the task type and task content of the remote sensing image classification task;

[0186] According to the task characteristics, an acquisition index is constructed; wherein the acquisition index is: an index for tracking and accessing remote sensing image classification data related to the task type and task content;

[0187] According to the acquisition index, a first remote sensing image classification data subset is acquired through a local node; wherein the local node is: a communication node of a local remote sensing image library, and the local remote sensing image library stores remote sensing images that have been classified and marked locally;

[0188] According to the obtained index, a second remote sensing image classification data subset is obtained through a big data node; wherein the big data node is a big data platform for storing remote sensing data, such as a GPS navigation platform, or a forest fire prevention remote sensing monitoring platform;

[0189] The first remote sensing image classification data subset and the second remote sensing image classification data subset are collectively used as a remote sensing image classification data set.

[0190] The working principle and beneficial effects of the above technical solution are:

[0191] This application introduces the task characteristics of remote sensing image classification tasks. Based on the task characteristics, an acquisition index for tracking and accessing remote sensing image classification data related to the task type and task content is constructed. According to the acquisition index, remote sensing image classification data sets are acquired at local nodes and big data nodes respectively. The comprehensiveness of remote sensing image classification data set acquisition is improved.

[0192] In one embodiment, according to the acquisition index, the second remote sensing image classification data subset is acquired through the big data node, including:

[0193] Obtain a node label of a big data node; wherein the node label is: a network identifier of the big data node;

[0194] Determine a first trust value of the big data node according to the node label; wherein the first trust value is: a trust evaluation value of the big data node stored in a public evaluation system of an Internet platform according to the node label;

[0195] Screening a large data node whose first credibility value is greater than or equal to a preset first target threshold value and using it as a first target node; wherein the preset first target threshold value is manually preset;

[0196] The remaining big data nodes except the first target node in the big data nodes are used as second target nodes;

[0197] Acquire feedback features of the data provision record of the second target node; wherein the data provision record is: a record of the second target node providing data on the big data platform; the feedback features are: an evaluation value of a data user of the data provision record;

[0198] According to the feedback characteristics, a feedback characteristic curve is fitted; wherein fitting the feedback characteristic curve refers to: using a mathematical method (such as polynomial fitting) to establish a curve describing the trend of feedback characteristics changing over time;

[0199] The feedback characteristic curve is derivatized to obtain a feedback increase rate curve; wherein the feedback increase rate curve is: a result obtained by derivatizing the feedback characteristic curve with respect to the time variable;

[0200] If the curve characteristics of the feedback increase rate curve meet the concave curve characteristics, the second target node is used as the third target node;

[0201] Determine the upward adjustment value of the feedback rate increase curve according to the concave curve feature of the third target node and the preset upward adjustment value determination library; wherein the preset upward adjustment value determination library includes: a plurality of one-to-one corresponding concave curve features to be matched and upward adjustment values ​​to be matched;

[0202] Determine a second credible value of the third target node according to the increased value and the first credible value of the third target node; wherein the second credible value is: the increased value multiplied by the first credible value;

[0203] If the second credible value is greater than or equal to a preset second target threshold, the corresponding third target node is used as the fourth target node; wherein the preset second target threshold is manually preset.

[0204] According to the acquisition index, a second remote sensing image classification data subset is acquired through the first target node and the fourth target node.

[0205] The working principle and beneficial effects of the above technical solution are:

[0206] The present application obtains the network identification of the big data node, and reads the first credible value stored in the public evaluation system of the big data node on the Internet platform according to the network identification. The first target node whose first credible value is greater than or equal to the first target threshold is screened out. The feedback characteristics of the data provided by the second target node are obtained, and the feedback characteristic curve is fitted. The feedback rate curve is derived to obtain the feedback rate curve. The curve characteristics of the feedback rate curve are extracted. If the curve characteristics meet the concave curve characteristics, the corresponding second target node is used as the third target node. The upward value determination library is introduced, and the upward value is determined according to the concave curve characteristics of the third target node. The upward value is multiplied by the first credible value to obtain the second credible value, and the fourth target node whose second credible value is greater than or equal to the second target threshold is screened. Based on the acquisition index, the second remote sensing image classification data subset is jointly obtained through the first target node and the fourth target node. The screening process of the big data node is more suitable, which improves the availability of the training data.

[0207] The embodiment of the present invention provides a remote sensing image classification method based on improved MobileNet v2, further comprising:

[0208] Step 9: Obtain the warning rules for the remote sensing image classification task, and make corresponding warnings according to the classification results and the warning rules; wherein the warning rules are: the standards or conditions used to determine when to trigger the warning in the remote sensing image classification;

[0209] Among them, step 9: obtain the warning rules of the remote sensing image classification task, and make corresponding warnings according to the classification results and warning rules, including:

[0210] According to the warning rules, the warning classification features are determined; wherein the warning classification features are: the characteristic representation of the classification results corresponding to the warning rules that will issue warnings;

[0211] According to the classification result, the actual classification feature is determined; wherein the actual classification feature is: the result obtained by characterizing the classification result;

[0212] Match the actual classification features with the warning classification features. If there is a match, trigger the warning response measures of the corresponding warning classification features; the warning response measures are: what measures to take to respond to the warning;

[0213] Obtaining a response path set; wherein the response path set includes: a set of communication node paths of responders that can be scheduled by the remote sensing image classification platform;

[0214] Parse the response path set and determine the response path characteristics. The response path characteristics include: measure matching degree, path response time and path emergency dispatch time. Among them, the measure matching degree is determined according to the matching degree between the measures that can be executed by the response path and the early warning response measures; the path response time is: the time range within which the response path can be dispatched; the path emergency dispatch time is: the time required for the response path to be temporarily online;

[0215] According to the early warning response measures, determine a first response path in the response path set; wherein the first response path is: a communication node of the responder that can execute the early warning response measures;

[0216] Obtaining path confirmation information for each first response path; wherein the path confirmation information is: relevant information for the responder to confirm whether to execute the assigned early warning response measures in the future;

[0217] Parse the path confirmation information, if there is a first response path that is not confirmed, determine the third response path according to the early warning response measures, the path characteristics of the confirmed first response path, and the path characteristics of the second response path other than the first response path in the response path set; wherein the third response path is: the alternate response path determined when the first response path is not answered; when determining the third response path, determine the unmatched measure part according to the early warning response measures and the path characteristics of the confirmed first response path, and match the third response path according to the unmatched measure part and the path characteristics of the second response path other than the first response path in the response path set;

[0218] Send early warning response measures to the third response path.

[0219] The working principle and beneficial effects of the above technical solution are:

[0220] This application introduces early warning rules to determine the early warning classification features of the classification results that will issue an early warning. According to the classification results, the actual classification features are determined. The actual classification features and the early warning classification features are matched to determine the early warning response measures that match the early warning classification features. Obtain a set of communication node paths of the responders that can be scheduled by the remote sensing image classification platform (response path set). Determine the response path features of each response path and preliminarily match the first response path. Due to the temporary nature of the early warning, the first response path may not always be able to give feedback in a timely manner. Therefore, the path confirmation information of each first response path is parsed. If there is an unconfirmed path, the unmatched measure part is determined based on the early warning response measures and the path features of the confirmed first response path. According to the unmatched measure part and the path features of the second response path in the response path set except the first response path, the third response path is matched, and the early warning response measures are sent to the third response path. When the response path cannot respond in time, an alternative plan is determined, thereby improving the reliability of the execution of the early warning response measures.

[0221] The embodiment of the present invention provides a remote sensing image classification system based on improved MobileNet v2, such as Figure 3 As shown, including:

[0222] The data set acquisition subsystem 1 is used to acquire the remote sensing image classification data set and divide the remote sensing image classification data set into a training set and a test set;

[0223] The preprocessing subsystem 2 is used to preprocess the remote sensing image classification data set to obtain processed data;

[0224] A first output feature acquisition subsystem 3, used to input the processed data into the MobileNet v2 network to obtain a first output feature;

[0225] A multi-scale mixed convolution subsystem 4, used for performing a multi-scale mixed convolution on the first output feature to obtain a second output feature;

[0226] A feature cross fusion subsystem 5 is used to perform feature cross fusion on the second output feature to obtain a third output feature;

[0227] A parallel attention subsystem 6, for weighting the third output feature to determine useful information;

[0228] Classification prediction subsystem 7, used to input useful information into the fully connected layer for classification prediction;

[0229] The classification subsystem 8 is used to perform multiple iterations according to the classification prediction results. When the requirements for stopping iteration are met, the weight file is saved, the target network model is obtained, and the classification result is determined based on the remote sensing image classification task of the target network model.

[0230] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A remote sensing image classification method based on improved MobileNet v2, characterized in that: include: Step 1: Obtain a remote sensing image classification dataset and divide the remote sensing image classification dataset into a training set and a test set; Step 2: Preprocess the remote sensing image classification data set to obtain processed data; Step 3: Input the processed data into the MobileNet v2 network to obtain the first output feature; Step 4: Perform multi-scale mixed convolution on the first output feature to obtain the second output feature; The step 4: performing multi-scale mixed convolution on the first output feature to obtain the second output feature includes: Configure a multi-scale mixed convolution module; Inputting the first output feature into a multi-scale mixed convolution module to obtain a second output feature; Among them, a multi-scale mixed convolution module is configured, including: Set 1*1 normal convolution, 3*3 normal convolution, 3*3 dilated convolution with a dilation rate of 6, and 3*3 dilated convolution with a dilation rate of 12. The receptive fields of these four branches are different. Set the BN layer and ReLU6 activation layer; Step 5: Perform feature cross fusion on the second output features to obtain the third output features; feature cross fusion is: combining the second output features of different levels or different sources; Step 6: Calculate the weight of the third output feature to determine useful information; Step 7: Input useful information into the fully connected layer for classification prediction; Step 8: Perform multiple iterations based on the classification prediction results. When the iteration stop requirement is met, save the weight file, obtain the target network model, and determine the classification results based on the remote sensing image classification task of the target network model. The remote sensing image classification method further comprises: Get the required receptive field; Calculate the current receptive field; Adjust the dilation rate of the dilated convolution according to the required receptive field and the current receptive field; Among them, calculating the current receptive field includes: in, is the current receptive field, is the size of the dilated convolution kernel, is the dilation rate of the dilated convolution; The obtaining of the required receptive field includes: Obtain historical remote sensing image classification tasks and extract the first task features of the historical remote sensing image classification tasks; Determine the classification accuracy of historical remote sensing image classification tasks; Construct a training sample vector based on the first task features and classification accuracy; According to the training sample vector, a receptive field determination model is constructed; Analyze the remote sensing image classification task to obtain the second task features and the required classification accuracy; Construct an input vector according to the second task characteristics and the required classification accuracy; The input vector is input into the receptive field determination model to obtain the desired receptive field.

2. A remote sensing image classification method based on improved MobileNet v2 as claimed in claim 1, characterized in that: Step 2: Preprocess the remote sensing image classification data set to obtain processed data, including: The remote sensing images in the remote sensing image classification dataset are aligned into RGB images of size 256*256 to obtain processed data.

3. A remote sensing image classification method based on improved MobileNet v2 as claimed in claim 1, characterized in that: Step 6: Calculate the weight of the third output feature to determine useful information, including: Construct channel attention block; Building a spatial attention block; Input the third output feature into the channel attention block to obtain the channel attention feature output by the channel attention block; Perform multiplication operation on the channel attention feature and the third output feature to obtain the input feature map; Input the input feature map into the spatial attention block to obtain the spatial attention feature; Multiply the input feature map and the spatial attention feature to obtain useful information; Among them, constructing the channel attention block includes: Setting the first adaptive global maximum pooling layer; Setting the first adaptive global average pooling layer; Set up a two-layer neural network. The first layer of the two-layer neural network is 1*1 convolution and Mish activation function, and the second layer of the two-layer neural network is 1*1 convolution; Set the first Mish activation function layer; Among them, the spatial attention block is constructed, including: Setting a second adaptive global maximum pooling layer; Set the second adaptive global average pooling layer; Set 7*7 convolution; Set the second Mish activation function layer.

4. A remote sensing image classification method based on improved MobileNet v2 as claimed in claim 1, characterized in that: Mish activation function, including: in, is the Mish activation function, is the feature map to be input into the activation function.

5. The remote sensing image classification method based on improved MobileNet v2 as claimed in claim 1, characterized in that: Step 1: Obtain remote sensing image classification dataset, including: Extracting task features of remote sensing image classification tasks; Build an acquisition index based on task characteristics; According to the acquisition index, a first remote sensing image classification data subset is acquired through a local node; According to the acquisition index, a second remote sensing image classification data subset is acquired through a big data node; The first remote sensing image classification data subset and the second remote sensing image classification data subset are collectively used as a remote sensing image classification data set.

6. A remote sensing image classification method based on improved MobileNet v2 as claimed in claim 5, characterized in that: According to the acquisition index, the second remote sensing image classification data subset is obtained through the big data node, including: Get the node label of the big data node; Determine a first trustworthy value of a big data node according to the node label; Filtering a large data node whose first credibility value is greater than or equal to a preset first target threshold value and using it as the first target node; The remaining big data nodes except the first target node in the big data nodes are used as second target nodes; Acquire feedback features of the data provision record of the second target node; the feedback features are: evaluation values ​​of the data user of the data provision record; According to the feedback characteristics, a feedback characteristic curve is fitted; fitting the feedback characteristic curve means using polynomial fitting to establish a curve describing the trend of feedback characteristics changing over time; The feedback characteristic curve is derived to obtain a feedback increase rate curve; the feedback increase rate curve is: the result obtained by derivation of the feedback characteristic curve with respect to the time variable; If the curve characteristics of the feedback increase rate curve meet the concave curve characteristics, the second target node is used as the third target node; Determine the upward adjustment value of the feedback rate increase curve according to the concave curve feature of the third target node and a preset upward adjustment value determination library; the preset upward adjustment value determination library includes: a plurality of one-to-one corresponding concave curve features to be matched and upward adjustment values ​​to be matched; Determining a second credible value of the third target node according to the upward-adjusted value and the first credible value of the third target node; If the second credible value is greater than or equal to the preset second target threshold, the corresponding third target node is used as the fourth target node; According to the acquisition index, a second remote sensing image classification data subset is acquired through the first target node and the fourth target node.

7. A remote sensing image classification method based on improved MobileNet v2 as claimed in claim 1, characterized in that: Also includes: Step 9: Obtain the warning rules for the remote sensing image classification task, and issue corresponding warnings based on the classification results and warning rules; Among them, step 9: obtain the warning rules of the remote sensing image classification task, and make corresponding warnings according to the classification results and warning rules, including: According to the warning rules, determine the warning classification characteristics; According to the classification results, determine the actual classification features; Match the actual classification features with the warning classification features. If there is a match, trigger the warning response measures of the corresponding warning classification features. Get the response path set; Parse the response path set and determine the response path characteristics, which include: measure matching degree, path response time, and path emergency dispatch time; Determine the first response path in the response path set based on the early warning response measures; Obtaining path confirmation information for each first response path; Parse the path confirmation information, and if there is an unconfirmed first response path, determine the third response path according to the early warning response measures, the path characteristics of the confirmed first response path, and the path characteristics of the second response path in the response path set except the first response path; Send early warning response measures to the third response path.

8. A remote sensing image classification system based on improved MobileNet v2, characterized in that: include: The data set acquisition subsystem is used to acquire the remote sensing image classification data set and divide the remote sensing image classification data set into a training set and a test set; The preprocessing subsystem is used to preprocess the remote sensing image classification data set to obtain processed data; A first output feature acquisition subsystem, used to input the processed data into the MobileNet v2 network to obtain a first output feature; A multi-scale mixed convolution subsystem, used for performing multi-scale mixed convolution on the first output feature to obtain a second output feature; The multi-scale mixed convolution subsystem performs multi-scale mixed convolution on the first output feature to obtain the second output feature, and performs the following operations: Configure a multi-scale mixed convolution module; Inputting the first output feature into a multi-scale mixed convolution module to obtain a second output feature; Among them, a multi-scale mixed convolution module is configured, including: Set 1*1 normal convolution, 3*3 normal convolution, 3*3 dilated convolution with a dilation rate of 6, and 3*3 dilated convolution with a dilation rate of 12. The receptive fields of these four branches are different. Set the BN layer and ReLU6 activation layer; The feature cross fusion subsystem is used to perform feature cross fusion on the second output feature to obtain the third output feature; feature cross fusion is: combining the second output features of different levels or different sources; A parallel attention subsystem is used to calculate the weight of the third output feature and determine the useful information; The classification prediction subsystem is used to input useful information into the fully connected layer for classification prediction; The classification subsystem is used to perform multiple iterations according to the classification prediction results. When the requirements for stopping iteration are met, the weight file is saved, the target network model is obtained, and the classification result is determined based on the remote sensing image classification task of the target network model; The remote sensing image classification system further performs the following operations: Get the required receptive field; Calculate the current receptive field; Adjust the dilation rate of the dilated convolution according to the required receptive field and the current receptive field; Among them, calculating the current receptive field includes: in, is the current receptive field, is the size of the dilated convolution kernel, is the dilation rate of the dilated convolution; The obtaining of the required receptive field includes: Obtain historical remote sensing image classification tasks and extract the first task features of the historical remote sensing image classification tasks; Determine the classification accuracy of historical remote sensing image classification tasks; Construct a training sample vector based on the first task features and classification accuracy; According to the training sample vector, a receptive field determination model is constructed; Analyze the remote sensing image classification task to obtain the second task features and the required classification accuracy; Construct an input vector according to the second task characteristics and the required classification accuracy; The input vector is input into the receptive field determination model to obtain the desired receptive field.

Images