Intelligent sand extraction method and system of multispectral remote sensing image considering waveband information

Abstract

The application discloses a kind of sand ground intelligent extraction method and system of multispectral remote sensing image considering waveband information, which comprises: 1) in U-Net model, obtain Y-Net model by adding deformable convolution, training, prediction and performance evaluation are carried out on each waveband sand ground image data set;2) according to the precision degree of sand ground information extraction of each waveband, determine the waveband weight value, and weight reconstruction is carried out to the original multispectral remote sensing image covering sand ground;3) expert annotation is carried out to the new image after weight reconstruction, and the waveband combination multispectral remote sensing image data set is constructed;4) the Y-Net network model is trained on the waveband fusion multispectral remote sensing image data set, and the best parameter of the Y-Net network model of combined waveband is obtained;5) the sand ground remote sensing image to be identified is predicted.The present application solves the problems of low utilization efficiency and poor universality of existing waveband by using spectral characteristics combined with deep convolutional neural network, and improves the extraction accuracy of ground objects.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of deep convolutional neural network in multispectral remote sensing image information extraction after band combination, and particularly relates to a multispectral remote sensing image sand intelligent extraction method considering band information. BACKGROUND

[0002] Multispectral remote sensing image refers to the image generated by using multiple sensors to observe the earth, and the obtained image data contains spectral information of multiple sensors. The specific spectral information of multispectral image can not only reflect the reflectivity of the object to the spectral information, but also can make up for the influence on sand extraction performance in the case of insufficient light. Compared with visible light image, multispectral image can obtain spatial information and spectral dimension information at the same time, and it exists in a multi-dimensional three-dimensional form. In addition to imaging in the visible light band, it can also image in the near-infrared, thermal infrared, ultraviolet and other bands at the same time.

[0003] At present, many studies are based on visual interpretation and remote sensing image information extraction means to monitor the desertification process. Through field investigation and experience judgment, the remote sensing data is interpreted to determine the desertification index. Although the visual interpretation accuracy is relatively accurate, due to the problems of large mobility and wide range of desert, it is difficult to meet the monitoring of sand changes in a wide range. Remote sensing feature intelligent extraction based on deep learning technology is more extensive, and the most extensive model is the fully convolutional neural network (FCN) model proposed in 2015. U-Net is the first improved model of it, which reduces the data cost while ensuring the extraction accuracy, but it is mainly applied to medical image binary classification extraction. In the case that sand and background features have great spectral similarity, the sand recognition accuracy based on visible light image will be greatly reduced.

[0004] In view of this, the present application provides a multispectral remote sensing image sand intelligent extraction method considering band information, which combines deformable convolution and U-Net model to construct a network model Y-Net suitable for sand extraction. At the same time, the band combination method is used to consider the image band information on the premise of considering the sand contour information, and the high robustness of sand information extraction is realized. SUMMARY

[0005] The present application provides a multispectral remote sensing image sand intelligent extraction method considering band information to solve the problems in the prior art.

[0006] In order to achieve the above purpose, the present application provides the following technical scheme:

[0007] On the one hand, a multispectral remote sensing image sand intelligent extraction method considering band information comprises the following steps:

[0008] S101: adding a deformable convolution in a U-Net network model to obtain a Y-Net network model; and training, predicting and performance evaluating the Y-Net network model on each band sand remote sensing image with known sand information to obtain the precision degree of each band sand remote sensing image for sand information extraction;

[0009] S102: determining the weight value of each band sand remote sensing image according to the precision degree of each band sand remote sensing image for sand information extraction, weighting and reconstructing the original multi-spectral remote sensing image covering the sand land to generate a new sand remote sensing image;

[0010] S103: expertly labeling the sand information in the weighted and reconstructed remote sensing image to generate a sample label and construct a band fusion multi-spectral sand remote sensing image dataset;

[0011] S104: training the Y-Net network model on the band fusion multi-spectral remote sensing image dataset to obtain the best parameters of the Y-Net network model of the combined band;

[0012] S105: using the Y-Net network model of the combined band with the best parameters to predict the sand remote sensing image to be identified to obtain the sand information detection result of different years;

[0013] The sand remote sensing image to be identified is obtained by weighting and reconstructing each band sand image to be identified according to S102.

[0014] Further, the Y-Net network model is composed of 8 3x3 deformable convolution modules and 4 maximum pooling modules; wherein the deformable convolution module refers to an offset added to the sampling position in the standard convolution operation, and the offset is obtained by learning image features through an additional convolution layer;

[0015] The offset added to the sampling position in the deformable convolution allows the convolution kernel to expand to a large range in the training process, and is not limited to fixed grid sampling. The conventional convolution operation can mainly be divided into two parts: (1) sampling on the input feature map using a regular grid R; (2) performing a weighted operation, R defines the size and expansion of the receptive field, as follows

[0016] R = {(-1, -1), (-1, 0),..., (0, 1), (0, -1)}

[0017] The offset is obtained by learning image features through an additional convolution layer, and the offset size learned by different image features through convolution is different.

[0018] The general convolution formula is as follows, for each position p0 on the output feature map y:

[0019]

[0020] wherein p k is the enumeration of sampling positions in the grid k, x is the input feature image, W k is the corresponding convolution kernel weight coefficient, y(p0) represents the output feature image of p0 point, and p0 is the output feature image point.

[0021] The deformable convolution formula with an offset is as follows, for each position p0 on the output feature map y:

[0022]

[0023] wherein p k is the enumeration of sampling positions in the grid k, Δp k represents the offset in the deformable convolution module, x is the input feature image, W k is the corresponding convolution kernel weight coefficient, y(p0) represents the output feature image of p0 point, and p0 is the output feature image point.

[0024] Further, the Y-Net network model is used in S101 to extract features of sand information in a sand remote sensing image, and the specific process is as follows:

[0025] First, the different convolutions of the feature extraction network in the Y-Net network model are used to extract preliminary feature images of a certain band of sand original image at the sampling point position:

[0026]

[0027] wherein I is the input certain band of sand original remote sensing image, M is the convolution kernel, p is the output feature image point, r is the width of the image, a and b are the horizontal and vertical offsets of the input certain band of sand original remote sensing image I, x and y are the horizontal and vertical coordinate values of a certain point in I; and X(p) represents the image value of the point (x, y) in the certain band of sand original remote sensing image I in the output preliminary feature image corresponding to p.

[0028] Then, a standard regular grid point k is used for feature sampling on the preliminary feature image, and a deformable convolution is used to obtain a high-order feature image y:

[0029]

[0030] wherein W k is the convolution kernel weight coefficient of the grid point k, the grid point k represents the sampling position, y(p) represents the output high-order feature image value of p point, p0 is the original sampling position, and p kdenotes the position of the sampling point in the adaptive deformable convolution kernel, s k denotes p k denotes the learning rate of p, Δp k denotes the offset in the deformable convolution, Δm k denotes the adjustable amount, c k denotes the adjustable amount Δm k denotes the learning rate of s, s k ∈ [0, 1], Δm k ∈ [0, 1], c k ∈ [0, 1].

[0031] Further, the process of determining the precision degree of each band sand remote sensing image on sand information extraction is as follows:

[0032] The Y-Net network model is used for training on each band sand remote sensing image data set, and the optimal model parameters of the Y-Net network model of each band are obtained.

[0033] Based on the optimal model parameters of the Y-Net network model of each band, the each band sand remote sensing image data test set is predicted, and the each band sand information prediction result is obtained.

[0034] The performance evaluation index is used to evaluate the each band sand information extraction result, and the precision degree of each band sand remote sensing image on sand information extraction is obtained.

[0035] The precision degree refers to which band image is more accurate for extracting sand information, which is selected by the Y-Net network model;

[0036] The precision degree is a comprehensive calculation value of several performance evaluation indexes;

[0037] In order to realize the objective evaluation of the precision degree of each band sand extraction result, the performance evaluation index, that is, the harmonic average of the precision and the recall, is used to obtain the evaluation index F1 value, and the sand information extraction result is evaluated. The greater the F1 index value is, the higher the sand extraction precision of the band is, and the precision degree is the F1 index value.

[0038] The original multi-spectral remote sensing image covering the sand land is obtained from the geographic spatial data cloud website.

[0039] Further, the precision degree of each band sand remote sensing image on sand information extraction in S102 is used to determine the band weight, that is, the precision degree of each band sand remote sensing image on sand information extraction is used to calculate the band weight by using the variance reciprocal method.

[0040] The weight calculation formula of the tth band is as follows:

[0041]

[0042] wherein Q t represents the sum of squares of differences between the true value and the predicted value of the sand information of the tth waveband sand remote sensing image, W t represents the weight value of the tth waveband, and m represents the number of wavebands, t = 1, 2, …, m;

[0043]

[0044] wherein, is the precision degree of the tth waveband optimal Y-Net network model for predicting the ith sand remote sensing image test set in the tth waveband, n represents the number of test sets, and i = 1, 2, …, n.

[0045] The precision degree of the tth waveband optimal Y-Net network model for predicting the ith sand remote sensing image test set in the tth waveband is a percentage value, i.e., indicates the credibility of the prediction, and is taken as the predicted value of the sand information of the tth waveband sand remote sensing image, while the true value is taken as 1; further, the sand remote sensing images of the corresponding wavebands are weighted and reconstructed in S102 to obtain a new sand remote sensing image after waveband fusion, and the specific process is as follows:

[0046] The waveband combination fusion calculation formula is as follows:

[0047] AN = A1xW1 + A2xW2 + … + A n xW n

[0048] wherein, AN represents the new sand remote sensing image after waveband fusion, A1, A2, …, A n represent the sand remote sensing images of the 1st, 2nd, …, nth wavebands, W1, W2, …, W n represent the weights of the 1st, 2nd, …, nth wavebands.

[0049] Further, the precision degree of each waveband sand remote sensing image for sand information extraction is the F1 value, which is the harmonic average evaluation index of Precision and Recall, and the formula is as follows:

[0050]

[0051] The F1 harmonic average evaluation is the reciprocal of the average value of the sum of reciprocals, the precision rate, and the recall rate, and the F1 value represents the precision degree of each waveband for sand extraction, and the greater the F1 value, the more obvious the influence of the waveband on the ground object;

[0052] Wherein, Precision represents precision, Recall represents recall, TP is the number of pixel points with pixel values of 255 in the detection image and the expert image, that is, the number of pixels correctly identified by the detection algorithm; FP is the number of pixel points with pixel value of 255 in the detection image but pixel value of 0 in the expert image, that is, the number of pixels misidentified by the detection algorithm; FN is the number of pixel points with pixel value of 0 in the detection image but pixel value of 255 in the expert image, that is, the number of pixel points of sand land missed by the detection algorithm; TN is the number of pixel points with pixel values of 0 in the detection image and the expert image, that is, the number of pixels correctly judged as background area by the detection algorithm.

[0053] Three performance indicators are used to evaluate the sand land extraction method in this paper, namely precision, recall and accuracy. F1 considers both accuracy and recall, and balances them to achieve the highest, that is, harmonic mean. F1 is irrelevant to precision (precision), and they are different evaluation indicators.

[0054] Further, the sand land information at least includes position, area and shape.

[0055] On the other hand, a band information considering multispectral remote sensing image sand land intelligent extraction system using the above band information considering multispectral remote sensing image sand land intelligent extraction method, comprising:

[0056] Y-Net network model construction module: adding deformable convolution in U-Net network model to obtain Y-Net network model;

[0057] Sand land remote sensing image band precision determination module: training, predicting and performance evaluating the Y-Net network model on each band sand land remote sensing image with known sand land information, to obtain the precision of each band sand land remote sensing image in extracting sand land information;

[0058] Image weighting reconstruction module: determining the weight value of each band sand land remote sensing image according to the precision of each band sand land remote sensing image in extracting sand land information, weighting and reconstructing the original multispectral remote sensing image covering sand land to produce a new sand land remote sensing image;

[0059] Sample data set construction module: expert labeling the sand land information in the weighted and reconstructed remote sensing image to generate sample labels and construct a band fusion multispectral sand land remote sensing image sample data set;

[0060] Training module: training the Y-Net network model on the band fusion multispectral remote sensing image data set to obtain the best parameters of the Y-Net network model of the combined band;

[0061] The prediction module: the Y-Net network model of the combined wave band with the optimal parameter is used to predict the sand land remote sensing image to be identified, so as to obtain the sand land information detection result of different years.

[0062] The sand land remote sensing image to be identified is obtained by weighting and reconstructing each wave band sand land image to be identified according to S103.

[0063] In another aspect, a readable storage medium stores a computer program, which is called by a processor to realize the steps of the above-mentioned sand land intelligent extraction method of a multi-spectral remote sensing image considering wave band information.

[0064] Advantages

[0065] According to the above technical solution, compared with the prior art, the present application provides a sand land intelligent extraction method and system of a multi-spectral remote sensing image considering wave band information, and the advantages include:

[0066] 1. The present application can increase the discrimination degree of sand land and other spectral similar objects by obtaining the original multi-spectral remote sensing image covering the sand land, reconstructing the wave band, and generating a new remote sensing image.

[0067] 2. The wave band weight value is determined according to the precision of sand land information extraction of a single wave band, a wave band combination multi-spectral remote sensing image data set is constructed, the accuracy of remote sensing image sand land extraction is improved, and high robustness extraction is realized.

[0068] 3. The Y-Net model is proposed by increasing the deformable convolution module in the U-Net model, and the Y-Net model can realize high-precision extraction of sand land in a wide range and complex background. BRIEF DESCRIPTION OF DRAWINGS

[0069] Figure 1 The network structure diagram of the sand land intelligent detection method of the wave band combination multi-spectral remote sensing image;

[0070] Figure 2 The flow chart of the sand land intelligent detection method of the wave band combination multi-spectral remote sensing image of the present application;

[0071] Figure 3 The sand land detection result diagram of the present application. DETAILED DESCRIPTION

[0072] With reference to the accompanying drawings of the embodiments of the present application, the present application further clearly illustrates a kind of sand intelligent extraction method of multispectral remote sensing image considering waveband information, and Landsat4-5 TM image covering Inner Mongolia Alashan League from April to October in 2000 is used to verify the present application.It should be noted that this is only used for example, the present application is not limited to data source.The technical solutions in the embodiments of the present application are clearly and completely described, apparently, the described embodiments are only a part of the embodiments of the present application, not all the embodiments.Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative efforts belong to the scope of the present application.

[0073] Referring to Figure 1 As shown in a kind of network structure diagram of sand intelligent extraction method of multispectral remote sensing image considering waveband information, it includes:U-Net model, deformable convolution module (D-Block).

[0074] Embodiment 1

[0075] Referring to Figure 2 As shown in the present application, an embodiment provides a kind of sand intelligent extraction method of multispectral remote sensing image considering waveband information, including the following steps:

[0076] S101: add deformable convolution in U-Net network model, obtain Y-Net network model;And the Y-Net network model is trained, predicted and performance evaluated on each waveband sand remote sensing image of known sand information, obtain the precision degree of each waveband sand remote sensing image to sand information extraction;

[0077] The sand information at least includes position, area, shape.

[0078] U-Net network model is composed of 8 3x3 convolution modules and 4 maximum pooling modules;

[0079] Specifically, by 8 deconvolution modules, target feature information is up-sampled, and the feature map containing low-order position information of target ground object extracted by feature extraction network is fused with the feature map preserving high-order semantic information of up-sampling part through 4 interlayer connections;

[0080] Y-Net network model is just deformed to U-Net network model, i.e.deformable convolution module is used instead of ordinary convolution module;

[0081] The Y-Net network model is composed of 8 3x3 deformable convolution modules and 4 maximum pooling modules;Wherein, deformable convolution module refers to an offset of sampling position in standard convolution operation, and the offset is obtained by additional convolution layer learning image features;

[0082] Specifically, the target feature information is up-sampled by 8 deformable convolution modules, and the feature map containing low-order position information of the target ground object extracted by the feature extraction network is fused with the feature map containing high-order semantic information reserved by the up-sampling part through 4 inter-layer connections, so as to extract and classify the target ground object of the preprocessed public data set in an end-to-end manner.

[0083] The offset added to the sampling position in deformable convolution enables the convolution kernel to expand to a large range during the training process, and is not limited to fixed grid sampling. The conventional convolution operation can mainly be divided into two parts: (1) sampling on the input feature map using a regular grid R; (2) performing a weighted operation, R defines the size and expansion of the receptive field, as follows

[0084] R={(-1,-1),(-1,0),...,(0,1),(0,-1)}

[0085] The offset is obtained by learning the image features through an additional convolution layer, and the offset sizes obtained by convolution learning are different for different image ground objects.

[0086] The general convolution formula is as follows, for each position p0 on the output feature map y:

[0087]

[0088] where p k is the enumerated sampling position in the grid k, x is the input feature image, W k is the corresponding convolution kernel weight coefficient, y(p0) represents the output feature image at p0, and p0 is the output feature image point.

[0089] The deformable convolution formula with the added offset is as follows, for each position p0 on the output feature map y:

[0090]

[0091] where p k is the enumerated sampling position in the grid k, Δp k represents the offset in the deformable convolution module, x is the input feature image, W k is the corresponding convolution kernel weight coefficient, y(p0) represents the output feature image at p0, and p0 is the output feature image point.

[0092] The deformable convolution module adds a two-dimensional offset to the regular grid sampling position in the standard convolution, and then fine tunes the offset and the sampling position by using two different learning rates, and then determines the adaptive deformation of the target object according to the grid deformation, so that it can better conform to the characteristics of the target object, wherein the two-dimensional offset is obtained by self-learning from the feature image by using the convolution layer.

[0093] In S101, the Y-Net network model is used to extract features of sand information in sand remote sensing images, and the specific process is as follows:

[0094] Firstly, the different convolutions of the feature extraction network in the Y-Net network model are used to extract preliminary feature images of the original sand image of a certain band at the sampling point position:

[0095]

[0096] Wherein, I is the input of the original sand remote sensing image of a certain band, M is the convolution kernel, p is the output feature image point, r is the width of the image, a and b are the offset of the input original sand remote sensing image I in the horizontal and vertical directions, x and y are the horizontal and vertical coordinate values of a certain point in I; X(p) represents the image value of the corresponding p in the output preliminary feature image of the point (x, y) in the original sand remote sensing image I of a certain band;

[0097] Then, the standard regular grid point k is used for feature sampling on the preliminary feature image, and the deformable convolution is used to obtain the high-order feature image y:

[0098]

[0099] Wherein, W k is the convolution kernel weight coefficient of the grid point k, the grid point k represents the sampling position, y(p) represents the output high-order feature image value of p point, p0 is the original sampling position, p k represents the position of the sampling point in the adaptive deformable convolution kernel, s k represents the learning rate of p k , Δp k represents the offset in the deformable convolution, Δm k represents the adjustable amount, c k represents the learning rate of the adjustable amount Δm k , s k ∈[0,1], Δm k ∈[0,1], c k ∈[0,1].

[0100] The process of determining the precision of sand information extraction of each band sand remote sensing image is as follows:

[0101] The Y-Net network model is used for training on each waveband sand land remote sensing image dataset, and the optimal model parameters of the Y-Net network model of each waveband are obtained.

[0102] Based on the optimal model parameters of the Y-Net network model of each waveband, the sand land remote sensing image data test set of each waveband is predicted to obtain the sand land information prediction result of each waveband.

[0103] The performance evaluation index is used to evaluate the sand land information extraction result of each waveband, and the precision degree of the sand land remote sensing image for sand land information extraction is obtained.

[0104] The precision degree refers to which waveband image is more accurate for extracting sand land information, and is selected by the Y-Net network model.

[0105] The precision degree is a comprehensive calculation value of several performance evaluation indexes.

[0106] In order to realize the objective evaluation of the precision degree of each waveband sand land extraction result, the performance evaluation index, that is, the harmonic mean of the precision and the recall, is used to obtain the evaluation index F1 value, and the sand land information extraction result is evaluated.

[0107] The original multi-spectral remote sensing image covering the sand land is obtained from the geographic spatial data cloud website.

[0108] S102: According to the precision degree of each waveband sand land remote sensing image for sand land information extraction, the weight value of each waveband sand land remote sensing image is determined, the original multi-spectral remote sensing image covering the sand land is weighted and reconstructed to generate a new sand land remote sensing image.

[0109] In S102, the precision degree of each waveband sand land remote sensing image for sand land information extraction is used to determine the waveband weight, the precision degree of each waveband sand land remote sensing image for sand land information extraction is used, and the variance reciprocal method is used to calculate the waveband weight.

[0110] The weight calculation formula of the tth waveband is as follows:

[0111]

[0112] Wherein, Q t The sum of squares of the difference between the sand land information true value and the predicted value of the tth waveband sand land remote sensing image is represented, W t The weight value of the tth waveband is represented, m represents the number of wavebands, and t=1, 2,..., m.

[0113]

[0114] In the formula, is the precision degree of the optimal Y-Net network model of the tth wave band to the ith sand remote sensing image test set of the current wave band, n represents the number of test sets of images, i = 1, 2, …, n.

[0115] The precision degree of the optimal Y-Net network model of the tth wave band to the ith sand remote sensing image test set of the tth wave band is a percentage value, that is, it represents the confidence degree of the prediction, and the prediction value of the sand information of the sand remote sensing image of the tth wave band, and the true value is considered as 1; further, the sand remote sensing image of the corresponding wave band is weighted and reconstructed in S102 to obtain a new sand remote sensing image after wave band fusion, and the specific process is as follows:

[0116] The wave band combination fusion calculation formula is as follows:

[0117] AN = A1 x W1 + A2 x W2 + … + An x Wn n n

[0118] In the formula, AN represents a new sand remote sensing image after wave band fusion, A1, A2 … An represent sand remote sensing images of the 1st, 2nd … nth wave band, n W1, W2 … Wn represent the weights of the 1st, 2nd … nth wave band. n

[0119] The precision degree of the sand remote sensing image of each wave band to the sand information extraction is the F1 value, and the F1 value refers to the harmonic average evaluation index of Precision and Recall, and the formula is as follows:

[0120]

[0121] The F1 harmonic average evaluation is the reciprocal of the average value of the sum of reciprocals, the precision rate and the recall rate, and the F1 value represents the precision degree of each wave band to the sand extraction, and the greater the F1 value, the more obvious the influence degree of the wave band to the ground object;

[0122] Among them, Precision represents the precision rate, Recall represents the recall rate, TP is the number of pixel points with pixel value of 255 in the detection image and the expert image, that is, the number of pixels correctly identified by the detection algorithm; FP is the number of pixel points with pixel value of 255 in the detection image but with pixel value of 0 in the expert image, that is, the number of pixels misidentified by the detection algorithm; FN is the number of pixel points with pixel value of 0 in the detection image but with pixel value of 255 in the expert image, that is, the number of pixel points of the sand land missed by the detection algorithm; TN is the number of pixel points with pixel value of 0 in the detection image and the expert image, that is, the number of pixels correctly judged as background area by the detection algorithm.​​

[0123] Three performance indicators are used to evaluate the sand extraction method in this paper, namely precision, recall, and accuracy. F1 takes into account both accuracy and recall, allowing both to reach the highest level, achieving a balance, i.e. harmonic mean. F1 is independent of precision (precision), and they are different evaluation indicators.

[0124] S103: Expert annotation of sand information in the weighted reconstructed remote sensing image to generate sample labels and construct a band fusion multispectral sand remote sensing image dataset;

[0125] S104: Training the Y-Net network model on the band fusion multispectral remote sensing image dataset to obtain the best parameters of the Y-Net network model for combined bands;

[0126] S105: Using the Y-Net network model with the best parameters for combined bands to predict the sand remote sensing image to be identified to obtain sand information detection results for different years;

[0127] The sand remote sensing image to be identified is obtained by weighting and reconstructing each band sand image to be identified according to S102.

[0128] To objectively evaluate the method of the present application for intelligent extraction of sand from multispectral remote sensing images considering band information, a mathematical model is constructed to evaluate the detection accuracy of the method. The detected sand image result graph (i.e. detection graph) and the expert fine-labeled sand image (i.e. expert graph) are binarized. The minimum circumscribed rectangle of the sand is 255, and the background is 0. The pixel-level comparison results of the two images are used as evaluation indicators. Three indicators, namely precision, recall, and F1 score, are used to evaluate the sand extraction model.

[0129] The effectiveness of the Y-Net model is verified by comparing the values of the three indicators for U-Net, unweighted U-Net (N-U-Net), Y-Net, and unweighted Y-Net (N-Y-Net), as shown in Table 1:

[0130] Table 1 Comparison of sand image extraction results

[0131]

[0132] The sand area result obtained by using the algorithm has optimal accuracy, recall rate and F1 value, and the corresponding accuracy is 96.1%, the recall rate is 94.1%, and the F1 index is 95.1%. Compared with the original N-U-Net model, the F1 value of the Y-Net model is increased by 7.8%, and the accuracy is higher than that of the expert annotation result in the prior art.

[0133] Embodiment 2

[0134] A band information considering multispectral remote sensing image sand intelligent extraction system using the above band information considering multispectral remote sensing image sand intelligent extraction method, comprising:

[0135] A Y-Net network model construction module: a deformable convolution is added to the U-Net network model to obtain a Y-Net network model;

[0136] A sand remote sensing image band precision determination module: the Y-Net network model is trained, predicted and performance evaluated on each band sand remote sensing image with known sand information to obtain the precision of each band sand remote sensing image in sand information extraction;

[0137] An image weighting reconstruction module: according to the precision of each band sand remote sensing image in sand information extraction, the weight value of each band sand remote sensing image is determined, the original multispectral remote sensing image covering the sand is weighted and reconstructed to generate a new sand remote sensing image;

[0138] A sample data set construction module: the sand information in the weighted and reconstructed remote sensing image is expertly annotated to generate a sample label and construct a band fusion multispectral sand remote sensing image sample data set;

[0139] A training module: the Y-Net network model is trained on the band fusion multispectral remote sensing image data set to obtain the best parameters of the Y-Net network model of the combined band;

[0140] A prediction module: the Y-Net network model of the combined band with the best parameters is used to predict the sand remote sensing image to be identified to obtain the sand information detection result of different years;

[0141] The sand remote sensing image to be identified is obtained by weighting and reconstructing each band sand image to be identified according to S103.

[0142] The implementation process of each specific module can refer to the content of the above method, which will not be described here. It should be understood that the division of the above functional modules is only a logical division, and there can be another division way in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. At the same time, the integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0143] Embodiment 3

[0144] A readable storage medium stores a computer program, the computer program is called by a processor to realize: the steps of the above-mentioned sand intelligent extraction method of multispectral remote sensing image considering wave band information; The steps are as follows:

[0145] S101: Add deformable convolution in the U-Net network model to obtain the Y-Net network model; and train, predict and performance evaluate the Y-Net network model on each wave band sand remote sensing image with known sand information, to obtain the precision degree of each wave band sand remote sensing image for sand information extraction;

[0146] S102: According to the precision degree of each wave band sand remote sensing image for sand information extraction, determine the weight value of each wave band sand remote sensing image, weight reconstruction of the original multispectral remote sensing image covering the sand, to produce a new sand remote sensing image;

[0147] S103: Expert annotation is performed on the sand information in the weighted reconstructed remote sensing image to generate sample labels and construct a wave band fusion multispectral sand remote sensing image dataset;

[0148] S104: Train the Y-Net network model on the wave band fusion multispectral remote sensing image dataset to obtain the best parameters of the Y-Net network model of the combined wave band;

[0149] S105: Use the Y-Net network model of the combined wave band with the best parameters to predict the sand remote sensing image to be identified to obtain the sand information detection result of different years;

[0150] The sand remote sensing image to be identified is obtained by weighting and reconstructing each wave band sand image to be identified according to S102.

[0151] The specific implementation process of each step can refer to the description of the foregoing method.

[0152] The readable storage medium is a computer readable storage medium, which can be an internal storage unit of the controller, such as a hard disk or a memory of the controller. The readable storage medium can also be an external storage device of the controller, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the readable storage medium can include both the internal storage unit and the external storage device of the controller. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium can also be used to temporarily store data that has been output or will be output.

[0153] Based on such understanding, the technical solutions of the present application, essentially or in other words, the part of the prior art that contributes to the present application, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in the embodiments of the present application. The aforementioned readable storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0154] It should be emphasized that the examples described in the present application are illustrative rather than limiting, and thus the present application is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art from the technical solutions of the present application without departing from the purpose and scope of the present application, whether modified or replaced, also belong to the protection scope of the present application.

Claims

1. A method for intelligent extraction of sandy land in multispectral remote sensing image considering waveband information, characterized in that, The method comprises the following steps: S101: adding a deformable convolution to a U-Net network model to obtain a Y-Net network model; and training, predicting and performance evaluating the Y-Net network model on each band of sand ground remote sensing images with known sand ground information to obtain the precision of sand ground information extraction of each band of sand ground remote sensing images; S102: determining the precision of sand ground information extraction of each band of sand ground remote sensing images according to the precision of sand ground information extraction of each band of sand ground remote sensing images, weighting and reconstructing the original multi-spectral remote sensing image covering the sand ground to generate a new sand ground remote sensing image; S103: expertly labeling the sand ground information in the weighted and reconstructed remote sensing image to generate a sample label and construct a band fusion multi-spectral sand ground remote sensing image dataset; S104: training the Y-Net network model on the band fusion multi-spectral remote sensing image dataset to obtain the best parameters of the Y-Net network model of the combined band; S105: predicting the sand ground remote sensing image to be identified by using the Y-Net network model of the combined band with the best parameters to obtain a sand ground information detection result; The sand ground remote sensing image to be identified is obtained by weighting and reconstructing each band of sand ground image to be identified according to S102; In S101, the Y-Net network model is used to extract features of sand ground information in a sand ground remote sensing image, and the specific process is as follows: First, the different convolutions of the feature extraction network in the Y-Net network model are used to preliminarily extract features of a certain band of sand ground original image at the sampling point position: Wherein, I is the input of a certain band of sand ground original remote sensing image, M is the convolution kernel, p is the output feature image point, r is the width of the image, a and b are the horizontal and vertical offsets of the input of a certain band of sand ground original remote sensing image I, x and y are the horizontal and vertical coordinate values of a certain point in I; X(p) represents the image value of point (x, y) in a certain band of sand ground original remote sensing image I in the output preliminary feature image p; Then, a standard regular grid point k is used for feature sampling on the preliminary feature image, and a deformable convolution is used to obtain a high-order feature image y: wherein W k is a convolution kernel weight coefficient of a grid point k, the grid point k represents a sampling position, y(p) represents a high-order feature image value output by a p point, p0 is an original sampling position, p k represents a position of a sampling point in an adaptive deformable convolution kernel, s k represents a learning rate of p k , Δp k represents a displacement in deformable convolution, Δm k represents an adjustable amount, c k represents a learning rate of the adjustable amount Δm k , s k ∈ [0, 1], Δm k ∈ [0, 1], c k ∈ [0, 1]. 2.The sand intelligent extraction method of multispectral remote sensing image considering waveband information according to claim 1, wherein, The Y-Net network model is composed of 8 3*3 deformable convolution modules and 4 maximum pooling modules; wherein, the deformable convolution module refers to an offset added to the sampling position in the standard convolution operation, and the offset is obtained by learning image features through an additional convolution layer; The deformable convolution formula with the added offset is as follows, for each position p0 on the output feature map y: wherein p k is the enumeration of sampling positions in the grid k, Δp k represents the offset in the variable convolution module, x is the input feature image, W k is the corresponding convolution kernel weight coefficient, y(p0) represents the output feature image of p0 point, and p0 is the output feature image point. 3.The sand intelligent extraction method of multispectral remote sensing image considering band information according to claim 1, wherein, The process of determining the precision of sand ground information extraction of each band of sand ground remote sensing image is as follows: Training the Y-Net network model on each band of sand ground remote sensing image dataset to obtain the optimal model parameters of each band of Y-Net network model; Based on the optimal model parameters of each band of Y-Net network model, predicting each band of sand ground remote sensing image data test set to obtain each band of sand ground information prediction result; Using the performance evaluation index to evaluate each band of sand ground information extraction result to obtain the precision of sand ground information extraction of each band of sand ground remote sensing image. 4.The sand intelligent extraction method of multispectral remote sensing image considering waveband information according to claim 1, characterized in that, The precision degree of sand information extraction of each band sand remote sensing image is used to determine the band weight in S102, which is the precision degree of sand information extraction of each band sand remote sensing image, and the band weight is calculated by using the inverse variance method; The weight calculation formula of the tth band is as follows: wherein Q t represents the sum of squares of differences between the real value and the predicted value of the sand information of the tth waveband sand remote sensing image, W t represents the weight value of the tth waveband, m represents the number of wavebands, and t = 1, 2,..., m; In the formula, is the precision of the optimal Y-Net network model of the tth wave band for predicting the ith sand remote sensing image test set of the current wave band, n represents the number of test set images, and i = 1, 2,..., n.

5. The sand intelligent extraction method of multispectral remote sensing image considering waveband information according to claim 1, characterized in that, In S102, the sand remote sensing image of the corresponding band is weighted and reconstructed to obtain a new sand remote sensing image after band fusion, and the specific process is as follows: The band combination fusion calculation formula is as follows: AN = A1 x W1 + A2 x W2 +... + A n x W n In the formula, AN represents a new sandy land remote sensing image after band fusion, A1, A2...A n represent the sandy land remote sensing images of the 1st, 2nd...nth bands, W1, W2...W n represent the 1st, 2nd...nth band weights.

6. The sand intelligent extraction method of multispectral remote sensing image considering waveband information according to any one of claims 1-5, characterized in that, The precision degree of sand information extraction of each band sand remote sensing image is F1 value, which is the harmonic mean evaluation index of Precision and Recall, and the formula is as follows: Wherein, Precision represents precision, Recall represents recall, TP is the number of pixel points with pixel value of 255 in the detection image and the expert image, that is, the number of pixels correctly identified by the detection algorithm; FP is the number of pixel points with pixel value of 255 in the detection image but pixel value of 0 in the expert image, that is, the number of pixels misidentified by the detection algorithm; FN is the number of pixel points with pixel value of 0 in the detection image but pixel value of 255 in the expert image, that is, the number of pixel points occupied by the missed sand land by the detection algorithm; TN is the number of pixel points with pixel value of 0 in the detection image and the expert image, that is, the number of pixels correctly judged as background area by the detection algorithm.

7. The sand intelligent extraction method of multispectral remote sensing image considering waveband information according to claim 1, characterized in that, The sand information at least includes position, area and shape.

8. A sand intelligent extraction system of multispectral remote sensing image considering waveband information, which adopts the method of any one of claims 1-7. It comprises: A Y-Net network model construction module: a deformable convolution is added to the U-Net network model to obtain a Y-Net network model; A sand remote sensing image band precision degree determination module: the Y-Net network model is trained, predicted and performance evaluated on each band sand remote sensing image with known sand information to obtain the precision degree of sand information extraction of each band sand remote sensing image; An image weighting reconstruction module: according to the precision degree of sand information extraction of each band sand remote sensing image, the weight value of each band sand remote sensing image is determined, and the original multi-spectral remote sensing image covering the sand land is weighted and reconstructed to generate a new sand remote sensing image; A sample data set construction module: the sand information in the weighted and reconstructed remote sensing image is expertly labeled to generate a sample label and construct a band fusion multi-spectral sand remote sensing image sample data set; A training module: the Y-Net network model is trained on the band fusion multi-spectral remote sensing image data set to obtain the best parameters of the Y-Net network model of the combined band; A prediction module: the Y-Net network model of the combined band with the best parameters is used to predict the sand remote sensing image to be identified to obtain the sand information detection result of different years; The sand remote sensing image to be identified is obtained by weighting and reconstructing each band sand image to be identified according to S103.

9. A readable storage medium characterized by: A computer program is stored, which is called by a processor to realize the steps of the method of any one of claims 1-7.

Images