Breeding area prediction method based on improved nested unet neural network

Abstract

The application discloses a breeding area prediction method based on an improved nested UNet neural network and belongs to the field of marine breeding area prediction methods. The method performs data preprocessing on high-resolution 3 radar image data and high-resolution 2 optical image data, performs band synthesis and image fusion on the optical image data and the radar image data, and constructs a data set. On the basis of applying a U2-Net neural network model main structure, the model network structure of an RSU module is modified, a channel attention mechanism based on maximum pooling and average pooling is added inside and outside a residual U-shaped module structure, a U2-Net_SEOUT model is constructed, and the prediction of the area and distribution of a breeding area is realized. The method comprehensively utilizes the advantages of optical images and SAR images, modifies the network structure, increases the number of network layers and the complexity, and improves the prediction precision of raft type breeding areas.

Description

TECHNICAL FIELD

[0001] The application discloses a breeding area prediction method based on an improved nested UNet neural network, and belongs to the field of breeding area prediction methods. BACKGROUND

[0002] The existing raft culture area prediction methods mainly include visual interpretation, characteristic index, deep learning and object-oriented method, and most of them are based on neural network extraction of a single data source, and the prediction accuracy of the breeding area is limited. And the raft culture facilities are often partially submerged in water, resulting in weak spectral information. The prediction accuracy of the traditional prediction method and the deep learning method for the breeding area has certain limitations. In recent years, with the rapid development of deep convolutional neural network (CNN), especially the rapid rise of full convolution network (FCN) in the field of area prediction, the level of significant target prediction has made great progress.

[0003] The deep learning method used in the application has strong data mining and feature extraction capability, which can autonomously learn image spectral information and spatial features through convolution layers, avoid complex spectral feature analysis of the image, reduce the workload, and realize large-scale automatic and accurate classification through image fusion of radar image and optical image data. It has advantages such as robustness, stability and invariance. SAR image is not affected by weather such as cloud and rain, and can be observed all day and all weather. The high-resolution 3 satellite image used in the application is one of the typical SAR satellite images. The application improves the U 2 -Net convolutional neural network by adding a channel attention mechanism, and comprehensively utilizes the advantages of optical images and SAR images to improve the accuracy and reliability of raft culture area prediction. SUMMARY

[0004] The application aims to provide a breeding area prediction method based on an improved nested UNet neural network, which increases the number of network layers and complexity of the UNet network, increases the scale of the feature map, and comprehensively utilizes the advantages of optical images and SAR images to improve the accuracy and reliability of raft culture area prediction.

[0005] The breeding area prediction method based on the improved nested UNet neural network comprises the following steps:

[0006] S1. Obtain high-resolution 3 image data of the breeding area through a high-resolution 3 satellite, and perform data preprocessing on the high-resolution 3 image data;

[0007] S2. Perform resampling and image registration processing on the preprocessed high-resolution 3 image data to obtain radar image data of the breeding area;

[0008] S3. Obtain Gaofen-2 image data of the aquaculture area through Gaofen-2 satellite, perform data preprocessing on the Gaofen-2 image data to obtain optical image data of the aquaculture area;

[0009] S4. By performing band synthesis and image fusion on radar image data and optical image data of the aquaculture area, a GF23B dataset is constructed.

[0010] S5. Based on the original UNet neural network model, the model complexity is increased by modifying the model network structure to obtain a nested UNet neural network;

[0011] S6. By adding a channel attention mechanism to the nested UNet neural network, a U... 2 -Net_Se 2 Neural network model;

[0012] S7. Based on the GF23B dataset, U 2 -Net_Se 2 Neural network models are trained and validated using U. 2 -Net_Se 2 Neural network models enable predictions about aquaculture areas.

[0013] In S1, data preprocessing of Gaofen-3 image data includes radiometric calibration, data conversion, multi-view processing, filtering, and geocoding.

[0014] In S3, the data preprocessing of Gaofen-2 image data includes radiometric calibration and atmospheric correction.

[0015] In S4, radar image data of the aquaculture area is combined with optical image data of the aquaculture area into a single band.

[0016] In S4, the NNDiffuse Pan Sharpening image fusion algorithm is used to perform pixel-level image fusion on radar image data and optical image data of the aquaculture area. The spatial and spectral features of the two types of image data are fused through image fusion to increase the texture features of the image data. The fused image data and the corresponding label data are uniformly cropped to a size of 512×512 to construct the GF23B dataset.

[0017] In S5, the nested UNet neural network is a two-layer nested U-shaped structure, including an encoder, a decoder and a saliency map fusion module, the encoder is composed of six roadside unit RSU modules of En_1 to En_6, the decoder is composed of five roadside unit RSU modules of De_1 to De_5, the saliency map fusion module is connected with the encoder and the decoder, and feature fusion is performed in a gradually connected manner, 3*3 convolution processing and up-sampling processing are performed on the En_6 module of the encoder and the De_5 to De_1 modules of the decoder, the output image is processed to be consistent with the size of the input image, and then a 1*1 convolution processing is performed to generate a saliency probability mapping graph by using an activation function Sigmoid function.

[0018] The RSU-L module is a highly determined RSU module, L is the height, which is determined by the spatial resolution of the input feature, the height of the RSU module of the En_5 module is 7, and the height of the RSU module of the En_6 module is 4, the RSU-7 module of the En_5 module and the RSU-4 module of the En_6 module are replaced into RSU-4F modules through the pooling operation and the up-sampling operation, F represents expansion, and the resolution of the intermediate feature map is processed to be the same as that of the input feature map through the RSU-4F module.

[0019] In S6, a channel attention module is added inside and outside the U-shaped structure of the nested UNet neural network to construct a U 2 -Net_Se 2 model.

[0020] The channel attention module is an SE module based on global maximum pooling and average pooling, which is located inside and outside the structure of the En_1 to En_4 modules in the encoder part of the U-shaped structure of the nested UNet neural network.

[0021] In S7, the U 2 -Net_Se 2 neural network model is trained based on the training set of the GF23B data set, the spatial features and spectral features of the image data in the GF23B data set are learned by the model, the area and distribution of the breeding area are predicted, the model is verified based on the verification set of the GF23B data set, and the prediction accuracy of the model is tested.

[0022] Compared with the prior art, the present application has the following beneficial effects:

[0023] The breeding area prediction method of the present application is based on the backbone network structure of the U 2 -Net neural network model, the En_5-En_6 module of the encoder is replaced into the RSU-4F module of the extended convolution, the channel attention module based on the maximum pooling and the average pooling is added in the residual U-shaped module, and the U2 -Net_Se 2 The model increases the number of network layers and the complexity of the neural network, and improves the prediction accuracy of the breeding area. In addition, the application performs band synthesis and image fusion on the GF2 optical image and the GF3 SAR image to construct the GF23B dataset, which can comprehensively utilize the advantages of optical images and SAR images, and solve the problem that the spectral information of raft type breeding facilities is weak due to local flooding, affecting the prediction accuracy of the breeding area.

[0024] U 2 The U-shaped structure inside and outside the -Net neural network is added with a channel attention module to form a U 2 The -Net_Se2 model has excellent results in extracting the raft type breeding area, and the overall accuracy of the typical area extraction reaches 98.05%, and the Kappa coefficient is 0.9589, which is improved by 4.32% and 0.095 respectively compared with the original neural network. The extraction accuracy of the weak signal area is more than 96.50%, which is improved by more than 8.5% compared with the original neural network. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 The flowchart of the breeding area prediction method based on the improved nested UNet neural network provided by the embodiment of the application is shown in the figure;

[0026] Figure 2 The network structure diagram of the UNet neural network provided by the embodiment of the application is shown in the figure;

[0027] Figure 3 The network structure diagram of the UNet neural network provided by the embodiment of the application is shown in the figure;

[0028] Figure 4 The structure diagram of the RSU module with a height L of 7 provided by the embodiment of the application is shown in the figure;

[0029] Figure 5 The channel attention module diagram based on maximum pooling and average pooling provided by the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical scheme and advantages of the application clearer, the technical scheme in the application will be described clearly and completely below. Obviously, the described embodiments are part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor belong to the protection scope of the application.

[0031] The breeding area prediction method based on the improved nested UNet neural network comprises the following steps:

[0032] S1. Obtain high-resolution 3 satellite image data of the aquaculture area through the high-resolution 3 satellite, and perform data preprocessing on the high-resolution 3 satellite image data;

[0033] S2. Obtain radar image data of the aquaculture area by resampling and image registration processing on the preprocessed high-resolution 3 satellite image data;

[0034] S3. Obtain high-resolution 2 satellite image data of the aquaculture area through the high-resolution 2 satellite, and perform data preprocessing on the high-resolution 2 satellite image data to obtain optical image data of the aquaculture area;

[0035] S4. Perform band synthesis and image fusion on the radar image data of the aquaculture area and the optical image data of the aquaculture area to construct a GF23B data set;

[0036] S5. Based on the original UNet neural network model, the model network structure is modified to improve the model complexity, and a nested UNet neural network is obtained;

[0037] S6. A U 2 -Net_Se 2 neural network model is constructed by adding a channel attention mechanism in the nested UNet neural network;

[0038] S7. Model training and verification are performed on the U 2 -Net_Se 2 neural network model based on the GF23B data set, and the U 2 -Net_Se 2 neural network model is used to predict the aquaculture area.

[0039] In S1, the data preprocessing of the high-resolution 3 satellite image data includes radiation calibration, data conversion, multi-view processing, filtering processing and geographic coding.

[0040] In S3, the data preprocessing of the high-resolution 2 satellite image data includes radiation calibration and atmospheric correction.

[0041] In S4, the radar image data of the aquaculture area is taken as a band for band synthesis with the optical image data of the aquaculture area.

[0042] In S4, the image fusion algorithm NNDiffuse Pan Sharpening is used to perform image fusion at the pixel level on the radar image data of the aquaculture area and the optical image data of the aquaculture area. The spatial features and spectral features of the two image data are fused through image fusion to increase the texture features of the image data. The fused image data and the label data corresponding to the image data are uniformly cropped to 512x512 to construct the GF23B data set.

[0043] In S5, the nested UNet neural network is a two-layer nested U-shaped structure, including an encoder, a decoder and a saliency map fusion module. The encoder is composed of six roadside unit RSU modules En_1 to En_6, the decoder is composed of five roadside unit RSU modules De_1 to De_5, and the saliency map fusion module is connected with the encoder and the decoder. The saliency map fusion module performs feature fusion in a gradually connected manner, performs 3×3 convolution processing and up-sampling processing on the En_6 module of the encoder and the De_5 to De_1 modules of the decoder, processes the output image to be consistent with the size of the input image, and then generates a saliency probability map using the activation function Sigmoid function after 1×1 convolution processing.

[0044] The RSU-L module is a highly determined RSU module, and L is the height determined by the spatial resolution of the input feature. The height of the RSU module of the En_5 module is 7, and the height of the RSU module of the En_6 module is 4. The RSU-7 module of the En_5 module and the RSU-4 module of the En_6 module are replaced by the RSU-4F module through the pooling operation and the up-sampling operation. F represents expansion, and the resolution of the intermediate feature map is processed to be the same as the resolution of the input feature map through the RSU-4F module.

[0045] In S6, the channel attention module is an SE module based on global maximum pooling and average pooling, which is located at the En_1 to En_4 module position of the encoder part of the U-shaped structure of the nested UNet neural network. By adding the channel attention module inside and outside the U-shaped structure of the nested UNet neural network, the U 2 -Net_Se 2 model; by adding the channel attention module inside the structure of the En_1 to En_4 module, the U 2 -Net_seIN model; by adding the channel attention module outside the structure of the En_1 to En_4 module, the U 2 -Net_SEOUT model. Based on the GF23B dataset, the model accuracy and reliability of the U 2 -Net_seIN model, the U 2 -Net_SEOUT model and the U 2 -Net_Se 2 model are compared and analyzed to determine the optimal network model and predict the breeding area. The U 2 -Net_Se 2 , the U 2 -Net_SEOUT, the U 2 -Net_seIN three improved models have better prediction effect on the typical area and the weak signal area than the original U 2The overall accuracy of typical area prediction of the improved Net model is 4.32%, 2.19% and 2.87% higher than that of the original model respectively, and the Kappa coefficient is 0.0946, 0.0482 and 0.0631 higher than that of the original model respectively, and the overall accuracy of weak signal area extraction is 8.72%, 0.84% and 3.16% higher than that of the original model respectively, and the Kappa coefficient is 0.1758, 0.0171 and 0.0639 higher than that of the original model respectively, which proves the effectiveness of the improved model.

[0046] In S7, the training set based on the GF23B data set is used to train the U 2 -Net_Se 2 The neural network model is trained, the spatial features and spectral features of the image data in the GF23B data set are learned by the model, the area and distribution of the breeding area are predicted, and the model is verified based on the verification set of the GF23B data set to test the prediction accuracy of the model. 2 -Net_Se 2 The neural network model has excellent results in predicting raft culture area, and the overall accuracy of predicting typical area reaches 98.05%, and the Kappa coefficient is 0.9589, which is 4.32% and 0.095 higher than that of the original neural network respectively. For weak signal area, the prediction accuracy is more than 96.50%, which is more than 8.5% higher than that of the original neural network.

[0047] The embodiment of the application.

[0048] Figure 1 The flowchart of the breeding area prediction method based on the improved nested UNet neural network provided by the embodiment of the application is provided, first, the high-resolution 3 image data is acquired, the high-resolution 3 image data is preprocessed, including radiation calibration, data conversion, multi-view processing, filtering processing and geographic coding, then the processed high-resolution 3 image is resampled and image registered, and the radar image data of the breeding area is acquired. The high-resolution 2 image data is acquired, the high-resolution 2 image data is radiometrically calibrated and atmospherically corrected to obtain optical image data of the breeding area, the preprocessed high-resolution 2 image and the image registered high-resolution 3 image are band synthesized and image fused, and the GF23B data set is constructed based on the optical image data and the fused image data. The U 2 -Net neural network model with added channel attention mechanism is constructed, including U 2 -Net_seIN model, U 2 -Net_SEOUT model and U 2 -Net_Se 2 The model is trained based on the GF23B data set, and the U 2 -Net_seIN model, U 2 -Net_SEOUT model and U2 -Net_Se 2 The optimal network model is determined by the model, and the U 2 -Net_Se 2 The model is used for predicting the breeding area and analyzing the prediction result.

[0049] Figure 2 The network structure diagram of the nested UNet neural network provided by the embodiment of the application is in U-shaped structure, including an encoder, a decoder and a saliency map fusion module, the encoder is composed of six RSU modules En_1 to En_6, the decoder is composed of five RSU modules De_1 to De_5, and the saliency map fusion module includes a saliency map fusion input module and a saliency map fusion output module, wherein the saliency map fusion input module is connected with the encoder, and the saliency map fusion output module is connected with the decoder. The saliency map fusion module adopts a gradually connected manner for feature fusion, first collects the outputs of De_1, De_2, De_3, De_4, De_5 and En_6, and then obtains feature maps with a layer number of 1 through a convolution layer with a convolution kernel number of 1 and a convolution kernel size of 3x 3. The size of the feature map is restored to the size of the input picture through bilinear interpolation scaling to obtain Sup1, Sup2, Sup3, Sup4, Sup5 and Sup6; then the six feature maps are spliced to obtain a feature map with channel = 6; finally, a 1x 1 convolution layer and a Sigmiod activation function are used to obtain the final prediction probability map. Wherein Sup(n) represents the nth feature map after upsampling operation, Sup(0) represents the feature map output from the RSU module but not yet subjected to upsampling operation, S (n) side represents the nth segmentation in the side direction. From Figure 2 It can be seen that each RSU module in the nested UNet neural network is also in U-shaped structure, so the nested UNet neural network is composed of two layers of nested U-shaped structures. The RSU module is in symmetrical structure, and the number of layers on one side is the module height L, wherein the green layer includes a convolution layer, a normalization layer and a ReLU activation function, the blue layer is a down-sampling layer including a convolution layer, a normalization layer and a ReLU activation function, the purple layer is an up-sampling layer including a convolution layer, a normalization layer and an activation function, and the gray layer is a green layer with a determined expansion coefficient.

[0050] The channel attention module SE module based on global maximum pooling and average pooling is added to the structure inside and outside the En_1 to En_4 modules of the encoder part of the U-shaped structure of the nested UNet neural network to construct a U 2 -Net_Se 2 model, and the network structure is as followsFigure 3 As shown, the SE module is located in the lower part of the En_1 to En_4 module. Taking the En_1 module as an example, the internal structure of the RSU module after adding the channel attention module when the height L is 7 is as shown in the figure Figure 4 As shown, the SE module is located in the lower part of the En_1 to En_4 module. Taking the En_1 module as an example, the internal structure of the RSU module after adding the channel attention module when the height L is 7 is as shown in the figure Figure 5 As shown, x represents the original image data, h represents the result after the "compression" operation of the feature map, w represents the learned weight, and γ is a scaling parameter of the batch normalization layer, which is used to adjust the normalized feature value. Given an input image X with a feature channel number of C1, a feature map with a feature channel number of C is obtained through convolution processing, and then dimension compression is performed. The upper and lower branches respectively obtain the pooling results with a size of 1x1xC through global average pooling and global maximum pooling. The results of the upper and lower branches are processed through a fully connected layer, and a ReLU function is used as an activation function, and the size of the results of the upper and lower branches is 1x1xC / γ. The role of γ is to control the attention intensity and parameter amount. After the upper and lower branches are processed through the fully connected layer, the channel-by-channel fusion is performed, and the Sigmoid activation function is used to compress the channel-by-channel fusion result to [0.1] to obtain the final channel weight value. Finally, the input feature map channels are multiplied by the channel weight value to obtain a feature map with channel weight information Adding a second global maximum pooling branch to the original SE module can obtain more rich global information, try to avoid the loss of too much information, make the channel weight information more convincing, and use the two characteristics to greatly improve the representation ability of the network.

[0051] The above-mentioned global semantic information and greater are only preferred embodiments of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A method for predicting a farming area based on an improved nested UNet neural network, characterized by, The method comprises the following steps: S1. Obtain high-resolution 3 satellite image data of the aquaculture area through high-resolution 3 satellite, and perform data preprocessing on the high-resolution 3 satellite image data; S2. Obtain radar image data of the aquaculture area by resampling and image registration processing on the preprocessed high-resolution 3 satellite image data; S3. Obtain high-resolution 2 satellite image data of the aquaculture area through high-resolution 2 satellite, and perform data preprocessing on the high-resolution 2 satellite image data to obtain optical image data of the aquaculture area; S4. Perform band synthesis and image fusion on the radar image data of the aquaculture area and the optical image data of the aquaculture area to construct a GF23B data set; S5. Based on the original UNet neural network model, the model network structure is modified to improve the model complexity, and a nested UNet neural network is obtained; S6. A U²-Net_Se² neural network model is constructed by adding a channel attention mechanism in the nested UNet neural network; S7. Model training and verification are performed on the U²-Net_Se² neural network model based on the GF23B data set, and the aquaculture area is predicted through the U²-Net_Se² neural network model; In S4, the radar image data of the aquaculture area is synthesized as a band with the optical image data of the aquaculture area; pixel-level image fusion is performed on the radar image data of the aquaculture area and the optical image data of the aquaculture area by using the image fusion algorithm NNDiffuse Pan Sharpening, the spatial features and spectral features of the two kinds of image data are fused through image fusion, the texture features of the image data are increased, the fused image data and the label data corresponding to the image data are uniformly cropped to 512x512, and the GF23B data set is constructed; In S5, the nested UNet neural network is a two-layer nested U-shaped structure, which comprises an encoder, a decoder and a saliency map fusion module, the encoder is composed of six road side units RSU modules En_1 to En_6, the decoder is composed of five road side units RSU modules De_1 to De_5, the saliency map fusion module is connected with the encoder and the decoder, and the feature fusion is performed in a gradually connected manner, 3x3 convolution processing and up-sampling processing are performed on the En_6 module of the encoder and the De_5 to De_1 modules of the decoder, the output image is processed to be consistent with the size of the input image, and then the saliency probability mapping graph is generated by using the activation function Sigmoid function after 1x1 convolution processing; the RSU-L module is a high-determined RSU module, L is the height which is determined by the spatial resolution of the input feature, the height of the RSU module of the En_5 module is 7, and the height of the RSU module of the En_6 module is 4, the RSU-7 module of the En_5 module and the RSU-4 module of the En_6 module are replaced into RSU-4F modules through pooling operation and up-sampling operation, F represents expansion, and the resolution of the intermediate feature map is processed to be the same as the resolution of the input feature map through the RSU-4F module. In S6, the U²-Net_Se² model is constructed by adding the channel attention module inside and outside the U-shaped structure of the nested UNet neural network.

2. The improved nested UNet neural network based aquaculture area prediction method according to claim 1, characterized in that, In S1, the data preprocessing of the GF-3 image data includes radiation calibration, data conversion, multi-view processing, filtering processing and geographic coding.

3. The improved nested UNet neural network based aquaculture area prediction method according to claim 2, characterized in that, In S3, the data preprocessing of the GF-2 image data includes radiation calibration and atmospheric correction.

4. The improved nested UNet neural network based aquaculture area prediction method according to claim 3, characterized in that, The channel attention module is an SE module based on global maximum pooling and average pooling, which is located inside and outside the structure of the En_1 to En_4 modules in the encoder part of the nested UNet neural network.

5. The improved nested UNet neural network based aquaculture area prediction method according to claim 4, characterized in that, In S7, the U²-Net_Se² neural network model is trained based on the training set of the GF23B dataset, the spatial and spectral features of the image data in the GF23B dataset are learned through model self-learning, the area and distribution of the breeding area are predicted, and the model verification is performed based on the validation set of the GF23B dataset to test the prediction accuracy of the model.

Images