A full-floating raft type culture area extraction method based on polarimetric SAR images

Abstract

The application discloses a kind of full floating raft type aquaculture area extraction methods based on polarized SAR image, to each sample polarized SAR image is respectively preprocessed, polarized decomposition, and band combination, obtain the multidimensional channel feature image corresponding to each sample polarized SAR image respectively, then apply slice processing, with multidimensional channel feature slice image as input, with full floating raft type aquaculture local area in multidimensional channel feature slice image as output, for the constructed feature network model is trained, obtain full floating raft type aquaculture area extraction model;Further, for the polarized SAR image to be analyzed, apply full floating raft type aquaculture area extraction model, obtain corresponding full floating raft type aquaculture area;Design method makes full use of the polarization characteristics of full floating raft type aquaculture in polarized SAR image, and considering the influence of different polarization characteristics on full floating raft type aquaculture area extraction, feature optimization is carried out on different polarization characteristics, and a feature set suitable for full floating raft type aquaculture area extraction is constructed, and the advantages of deep learning are combined to realize the full floating raft type aquaculture area extraction task, and reduce its influence by speckle noise and background environment.

Description

TECHNICAL FIELD

[0001] The application relates to a full-floating raft type culture area extraction method based on polarimetric SAR images and belongs to the technical field of image recognition. BACKGROUND

[0002] Raft type culture is a culture mode in which seaweed (such as kelp and laver) and shellfish animals (such as oysters and mussels) are hung on a floating raft, which is one of the main culture modes of mariculture and has brought huge economic and social benefits to coastal areas. Timely and accurate grasp of raft type culture information is of great significance for rational planning of culture sea area and scientific management of coastal zone environment. The complex marine environment restricts the development of manual field investigation, and remote sensing technology has become an important means for monitoring and evaluating offshore raft type culture due to its wide coverage, strong timeliness, low cost and small risk. Related scholars have carried out research on offshore raft type culture area extraction using different types of satellite remote sensing images, including optical images such as Landsat, SPOT, Sentinel-2, ZY-3, GF-1 / 2 / 5 / 6, Worldview, and SAR images such as Sentinel-1 and GF-3. However, raft type culture includes various types such as pole type, semi-floating and full-floating raft type culture, among which full-floating raft type culture is a culture mode composed of raft frames and buoys, with net curtains hung horizontally on the raft frames, relying on the buoyancy of the buoys to float on the sea surface at all times, and is usually selected in sea areas above the lowest tide level of 5 m. Due to the short wavelength and poor penetration ability of electromagnetic waves, optical remote sensing is not sensitive to roughness, so full-floating raft type culture is not easy to be identified in optical remote sensing images.

[0003] Unlike this, synthetic aperture radar (SAR) has high sensitivity to surface roughness. Since the culture objects under the water surface of full-floating raft type culture affect the flow of seawater, the roughness of the sea surface changes, thereby causing changes in radar backscatter, so that the full-floating raft type culture area can be observed in SAR images. Existing offshore raft type culture area extraction methods based on SAR images mainly include pixel-level classification, object-oriented classification and deep learning methods. However, these methods are not designed for the image characteristics of full-floating raft type culture area, and have problems such as being seriously affected by speckle noise and low extraction accuracy. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a full-floating raft type culture area extraction method based on polarimetric SAR images, which fully utilizes the polarization characteristics of polarimetric SAR images and combines the advantages of deep learning to improve the accuracy of full-floating raft type culture area extraction.

[0005] The present application adopts the following technical solution to solve the above technical problems: the present application designs a full-floating raft type aquaculture area extraction method based on polarimetric SAR images, executes steps A to E as follows to obtain a full-floating raft type aquaculture area extraction model, and then executes steps i to iii as follows to realize full-floating raft type aquaculture area extraction for a polarimetric SAR image to be analyzed;

[0006] Step A. For a preset number of sample polarimetric SAR images each containing a known full-floating raft type aquaculture area, pre-processing is performed, and a sample image set is constructed, and then step B is entered;

[0007] Step B. For each sample polarimetric SAR image in the sample image set, polarization decomposition is performed to obtain each target polarization feature image corresponding to the sample polarimetric SAR image, and a multi-dimensional channel feature image is constructed through band combination, i.e., a multi-dimensional channel feature image corresponding to each sample polarimetric SAR image is obtained, and then step C is entered;

[0008] Step C. A feature network model is constructed based on the Unet model, and then step D is entered;

[0009] Step D. For each multi-dimensional channel feature image, slicing is performed on the multi-dimensional channel feature image according to a preset pixel size and a preset slice overlap rate to obtain each multi-dimensional channel feature slice image, and a full-floating raft type aquaculture local area in each multi-dimensional channel feature slice image is determined according to the known full-floating raft type aquaculture area in the sample polarimetric SAR image, and then step E is entered;

[0010] Step E. Based on each multi-dimensional channel feature slice image corresponding to all multi-dimensional channel feature images, the multi-dimensional channel feature slice image is input, and the full-floating raft type aquaculture local area in the multi-dimensional channel feature slice image is output, the feature network model is trained, and a full-floating raft type aquaculture area extraction model is obtained;

[0011] Step i. According to the methods of steps A and B, pre-processing and polarization decomposition are performed on the polarimetric SAR image to be analyzed to obtain a multi-dimensional channel feature image to be analyzed corresponding to the polarimetric SAR image to be analyzed, and slicing is performed on the multi-dimensional channel feature image to be analyzed according to the method of step D to obtain each multi-dimensional channel feature slice image to be analyzed, and then step ii is entered;

[0012] Step ii. For each multi-dimensional channel feature slice image to be analyzed, the full-floating raft type aquaculture area extraction model is applied for processing to obtain a full-floating raft type aquaculture local area in the multi-dimensional channel feature slice image to be analyzed, and then step iii is entered;

[0013] Step iii. As the inverse operation of the slicing in step i, the multi-dimensional channel feature slice images of each to-be-analyzed are combined to obtain the combination of each local area of the full floating raft culture in the to-be-analyzed multi-dimensional channel feature image, that is, the full floating raft culture area in the to-be-analyzed polarimetric SAR image corresponding to the to-be-analyzed multi-dimensional channel feature image.

[0014] As a preferred technical solution of the present application: in step A, for each sample polarimetric SAR image in the sample image set, firstly, the scattering matrix S of the sample polarimetric SAR image is extracted, the scattering matrix S is subtracted from its conjugate transpose S H matrix multiplication is performed to obtain the coherence matrix T3, and then the covariance matrix C3 is obtained by subtracting the diagonal matrix of the coherence matrix T3; then, according to the coherence matrix T3 and the covariance matrix C3, geographic encoding, radiation scaling, and filtering by the Refined Lee filtering method are sequentially performed to update each sample polarimetric SAR image, and further update the sample image set.

[0015] As a preferred technical solution of the present application: in step B, for each sample polarimetric SAR image in the sample image set, the following steps B1-1 to B1-3 are performed to obtain the multi-dimensional channel feature image corresponding to each sample polarimetric SAR image;

[0016] Step B1-1. According to the coherence matrix T3 and the covariance matrix C3 corresponding to each sample polarimetric SAR image obtained in step A, pre-set coherent and incoherent polarization decompositions are performed on the sample polarimetric SAR image to obtain each corresponding polarization feature image, and then step B1-2 is entered;

[0017] Step B1-2. For each polarization feature image, the JM distance value between the background region and the full floating raft culture area in the polarization feature image is calculated, and the highest separability between the background region and the full floating raft culture area is taken as the criterion to obtain each target polarization feature image, and then step B1-3 is entered;

[0018] Step B1-3. For each target polarization feature image corresponding to the sample polarimetric SAR image, a multi-dimensional channel feature image is constructed by band combination.

[0019] As a preferred technical solution of the present application: in step B1-2, for each polarization feature image, the following formula is used:

[0020] J = 2 (1 - e -B ) # (1)

[0021]

[0022] Calculate the JM distance value J between the background region in the polarized feature image and the full floating raft culture area, wherein m1 and m2 represent the polarized feature mean value of the background region in the polarized feature image and the polarized feature mean value of the full floating raft culture area respectively, and σ1 and σ2 represent the polarized feature standard deviation of the background region in the polarized feature image and the polarized feature standard deviation of the full floating raft culture area respectively.

[0023] As a preferred technical solution of the present application: in step B, for each sample polarimetric SAR image in the sample image set, the following steps B2-1 to B2-2 are performed to obtain a multi-dimensional channel feature image corresponding to each sample polarimetric SAR image respectively.

[0024] Step B2-1. According to the coherence matrix T3 and the covariance matrix C3 corresponding to each sample polarimetric SAR image obtained in step A, perform a preset coherent polarization decomposition and an incoherent polarization decomposition on the sample polarimetric SAR image to obtain a Neu_mod polarization feature image under Neumann polarization decomposition, a Y4_Dbl polarization feature image under Yamaguchi polarization decomposition, a H polarization feature image under polarization decomposition, and a VV polarization feature image under S2 matrix feature polarization decomposition, which constitute each target polarization feature image, and then enter step B2-2.

[0025] Step B2-2. For each target polarization feature image corresponding to the sample polarimetric SAR image, a multi-dimensional channel feature image is constructed by band combination.

[0026] As a preferred technical solution of the present application: in step C, the feature network model constructed from the input end to the output end includes an encoding network, an empty space pyramid pooling module ASPP, a decoding network, and an output module connected in series; wherein the encoding network from the input end to the output end includes a pre-convolution normalization activation module and three groups of encoding feature modules connected in series, the input end of the pre-convolution normalization activation module constitutes the input end of the encoding network, and the output end of the last group of encoding feature modules constitutes the output end of the encoding network; the decoding network from the input end to the output end includes three groups of decoding feature modules and a middle convolution normalization activation module connected in series, the input end of the first group of decoding feature modules constitutes the input end of the decoding network, and the output end of the middle convolution normalization activation module constitutes the output end of the decoding network; the output module from the input end to the output end includes a squeeze and excitation module SE and a post-convolution normalization activation module connected in series, the input end of the squeeze and excitation module SE constitutes the input end of the output module, and the output end of the post-convolution normalization activation module constitutes the output end of the output module.

[0027] ​The structures of the encoding feature modules are the same as each other, each encoding feature module comprises, in sequence from an input end to an output end, a max-pooling layer, a first residual convolutional layer, an input end of the max-pooling layer constitutes an input end of the encoding feature module, and an output end of the first residual convolutional layer constitutes an output end of the encoding feature module; the structures of the decoding feature modules are the same as each other, each decoding feature module comprises, in sequence from an input end to an output end, an up-sampling layer, a main splicing layer, and a sub-convolutional normalization activation module, an input end of the up-sampling layer constitutes an input end of the decoding feature module, and an output end of the sub-convolutional normalization activation module constitutes an output end of the decoding feature module; the structures of the front convolutional normalization activation module, the middle convolutional normalization activation module, the rear convolutional normalization activation module, and the sub-convolutional normalization activation module in each decoding feature module are the same as each other, each convolutional normalization activation module comprises, in sequence from an input end to an output end, a convolutional layer, a normalization layer, and a ReLU activation layer, an input end of the convolutional layer constitutes an input end of the convolutional normalization activation module, and an output end of the ReLU activation layer constitutes an output end of the convolutional normalization activation module;

[0028] An output end of the front convolutional normalization activation module is connected to input ends of the main splicing layers in the third group of decoding feature modules in sequence; output ends of the first group of encoding feature modules are connected to input ends of the main splicing layers in the second group of decoding feature modules in sequence; and output ends of the second group of encoding feature modules are connected to input ends of the main splicing layers in the first group of decoding feature modules in sequence.

[0029] As a preferred technical scheme of the present application: it further comprises a detail perception branch, the detail perception branch comprises four second residual convolutional layers, three attention upsampling modules, three secondary splicing layers, and a spatial attention mechanism module; the decoding network further comprises a terminal splicing layer, and the output ends of the sequentially third group of decoding feature modules in the decoding network are connected to the middle convolutional normalization activation module through the terminal splicing layer; the output end of the front convolutional normalization activation module in the encoding network, the output end of the sequentially first encoding feature module, the output end of the second encoding feature module, and the output end of the third encoding feature module are respectively and one by one connected to the input ends of the second residual convolutional layers, the output end of the second residual convolutional layer connected to the front convolutional normalization activation module is connected to the input end of the spatial attention mechanism module, the output ends of the second residual convolutional layers connected to the encoding feature modules are respectively and one by one connected to the input ends of the attention upsampling modules, the output end of the attention upsampling module connected to the sequentially third encoding feature module is connected to the input end of the first secondary splicing layer, the output end of the first secondary splicing layer is connected to the input end of the second secondary splicing layer connected to the output end of the sequentially first encoding feature module, the output end of the second secondary splicing layer is connected to the input end of the third secondary splicing layer connected to the output end of the spatial attention mechanism module, and the output end of the third secondary splicing layer is connected to the input end of the terminal splicing layer in the decoding network; the structures of the attention upsampling modules are the same as each other, and each attention upsampling module comprises, from the input end to the output end, a spatial attention mechanism module and an upsampling module connected in sequence, wherein the input end of the spatial attention mechanism module constitutes the input end of the attention upsampling module, and the output end of the upsampling module constitutes the output end of the attention upsampling module.

[0030] As a preferred technical scheme of the present application: the structures of the spatial attention mechanism modules are the same as each other, and each spatial attention mechanism module comprises, from the input end to the output end, a convolutional layer, a normalization layer, a ReLu activation layer, a convolutional layer, a normalization layer, and a Sigmoid activation layer connected in sequence, wherein the input end of the first convolutional layer constitutes the input end of the spatial attention mechanism module, and the output end of the Sigmoid activation layer constitutes the output end of the spatial attention mechanism module.

[0031] As a preferred technical scheme of the present application: the empty space pyramid pooling module ASPP includes a pooling layer, a connection layer, an up-sampling layer, and three 1x1 convolution layers and three 3x3 convolution layers, the input end of the pooling layer, the input end of the three 3x3 convolution layers, and the input end of the first 1x1 convolution layer are connected to form the input end of the empty space pyramid pooling module ASPP, the output end of the pooling layer is connected to the input end of the second 1x1 convolution layer, the output end of the second 1x1 convolution layer is connected to the input end of the up-sampling layer, the output end of the first 1x1 convolution layer, the output end of the three 3x3 convolution layers, and the output end of the up-sampling layer are connected to the input end of the connection layer, the output end of the connection layer is connected to the input end of the third 1x1 convolution layer, and the output end of the third 1x1 convolution layer forms the output end of the empty space pyramid pooling module ASPP.

[0032] The structure of each first residual convolution layer and the structure of each second residual convolution layer are the same as each other, and each residual convolution layer sequentially connects a convolution layer, a normalization layer, a ReLu activation layer, a convolution layer, a normalization layer, a connection layer, and a ReLu activation layer from the input end to the output end, wherein the input end of the first convolution layer in sequence constitutes the input end of the residual convolution layer, and the input end of the residual convolution layer is connected to the input end of the connection layer, and the output end of the second ReLu activation layer in sequence constitutes the output end of the residual convolution layer.

[0033] As a preferred technical scheme of the present application: the step E trains the obtained full-floating raft culture area extraction model, and the overall accuracy OA, the intersection over union IoU, and the F1 index are used to realize precision evaluation.

[0034]

[0035]

[0036]

[0037]

[0038]

[0039] Wherein, k represents the number of classification categories, pii represents the number of i-th category samples predicted as i-th category, pij represents the number of i-th category samples predicted as j-th category, pji represents the number of j-th category samples predicted as i-th category, and N represents the total number of pixels in the image.

[0040] Compared with the prior art, the full-floating raft culture area extraction method based on polarimetric SAR images has the following technical effects:

[0041] (1) The method for extracting a full-floating raft aquaculture area based on polarimetric SAR images, each sample polarimetric SAR image is preprocessed, polarimetric decomposed and band combined to obtain a corresponding multi-dimensional channel feature image, a multi-dimensional channel feature slice image is inputted, a full-floating raft aquaculture local area in the multi-dimensional channel feature slice image is outputted, a feature network model is trained, and a full-floating raft aquaculture area extraction model is obtained; then the full-floating raft aquaculture area extraction model is applied to the polarimetric SAR image to be analyzed to obtain a corresponding full-floating raft aquaculture area; the designed method fully utilizes the polarization characteristics of the full-floating raft aquaculture in the polarimetric SAR image, and the influence of different polarization characteristics on the extraction of the full-floating raft aquaculture area is considered, different polarization characteristics are optimized, a feature set suitable for the extraction of the full-floating raft aquaculture area is constructed, and the advantages of deep learning are combined to realize the extraction task of the full-floating raft aquaculture area and reduce the influence of spot noise and the background environment;

[0042] (2) In the method for extracting a full-floating raft aquaculture area based on polarimetric SAR images, a feature network model based on detail perception multi-scale features is constructed, the ASPP module is referenced in the model to further mine deeper semantic features of the full-floating raft aquaculture area, and the breakpoint problem is effectively solved; meanwhile, a detail perception module is designed, the spatial attention mechanism and the squeeze excitation mechanism are referenced in the detail perception module, the detail features of the full-floating raft aquaculture area are effectively extracted, the interference of the detail features of the non-full-floating raft aquaculture area is inhibited, the ability of the detail perception multi-scale feature network model to improve local feature extraction and global information expression is exerted, and the integrity of the extracted full-floating raft aquaculture area is improved. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 It is a flowchart of the method for extracting a full-floating raft aquaculture area based on polarimetric SAR images designed by the application;

[0044] Figure 2 It is a structural schematic diagram of the feature network model designed by the application;

[0045] Figure 3 It is a structural schematic diagram of the residual convolutional layer in the feature network model designed by the application;

[0046] Figure 4 It is a structural schematic diagram of the ASPP module in the feature network model designed by the application;

[0047] Figure 5 It is a structural schematic diagram of the spatial attention mechanism module in the feature network model designed by the application;

[0048] Figure 6is a structure schematic diagram of an extrusion excitation module SE in a feature network model designed by the application;

[0049] Figure 7 is a full floating raft aquaculture area extraction result graph in a test set under the application of an embodiment designed by the application;

[0050] Figure 8 is a full floating raft aquaculture area extraction result graph in a to-be-analyzed Gaofen-3 image under the application of an embodiment designed by the application. DETAILED DESCRIPTION

[0051] The specific embodiments of the application will be further described in detail below with reference to the accompanying drawings of the specification. Based on the embodiments in the application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the application.

[0052] It can be understood that the related features in the methods in the specification and claims and drawings of the application can be mutually referred. In addition, the "first", "second", etc. in the specification and claims and drawings of the application are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein.

[0053] The core technical problem to be solved by the application is that optical images cannot observe full floating raft aquaculture, while polarized SAR images have this advantage, but the existing technology has the problem of low precision in extracting full floating raft aquaculture based on polarized SAR images. In view of this technical problem, the application proposes a full floating raft aquaculture area extraction method based on polarized SAR images, as shown in the following formula (1), the following steps A to E are performed to obtain a full floating raft aquaculture area extraction model. Figure 1

[0054] Step A. For a preset number of sample polarized SAR images each containing a known full floating raft aquaculture area, for each sample polarized SAR image in the sample image set, first extract the scattering matrix S of the sample polarized SAR image, perform matrix multiplication on the scattering matrix S and its conjugate transpose S H to obtain the coherence matrix T3, and then subtract the diagonal matrix of the coherence matrix T3 to obtain the covariance matrix C3; then according to the coherence matrix T3 and the covariance matrix C3, sequentially perform geographic coding, radiation scaling, and filtering using the Refined Lee filtering method to update each sample polarized SAR image, and construct a sample image set, and then proceed to step B.

[0055] ​Step B. For each sample polarimetric SAR image in the sample image set, perform polarimetric decomposition to obtain each target polarimetric feature image corresponding to the sample polarimetric SAR image, and construct a multi-dimensional channel feature image through band combination, that is, obtain a multi-dimensional channel feature image corresponding to each sample polarimetric SAR image, and then enter Step C.

[0056] For obtaining the multi-dimensional channel feature image corresponding to each sample polarimetric SAR image in Step B, two embodiment modes are specifically designed in actual application. In Embodiment One, for each sample polarimetric SAR image in the sample image set, perform the following steps B1-1 to B1-3 to obtain the multi-dimensional channel feature image corresponding to each sample polarimetric SAR image.

[0057] Step B1-1. According to the coherence matrix T3 and the covariance matrix C3 corresponding to each sample polarimetric SAR image obtained in Step A, perform a preset coherent polarimetric decomposition and an incoherent polarimetric decomposition on the sample polarimetric SAR image to obtain each corresponding polarimetric feature image, and then enter Step B1-2.

[0058] Step B1-2. For each polarimetric feature image, calculate the JM distance value J between the background region and the full-floating raft farming area in the polarimetric feature image according to the following formula:

[0059] J = 2(1 - e -B ) (1)

[0060]

[0061] The JM distance value J between the background region and the full-floating raft farming area in the polarimetric feature image is calculated, and the highest separability between the background region and the full-floating raft farming area is taken as the criterion to obtain each target polarimetric feature image, and then enter Step B1-3; wherein m1 and m2 represent the mean value of the polarimetric feature of the background region and the mean value of the polarimetric feature of the full-floating raft farming area in the polarimetric feature image, and σ1 and σ2 represent the standard deviation of the polarimetric feature of the background region and the standard deviation of the polarimetric feature of the full-floating raft farming area in the polarimetric feature image.

[0062] Step B1-3. For each target polarimetric feature image corresponding to the sample polarimetric SAR image, construct a multi-dimensional channel feature image through band combination.

[0063] In Embodiment Two, for each sample polarimetric SAR image in the sample image set, perform the following steps B2-1 to B2-2 to obtain the multi-dimensional channel feature image corresponding to each sample polarimetric SAR image.

[0064] Step B2-1. According to the coherence matrix T3 and the covariance matrix C3 corresponding to each sample polarimetric SAR image, preset coherent and incoherent polarimetric decompositions are performed on the sample polarimetric SAR image to obtain a Neu_mod polarimetric feature image under Neumann polarimetric decomposition, a Y4_Dbl polarimetric feature image under Yamaguchi polarimetric decomposition, a H polarimetric feature image under H polarimetric decomposition, and a VV polarimetric feature image under S2 matrix polarimetric decomposition, which constitute each target polarimetric feature image, and then enter step B2-2.

[0065] Step B2-2. Each target polarimetric feature image corresponding to the sample polarimetric SAR image is combined by a band to form a multi-dimensional channel feature image.

[0066] Step C. Based on the Unet model, a detail perception module is designed and an ASPP module is introduced to construct a feature network model based on detail perception multi-scale features, and then enter step D.

[0067] In actual application, as shown in Figure 2 the feature network model includes a main road and a detail perception branch, the main road includes an encoding network, an ASPP module, a decoding network, and an output module in series from the input end to the output end; the encoding network includes a pre-convolution normalization activation module and three groups of encoding feature modules in series from the input end to the output end, the input end of the pre-convolution normalization activation module constitutes the input end of the encoding network, and the output end of the last group of encoding feature modules constitutes the output end of the encoding network; the decoding network includes three groups of decoding feature modules, a terminal splicing layer, and a middle convolution normalization activation module in series from the input end to the output end, the input end of the first group of decoding feature modules constitutes the input end of the decoding network, and the output end of the middle convolution normalization activation module constitutes the output end of the decoding network; the output module includes a SE module and a post-convolution normalization activation module in series from the input end to the output end, the input end of the SE module constitutes the input end of the output module, and the output end of the post-convolution normalization activation module constitutes the output end of the output module.

[0068] The structures of the encoding feature modules are the same as each other, each encoding feature module comprises, in sequence from an input end to an output end, a max-pooling layer, a first residual convolutional layer, an input end of the max-pooling layer constitutes an input end of the encoding feature module, and an output end of the first residual convolutional layer constitutes an output end of the encoding feature module; the structures of the decoding feature modules are the same as each other, each decoding feature module comprises, in sequence from an input end to an output end, an up-sampling layer, a main splicing layer, and a sub-convolutional normalization activation module, an input end of the up-sampling layer constitutes an input end of the decoding feature module, and an output end of the sub-convolutional normalization activation module constitutes an output end of the decoding feature module; the structures of the front convolutional normalization activation module, the middle convolutional normalization activation module, the rear convolutional normalization activation module, and the sub-convolutional normalization activation module in each decoding feature module are the same as each other, each convolutional normalization activation module comprises, in sequence from an input end to an output end, a convolutional layer, a normalization layer, and a ReLU activation layer, an input end of the convolutional layer constitutes an input end of the convolutional normalization activation module, and an output end of the ReLU activation layer constitutes an output end of the convolutional normalization activation module.

[0069] An output end of the front convolutional normalization activation module is connected to input ends of the main splicing layers in the third group of decoding feature modules in sequence; output ends of the first group of encoding feature modules are connected to input ends of the main splicing layers in the second group of decoding feature modules in sequence; and output ends of the second group of encoding feature modules are connected to input ends of the main splicing layers in the first group of decoding feature modules in sequence.

[0070] The detail-aware branch includes four second residual convolutional layers, three attention upsampling modules, three secondary splicing layers, and a spatial attention mechanism module; the output end of the pre-convolution normalization activation module in the encoding network, the output end of the first encoding feature module, the output end of the second encoding feature module, and the output end of the third encoding feature module are respectively connected to the input ends of the second residual convolutional layers one by one, the output end of the second residual convolutional layer connected to the pre-convolution normalization activation module is connected to the input end of the spatial attention mechanism module, the output ends of the second residual convolutional layers connected to the encoding feature modules are respectively connected to the input ends of the attention upsampling modules one by one, the output end of the attention upsampling module connected to the third encoding feature module is connected to the input end of the first secondary splicing layer, the output end of the first secondary splicing layer is connected to the input end of the second secondary splicing layer connected to the output end of the attention upsampling module connected to the first encoding feature module, the output end of the second secondary splicing layer is connected to the input end of the third secondary splicing layer connected to the output end of the spatial attention mechanism module, and the output end of the third secondary splicing layer is connected to the input end of the terminal splicing layer in the decoding network; the structures of the attention upsampling modules are the same as each other, and each attention upsampling module includes a spatial attention mechanism module and an upsampling module connected in series from the input end to the output end, wherein the input end of the spatial attention mechanism module constitutes the input end of the attention upsampling module, and the output end of the upsampling module constitutes the output end of the attention upsampling module.

[0071] And in the application, as shown in Figure 4 The ASPP includes a pooling layer, a connection layer, an upsampling layer, and three 1x1 convolutional layers and three 3x3 convolutional layers, the input ends of the pooling layer, the three 3x3 convolutional layers, and the first 1x1 convolutional layer are connected to constitute the input end of the ASPP, the output end of the pooling layer is connected to the input end of the second 1x1 convolutional layer, the output end of the second 1x1 convolutional layer is connected to the input end of the upsampling layer, the output end of the first 1x1 convolutional layer, the output ends of the three 3x3 convolutional layers, and the output end of the upsampling layer are connected to the input end of the connection layer, the output end of the connection layer is connected to the input end of the third 1x1 convolutional layer, and the output end of the third 1x1 convolutional layer constitutes the output end of the ASPP.

[0072] The structures of the spatial attention mechanism modules are the same as each other, as shown in Figure 5As shown, each spatial attention mechanism module includes, in sequence from the input end to the output end, a convolutional layer, a normalization layer, a ReLu activation layer, a convolutional layer, a normalization layer, and a Sigmoid activation layer, wherein the input end of the first convolutional layer constitutes the input end of the spatial attention mechanism module, and the output end of the Sigmoid activation layer constitutes the output end of the spatial attention mechanism module.

[0073] The structure of each first residual convolutional layer and the structure of each second residual convolutional layer are the same as each other, as shown in Figure 3 As shown, each residual convolutional layer includes, in sequence from the input end to the output end, a convolutional layer, a normalization layer, a ReLu activation layer, a convolutional layer, a normalization layer, a connection layer, and a ReLu activation layer, wherein the input end of the first convolutional layer constitutes the input end of the residual convolutional layer, the input end of the residual convolutional layer is connected to the input end of the connection layer, the output end of the second ReLu activation layer constitutes the output end of the residual convolutional layer, and in actual applications, the application of the squeeze-and-excitation module SE is as shown in Figure 6 .

[0074] Step D: For each multi-dimensional channel feature image, slice the multi-dimensional channel feature image according to a preset pixel size and a preset slice overlap rate to obtain each multi-dimensional channel feature slice image, and determine the full-floating raft farming local area in each multi-dimensional channel feature slice image according to the known full-floating raft farming area in the sample polarimetric SAR image, and then proceed to Step E. In actual applications, the training set, the validation set, and the test set are divided according to a certain proportion, and the training set and the validation set are randomly cropped, randomly rotated, randomly flipped, enhanced in contrast, and processed with added noise.

[0075] Step E: Based on each multi-dimensional channel feature slice image corresponding to the full-floating raft farming area extraction model obtained by training the feature network model according to the multi-dimensional channel feature slice image as the input and the full-floating raft farming local area in the multi-dimensional channel feature slice image as the output.

[0076] In actual applications, the overall accuracy OA, the intersection over union IoU, and the F1 index are used to evaluate the precision of the full-floating raft farming area extraction model obtained by the above training.

[0077]

[0078]

[0079]

[0080]

[0081]

[0082] wherein k represents the number of classification categories, pi i represents the number of i-th category samples predicted as i-th category, pi j represents the number of i-th category samples predicted as j-th category, pji represents the number of j-th category samples predicted as i-th category, and N represents the total number of pixels in the image.

[0083] Based on the obtaining of the full-floating raft type aquaculture area extraction model, further according to Figure 1 As shown in the figure, the following steps i to iii are performed to realize full-floating raft type aquaculture area extraction for the polarimetric SAR image to be analyzed.

[0084] Step i. According to the methods of steps A and B, pre-processing and polarimetric decomposition are performed on the polarimetric SAR image to be analyzed to obtain the multi-dimensional channel feature image corresponding to the polarimetric SAR image to be analyzed, and according to the method of step D, slicing is performed on the multi-dimensional channel feature image to be analyzed to obtain each multi-dimensional channel feature slice image, and then step ii is entered.

[0085] Step ii. For each multi-dimensional channel feature slice image to be analyzed, a full-floating raft type aquaculture area extraction model is applied for processing to obtain a full-floating raft type aquaculture local area in the multi-dimensional channel feature slice image to be analyzed, and then step iii is entered.

[0086] Step iii. According to the inverse operation of slicing in step i, the multi-dimensional channel feature slice images to be analyzed are combined to obtain the combination of each full-floating raft type aquaculture local area, that is, the full-floating raft type aquaculture area in the polarimetric SAR image corresponding to the multi-dimensional channel feature image to be analyzed is obtained.

[0087] The technical scheme of the above embodiment method will be described in detail using a specific application scenario.

[0088] A QPSI full-polarization strip mode high-resolution SAR image covering the sea area of the Haizhou Bay area in Lianyungang, China, was obtained from the China Resource Satellite Application Center, with an imaging time of November 29, 2020, a center incidence angle of 24.87°, an azimuth resolution of 5.24 meters, and a range resolution of 2.25 meters. Full-floating raft type aquaculture appears as a long strip of gray-white area in this image, and the boundary is relatively fuzzy. When drawing the sample area, attention should be paid to the definition of the full-floating raft type aquaculture boundary to improve the quality of the sample set.

[0089] After obtaining the Gaofen-03 image and analyzing the characteristics of the full-floating raft culture in the Gaofen-03 image, firstly, a scattering matrix S is extracted from the Gaofen-03 image; then the scattering matrix S and its conjugate transpose SH are multiplied to obtain a coherence matrix T3, and the coherence matrix T3 is subtracted from its diagonal matrix to obtain a covariance matrix C3; then geographic coding, radiation scaling, and filtering by using the Refined Lee filtering method are sequentially performed to update the sample Gaofen-03 image; and then the sample Gaofen-03 image is cropped to divide it into a sample polarimetric SAR image and a polarimetric SAR image to be analyzed. Then, the full-floating raft culture local area is vectorized for the sample polarimetric SAR image one by one, and a label image is obtained through rasterization to construct a sample image set.

[0090] Then, according to the coherence matrix T3 and the covariance matrix C3, preset coherent and incoherent polarization decompositions are performed on the sample polarimetric SAR image to obtain a polarimetric characteristic image; the JM distance value between the background area and the full-floating raft culture area in the polarimetric characteristic image is calculated, and the highest separability between the background area and the full-floating raft culture area is taken as the criterion to obtain each target polarimetric characteristic image corresponding to the sample Gaofen-03 image, and a multi-dimensional channel characteristic image is constructed by band combination; according to the multi-dimensional channel characteristic image and the corresponding label image, the multi-dimensional channel characteristic image and the label image are sliced according to a preset pixel size and a preset slice overlap rate to obtain a multi-dimensional channel characteristic slice image and a corresponding label slice image, and a training set, a validation set and a test set are divided according to a certain proportion, and the training set and the validation set are randomly cropped, randomly rotated, randomly flipped, enhanced in contrast, and processed by adding noise.

[0091] The enhanced training set and the validation set are used as the input of the network model based on the detail perception multi-scale feature, the loss value between the prediction result and the real label is calculated by using the loss function, and the obtained loss value is back propagated to update the network parameters; through multiple iterative training, when the model performance no longer improves, the best model parameters are saved, that is, the full-floating raft culture area extraction model is obtained. The test sample is tested based on the trained best network model, and the extraction result of the full-floating raft culture local area of the test set is as shown in FIG. 6. Figure 7 The OA, IoU and F1-score indexes are used to evaluate the extraction accuracy, and the extraction accuracy is compared with that of other popular semantic segmentation network models, as shown in Table 1. The indexes of the network based on the detail perception multi-scale feature are better than those of other models.

[0092] Table 1

[0093] Model OA IoU F1-score Deeplabv3+ 96.22 56.50 72.20 SegNet 96.46 61.35 76.04 UNet 97.04 64.29 78.26 DBHENet 97.26 69.67 82.12

[0094] Then, the pre-processing and polarization decomposition are performed on the to-be-analyzed GF-3 image to obtain a to-be-analyzed multi-dimensional channel feature image corresponding to the to-be-analyzed GF-3 image, and the to-be-analyzed multi-dimensional channel feature image is sliced according to the method in step D to obtain each to-be-analyzed multi-dimensional channel feature slice image; the full-floating raft culture area extraction model is applied to process the to-be-analyzed multi-dimensional channel feature slice image to obtain a full-floating raft culture local area in the to-be-analyzed multi-dimensional channel feature slice image; finally, the to-be-analyzed multi-dimensional channel feature slice images are combined according to the inverse operation of slicing in step i to obtain a combination of each full-floating raft culture local area in the to-be-analyzed multi-dimensional channel feature image, that is, a full-floating raft culture local area in the to-be-analyzed polarimetric SAR image corresponding to the to-be-analyzed multi-dimensional channel feature image, as shown in Figure 8

[0095] The full-floating raft culture area extraction method designed in the application obtains a multi-dimensional channel feature image corresponding to each sample polarimetric SAR image through pre-processing, polarization decomposition, and band combination of each sample polarimetric SAR image, and then applies slicing processing, takes the multi-dimensional channel feature slice image as input, takes the full-floating raft culture local area in the multi-dimensional channel feature slice image as output, trains the feature network model constructed to obtain the full-floating raft culture area extraction model, and then applies the full-floating raft culture area extraction model to the to-be-analyzed polarimetric SAR image to obtain the corresponding full-floating raft culture area. The designed method fully utilizes the polarization features of the full-floating raft culture in the polarimetric SAR image, and considers the influence of different polarization features on the extraction of the full-floating raft culture area, optimizes the features of different polarization features, constructs a feature set suitable for the extraction of the full-floating raft culture area, and combines the advantages of deep learning to realize the extraction task of the full-floating raft culture area and reduce the influence of speckle noise and background environment.

[0096] A feature network model based on detail perception multi-scale features is constructed, the ASPP module is referenced in the model to further mine deeper semantic features of the full-floating raft culture area, and the breakpoint problem is effectively solved. Meanwhile, a detail perception module is designed, the spatial attention mechanism and the squeeze-and-excitation mechanism are referenced in the detail perception module, the detail features of the full-floating raft culture area are effectively extracted, and the interference of the detail features of the non-full-floating raft culture area is suppressed, the ability of the detail perception multi-scale feature network model to improve local feature extraction and global information expression is exerted, and the integrity of the extracted full-floating raft culture area is improved.

[0097] The embodiments of the application are described in detail above in combination with the drawings, but the application is not limited to the above-described embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the purpose of the application.​

Claims

1. A method for extracting fully floating raft aquaculture areas based on polarimetric SAR images, characterized in that: Perform steps A to E to obtain the fully floating raft aquaculture area extraction model, and then perform steps i to iii to extract the fully floating raft aquaculture area for the polarimetric SAR image to be analyzed. Step A. Perform preprocessing on a preset number of polarimetric SAR images, each containing a known fully floating raft aquaculture area, and construct a sample image set before proceeding to Step B; Step B. Perform polarization decomposition on each sample polarimetric SAR image in the sample image set to obtain the target polarization feature images corresponding to each sample polarimetric SAR image. Then, construct multi-dimensional channel feature images by combining bands. That is, obtain the multi-dimensional channel feature images corresponding to each sample polarimetric SAR image. Then proceed to step C. Step C. Based on the Unet model, construct a feature network model, and then proceed to step D; Step D. For each multi-channel feature image, slice the multi-channel feature image according to the preset pixel size and the preset overlap rate between slices to obtain each multi-channel feature slice image. Based on the known fully floating raft aquaculture area in the sample polarimetric SAR image, determine the fully floating raft aquaculture local area in each multi-channel feature slice image, and then proceed to step E. Step E. Based on the multi-dimensional channel feature slice images corresponding to all multi-dimensional channel feature images, take the multi-dimensional channel feature slice images as input and the local area of ​​the fully floating raft aquaculture in the multi-dimensional channel feature slice images as output, train the feature network model to obtain the fully floating raft aquaculture area extraction model. Step i. Following the methods in Step A and Step B, perform preprocessing and polarization decomposition on the polarimetric SAR image to be analyzed to obtain the multi-dimensional channel feature image to be analyzed corresponding to the polarimetric SAR image to be analyzed. Following the method in Step D, slice the multi-dimensional channel feature image to be analyzed to obtain the multi-dimensional channel feature slice image to be analyzed. Then proceed to Step ii. Step ii. For each multi-dimensional channel feature slice image to be analyzed, apply the fully floating raft aquaculture area extraction model to process it, obtain the fully floating raft aquaculture local area in the multi-dimensional channel feature slice image to be analyzed, and then proceed to step iii. Step iii. Following the reverse operation of slicing in step i, combine the multi-dimensional channel feature slice images to be analyzed to obtain the combination of each fully floating raft aquaculture local area, that is, to obtain the fully floating raft aquaculture area in the polarimetric SAR image to be analyzed corresponding to the multi-dimensional channel feature image to be analyzed.

2. The method for extracting fully floating raft aquaculture areas based on polarimetric SAR images according to claim 1, characterized in that: In step A, for each sample polarimetric SAR image in the sample image set, the scattering matrix S of the sample polarimetric SAR image is first extracted, and the scattering matrix S is then converted to its conjugate transpose S. H Perform matrix multiplication to obtain the coherence matrix T3, then subtract its diagonal matrix from the coherence matrix T3 to obtain the covariance matrix C3; Then, based on the coherence matrix T3 and the covariance matrix C3, geocoding, radiometric calibration, and filtering using the Refined Lee filtering method are performed sequentially to update the polarimetric SAR images of each sample, thereby updating the sample image set.

3. The method for extracting fully floating raft aquaculture areas based on polarimetric SAR images according to claim 2, characterized in that: In step B, for each sample polarimetric SAR image in the sample image set, the following steps B1-1 to B1-3 are performed to obtain the multi-dimensional channel feature images corresponding to each sample polarimetric SAR image. Step B1-1. Based on the coherence matrix T3 and covariance matrix C3 corresponding to each sample polarimetric SAR image obtained in step A, perform preset coherent polarization decomposition and incoherent polarization decomposition on the sample polarimetric SAR image to obtain the corresponding polarization feature images, and then proceed to step B1-2. Step B1-2. For each polarization feature image, calculate the JM distance value between the background region and the fully floating raft aquaculture area in the polarization feature image. Based on the highest separability between the background region and the fully floating raft aquaculture area, obtain the polarization feature image of each target, and then proceed to step B1-3. Step B1-3. For each target polarization feature image corresponding to the sample polarimetric SAR image, a multi-dimensional channel feature image is constructed by combining bands.

4. The method for extracting fully floating raft aquaculture areas based on polarimetric SAR images according to claim 3, characterized in that: In step B1-2, for each polarization feature image, the following formula is used: J=2(1-e -B )#(1) The JM distance J between the background region and the fully floating raft aquaculture area in the polarization feature image is calculated, where m1 and m2 represent the mean polarization feature of the background region and the mean polarization feature of the fully floating raft aquaculture area in the polarization feature image, respectively, and σ1 and σ2 represent the standard deviation of polarization feature of the background region and the standard deviation of polarization feature of the fully floating raft aquaculture area in the polarization feature image, respectively.

5. The method for extracting fully floating raft aquaculture areas based on polarimetric SAR images according to claim 2, characterized in that: In step B, for each sample polarimetric SAR image in the sample image set, the following steps B2-1 to B2-2 are performed to obtain the multi-dimensional channel feature images corresponding to each sample polarimetric SAR image. Step B2-1. Based on the coherence matrix T3 and covariance matrix C3 corresponding to each sample polarimetric SAR image obtained in Step A, perform preset coherent polarization decomposition and incoherent polarization decomposition on the sample polarimetric SAR images to obtain the Neu_mod polarization feature image under Neumann polarization decomposition and the Y4_Dbl polarization feature image under Yamaguchi polarization decomposition. H-polarization feature image under polarization decomposition and The polarization feature images and the VV polarization feature images under the S2 matrix eigenpolarization decomposition constitute the polarization feature images of each target, and then proceed to step B2-2; Step B2-2. For each target polarization feature image corresponding to the sample polarimetric SAR image, a multi-dimensional channel feature image is constructed by combining bands.

6. The method for extracting fully floating raft aquaculture areas based on polarimetric SAR images according to claim 1, characterized in that: The feature network model constructed in step C includes, from input to output, a sequentially connected encoding network, a spatial pyramid pooling module (ASPP), a decoding network, and an output module. The encoding network, from input to output, includes a sequentially connected pre-convolutional normalized activation module and three sets of encoding feature modules. The input of the pre-convolutional normalized activation module constitutes the input of the encoding network, and the output of the last set of encoding feature modules constitutes the output of the encoding network. The decoding network, from input to output, includes three sets of decoding feature modules and a middle convolutional normalized activation module. The input of the first set of decoding feature modules constitutes the input of the decoding network, and the output of the middle convolutional normalized activation module constitutes the output of the decoding network. The output module, from input to output, includes a sequentially connected squeeze activation module (SE) and a post-convolutional normalized activation module. The input of the squeeze activation module (SE) constitutes the input of the output module, and the output of the post-convolutional normalized activation module constitutes the output of the output module. Each encoding feature module has the same structure. From input to output, each encoding feature module includes a max pooling layer and a first residual convolutional layer connected in series. The input of the max pooling layer constitutes the input of the encoding feature module, and the output of the first residual convolutional layer constitutes the output of the encoding feature module. Similarly, each decoding feature module has the same structure. From input to output, each decoding feature module includes an upsampling layer, a main concatenation layer, and a sub-convolutional normalization activation module connected in series. The input of the upsampling layer constitutes the input of the decoding feature module, and the output of the sub-convolutional normalization activation module constitutes the output of the decoding feature module. The structures of the pre-convolutional normalization activation module, the mid-convolutional normalization activation module, the post-convolutional normalization activation module, and the sub-convolutional normalization activation modules in each decoding feature module are identical. Each convolutional normalization activation module includes a convolutional layer, a normalization layer, and a ReLU activation layer connected in series from input to output. The input of the convolutional layer constitutes the input of the convolutional normalization activation module, and the output of the ReLU activation layer constitutes the output of the convolutional normalization activation module. The output of the pre-convolutional normalized activation module is simultaneously connected to the input of the main splicing layer in the third group of sequential decoding feature modules; the output of the first group of sequential encoding feature modules is simultaneously connected to the input of the main splicing layer in the second group of sequential decoding feature modules; and the output of the second group of sequential encoding feature modules is simultaneously connected to the input of the main splicing layer in the first group of sequential decoding feature modules.

7. The method for extracting fully floating raft aquaculture areas based on polarimetric SAR images according to claim 6, characterized in that: It also includes a detail-aware branch, which comprises four second residual convolutional layers, three attention upsampling modules, three sub-concatenation layers, and one spatial attention mechanism module. The decoding network also includes an end-concatenation layer, where the output of the third group of sequential decoding feature modules is connected to the middle convolutional normalization activation module via the end-concatenation layer. In the encoding network, the outputs of the pre-convolutional normalization activation module, the first sequential encoding feature module, the second encoding feature module, and the third encoding feature module are each connected to the input of a second residual convolutional layer. The output of the second residual convolutional layer connected to the pre-convolutional normalization activation module is connected to the input of the spatial attention mechanism module. The outputs of the second residual convolutional layers connected to each encoding feature module are each connected to the input of an attention upsampling module. The third sequential encoding feature module... The output of the attention upsampling module connected to the first secondary concatenation layer is connected to the output of the attention upsampling module connected to the second sequential encoding feature module. The output of the first secondary concatenation layer is connected to the input of the second secondary concatenation layer. The output of the second secondary concatenation layer is connected to the input of the spatial attention mechanism module connected to the third secondary concatenation layer. The output of the third secondary concatenation layer is connected to the input of the final concatenation layer in the decoding network. The structures of each attention upsampling module are identical. Each attention upsampling module includes a spatial attention mechanism module and an upsampling module connected in series from the input to the output. The input of the spatial attention mechanism module constitutes the input of the attention upsampling module, and the output of the upsampling module constitutes the output of the attention upsampling module.

8. The method for extracting fully floating raft aquaculture areas based on polarimetric SAR images according to claim 7, characterized in that: Each spatial attention mechanism module has the same structure. Each spatial attention mechanism module consists of a convolutional layer, a normalization layer, a ReLU activation layer, another convolutional layer, a normalization layer, and a Sigmoid activation layer connected in series from the input end to the output end. The input end of the first convolutional layer constitutes the input end of the spatial attention mechanism module, and the output end of the Sigmoid activation layer constitutes the output end of the spatial attention mechanism module.

9. The method for extracting fully floating raft aquaculture areas based on polarimetric SAR images according to claim 7 or 8, characterized in that: The Aperture Spatial Pyramid Pooling Module (ASPP) includes a pooling layer, a connection layer, an upsampling layer, three 1×1 convolutional layers, and three 3×3 convolutional layers. The inputs of the pooling layer, the three 3×3 convolutional layers, and the first 1×1 convolutional layer are connected to form the input of the Aperture Spatial Pyramid Pooling Module (ASPP). The output of the pooling layer is connected to the input of the second 1×1 convolutional layer. The output of the second 1×1 convolutional layer is connected to the input of the upsampling layer. The outputs of the first 1×1 convolutional layer, the three 3×3 convolutional layers, and the upsampling layer are connected to the input of the connection layer. The output of the connection layer is connected to the input of the third 1×1 convolutional layer. The output of the third 1×1 convolutional layer constitutes the output of the Aperture Spatial Pyramid Pooling Module (ASPP). The structures of each first residual convolutional layer and each second residual convolutional layer are identical. Each residual convolutional layer consists of a convolutional layer, a normalization layer, a ReLU activation layer, a convolutional layer, a normalization layer, a connection layer, and a ReLU activation layer, connected in series from the input to the output. The input of the first convolutional layer in sequence constitutes the input of the residual convolutional layer, and the input of the residual convolutional layer is connected to the input of the connection layer. The output of the second ReLU activation layer in sequence constitutes the output of the residual convolutional layer.

10. The method for extracting fully floating raft aquaculture areas based on polarimetric SAR images according to claim 1, characterized in that: The fully floating raft aquaculture area extraction model obtained in step E is evaluated for accuracy using the following overall accuracy OA, intersection-over-union ratio (IoU), and F1 score. Where k represents the number of categories, pii represents the number of samples of category i predicted as category i, pij represents the number of samples of category i predicted as category j, pji represents the number of samples of category j predicted as category i, and N represents the total number of pixels in the image.

Images