A method and system for automatic segmentation of farmland

Abstract

The present invention discloses a method and system for automatic segmentation of farmland, including: obtaining the original remote sensing image, cutting the original remote sensing image into image blocks, sequentially inputting them into the encoder module of the farmland coarse segmentation network, obtaining farmland features at different scales; decoding the farmland features at different scales and the shallow features in the convolution network to obtain the coarse segmentation result; using the coarse segmentation result to locate the pixels belonging to the farmland in the original remote sensing image, performing secondary judgment on the pixels through the farmland fine segmentation network composed of the convolution neural network, eliminating the pixels mistakenly classified as farmland in the coarse segmentation result, and obtaining the fine segmentation result; splicing the fine segmentation results of all the input image blocks in a mean superposition manner to complete the segmentation of the original large image. Advantages: The present invention can solve the problem of different farmland scales and avoid mistakenly segmenting road information such as ridges as farmland.

Description

Technical Field

[0001] The invention relates to a method and system for automatic segmentation of farmland, and belongs to the technical field of image information processing. Background Art

[0002] With the rapid development of aerospace technology, the spatial resolution of remote sensing satellite images has become higher and higher. For example, the spatial resolution of Sentinel-2 images reaches 10 meters. Thanks to such rich spatial information, remote sensing satellite technology can play a very important role in application fields such as land survey, environmental monitoring, crop estimation, construction planning, etc. For example, the present invention focuses on the application of remote sensing images in farmland area statistics, and it is necessary to accurately identify the farmland information in the image. In order to identify the information required for the above-mentioned applications in the image, a variety of recognition methods including semantic segmentation have been proposed, which has played an important role in promoting the realization of agricultural refinement and improving agricultural productivity.

[0003] Semantic segmentation is a typical task in the field of computer vision, which aims to assign a corresponding category label to each pixel in the input image. Early semantic segmentation methods mainly relied on manually designed features, lacked robustness and discriminability, and it was difficult to obtain ideal segmentation results. With the breakthrough progress of convolutional neural networks in computer vision tasks, semantic segmentation models based on fully convolutional networks have achieved great success. It adaptively learns task-related image features through neural networks and converts the last fully connected layer of the network into a convolutional layer, and then uses deconvolution for upsampling to obtain segmentation results consistent with the input image size. However, the strided convolution and pooling operations in the fully convolutional network can easily lead to the problem of spatial semantic information loss, resulting in blurred object boundaries in the segmentation results. In addition, the local receptive field of the convolution operation (experiments have found that the receptive field of the convolutional neural network is much smaller than the theoretical one, especially in the deep layers of the network) also limits the model's ability to extract contextual information.

[0004] In order to capture richer context information, existing work has attempted to fuse network features at multiple scales. For example, an image pyramid structure is used to fuse multi-scale features. Compared with the original fully convolutional network-based model, these methods can obtain richer context information, enhance the ability to recognize object boundary information, and improve the segmentation performance of the model. However, it is still unable to solve the problems of large changes in the same type of objects and small differences between different types of objects in remote sensing images. For example, in some seasons, farmlands in remote sensing images and forests in mountains are both green. Farmlands of different types and regions will show different scales and appearances. When identifying this information, the above methods cannot distinguish these features well, resulting in mis-segmentation. In addition, how to identify small roads in images is also a difficult problem in semantic segmentation. Compared with natural images, the problem of unclear road features such as ridges in high-resolution remote sensing images is more serious. Summary of the invention

[0005] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a method and system for automatic segmentation of farmland. First, multi-scale convolution features are used to solve the problem of different scales of farmland blocks, and then fine segmentation technology is used to identify roads (ridges) between farmlands, thereby obtaining better segmentation performance.

[0006] In order to solve the above technical problems, the present invention provides a method for automatic segmentation of farmland, comprising:

[0007] Obtain the original remote sensing image, crop the original remote sensing image into image blocks, and input them into the encoder module of the farmland coarse segmentation network in sequence to obtain farmland features at different scales;

[0008] Obtaining a coarse segmentation result according to the farmland features at different scales and shallow feature decoding in the convolutional network;

[0009] The pixels belonging to the farmland in the original remote sensing image are located by using the coarse segmentation result, and the pixels are secondarily judged by a farmland fine segmentation network composed of a convolutional neural network, and the pixels mistakenly classified as farmland in the coarse segmentation result are eliminated to obtain a fine segmentation result;

[0010] The fine segmentation results of all input image blocks are spliced ​​by means of mean superposition to complete the segmentation of the original large image.

[0011] Furthermore, the original remote sensing image is cropped into image blocks, which are sequentially input into the encoder module of the farmland coarse segmentation network to obtain farmland features at different scales, including:

[0012] Crop the original remote sensing image X to obtain the cropped image block x i , i∈{1, 2, …, n}, n represents the number of image blocks obtained by cropping;

[0013] Image Block x i As the input of the encoder, feature extraction is performed through the Xception network with hole convolution to obtain the shallow feature f in the network N And the output feature f X , and then use 1×1 convolution layer, 3×3 dilated convolution layer with dilation rate r=6, 3×3 dilated convolution layer with dilation rate r=12, 3×3 dilated convolution layer with dilation rate r=18, and image pooling operation to perform image pooling on f X Perform multi-scale feature extraction to capture farmland features at different scales M , expressed as:

[0014]

[0015] Among them, F, P and C represent convolution, image pooling and feature combination operations respectively, and the upper and lower scripts of the convolution kernel ω represent the expansion rate r and the convolution kernel size respectively.

[0016] Furthermore, the coarse segmentation result is obtained according to the farmland features at different scales and shallow feature decoding in the convolutional network, including:

[0017] The farmland characteristics f at different scales are obtained M Compared with the shallow features f in the backbone network Xception N Fusion is performed to obtain the fusion feature f MN , expressed as:

[0018] f MN =C(F(ω 1×1 ,U(f M )), F(ω 1×1 , f N ))(3)

[0019] Among them, U represents 4 times upsampling using bilinear interpolation;

[0020] Use a 1×1 convolutional layer to fuse the features f MN The number of channels is mapped to the number of categories;

[0021] After the channels are mapped to the number of categories, bilinear interpolation is used to upsample to the same size as the input image block to obtain the coarse segmentation result y of the farmland coarse segmentation network. Fi , expressed as:

[0022] y Fi =U(F(ω 1×1 , f MN ))(4).

[0023] Furthermore, the pixels belonging to farmland in the remote sensing image are located by using the coarse segmentation result, and the pixels are secondarily judged by a farmland fine segmentation network composed of a convolutional neural network, and the pixels mistakenly classified as farmland in the coarse segmentation result are eliminated to obtain a fine segmentation result, including:

[0024] Using the rough segmentation result y Fi , locate x i The pixel points predicted as farmland are judged by the farmland fine segmentation network, and each judgment result d R , according to each judgment result d R Get the judgment matrix D R , and then according to the judgment matrix D R And the rough segmentation result y Fi Get the fine segmentation result y i , expressed as:

[0025] y i =D R ·y Fi (7)

[0026] Each judgment result d R ,include:

[0027] Crop a fixed area with the pixel as the center As the input of the fine segmentation network, the road information is extracted through three convolutional layers to obtain the road features f in the area. R :

[0028]

[0029] f R Expand it into a one-dimensional vector and input it into a fully connected layer with 2 hidden nodes. R Mapped into a two-dimensional vector, the Sigmoid activation function is used on the two-dimensional vector to obtain the probability that the central pixel belongs to the road and farmland. The judgment result d is obtained by the following formula R :

[0030] d R =A(σ(L(f(f R ), 2)))(6)

[0031] Among them, A, σ, L and f represent the argmax function, sigmoid function, fully connected layer and flatten operation respectively.

[0032] Furthermore, the fine segmentation results of all input image blocks are spliced ​​in a mean superposition manner to complete the segmentation of the original large image, including:

[0033] The fine segmentation result y of the image block i The stitching is performed by single-row stitching, single-column stitching and multi-row and multi-column stitching. For the image blocks at the edge of the original image, if there are left and right overlapping areas when the fine segmentation results are stitched, single-row stitching is performed. For the image blocks at non-edge positions of the original image, if there are left and right overlapping areas when the fine segmentation results are stitched, multi-row and multi-column stitching is performed. The predicted value of the overlapping area is replaced by the mean to obtain the remote sensing image segmentation result Y.

[0034] A farmland automatic segmentation system, comprising:

[0035] The cropping module is used to obtain the original remote sensing image, crop the original remote sensing image into image blocks, and input them into the encoder module of the farmland coarse segmentation network in sequence to obtain farmland features at different scales;

[0036] A coarse segmentation module, used for decoding the farmland features at different scales and shallow features in the convolutional network to obtain a coarse segmentation result;

[0037] A fine segmentation module is used to locate the pixels belonging to the farmland in the original remote sensing image by using the coarse segmentation result, and to perform a secondary judgment on the pixels through a farmland fine segmentation network composed of a convolutional neural network, thereby eliminating the pixels that are mistakenly classified as farmland in the coarse segmentation result, and obtaining a fine segmentation result;

[0038] The stitching module is used to stitch the fine segmentation results of all input image blocks in a mean superposition manner to complete the segmentation of the original large image.

[0039] A computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform any of the methods described.

[0040] A computing device comprising:

[0041] One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the methods described.

[0042] The beneficial effects achieved by the present invention are:

[0043] The present invention uses a farmland coarse segmentation network and a farmland fine segmentation network to extract farmland and road information respectively. First, the encoder of the farmland coarse segmentation network uses a convolutional neural network to extract image features, and uses different scales of hole convolution to extract multi-scale information on the features to solve the problem of different scales of farmland; secondly, in order to retain detailed information such as the edge of the farmland, the decoder of the farmland coarse segmentation network uses the output features of the encoder and the shallow feature decoding in the convolutional neural network to obtain the segmentation result; finally, the farmland fine segmentation network performs a secondary judgment on the result of the farmland coarse segmentation network to avoid misclassifying road information such as ridges as farmland. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of the process of the automatic farmland segmentation method of the present invention;

[0045] Figure 2 It is a basic schematic diagram of the automatic farmland segmentation method of the present invention;

[0046] Figure 3 This is a diagram showing the encoder structure of the farmland coarse segmentation network of the sentinel remote sensing image of the present invention;

[0047] Figure 4 This is a decoder structure diagram of the farmland coarse segmentation network of the sentinel remote sensing image of the present invention;

[0048] Figure 5 This is a network structure diagram of the sentinel remote sensing image farmland fine segmentation method of the present invention;

[0049] FIG6(a), FIG6(b), and FIG6(c) are examples of a single-row stitching method, a single-column stitching method, and a multi-row and multi-column stitching method of image blocks proposed in the present invention, respectively;

[0050] FIG. 7( a ) and FIG. 7( b ) are respectively a sentinel remote sensing image of Jinhu County, Huai'an City, Jiangsu Province, and a schematic diagram after road information annotation;

[0051] Figure 8(a), Figure 8(b), and Figure 8(c) are the sentinel remote sensing image of Jinhu County, Huai'an City, Jiangsu Province, the corresponding coarse segmentation visualization result map, and the corresponding fine segmentation visualization result map, respectively;

[0052] FIG9(a), FIG9(b), and FIG9(c) are respectively the sentinel remote sensing image of Jiangsu Province, the corresponding coarse segmentation visualization result map, and the corresponding fine segmentation visualization result map of the present invention. DETAILED DESCRIPTION

[0053] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0054] like Figure 1 and 2 As shown, a method for automatic segmentation of farmland comprises the following steps:

[0055] Step 1) Crop the original remote sensing image into image blocks and input them into the encoder module of the farmland coarse segmentation network in sequence. This module uses hole convolutions of different scales to extract multi-scale features of the image, thereby solving the problem of inconsistent farmland scales in the same scene;

[0056] First, due to the large scale of a single remote sensing image, direct processing requires high computing resources. To this end, the present invention crops the original image and divides it into image blocks for processing. Assuming the original image is X, X is cropped according to a sliding window of 256×256 with a step size of 64 to obtain the cropped image block x i , i∈{1, 2, …, n}, n represents the number of image blocks obtained by cropping.

[0057] In order to capture the rich spatial context information of remote sensing images, the encoder of the farmland coarse segmentation network of the present invention uses dilated convolution for feature extraction to reduce the spatial information loss problem caused by stride convolution or pooling operations. The output feature U can be obtained from the input feature V by the following formula:

[0058] U[j]=∑ k V[j+r·k]ω[k](1)

[0059] Where j represents the spatial position of the input image, ω is the convolution kernel, and r represents the expansion rate of the dilated convolution, which determines the sampling step of the input signal and can expand the receptive field of k*k to k. e =k+(k-1)(r-1). When r=1, it is a standard convolution.

[0060] The encoder of the farmland coarse segmentation network is as follows Figure 3 As shown. Image block x i As the input of the encoder, feature extraction is performed through the Xception network with hole convolution to obtain the shallow feature f in the network N And the output feature f X Then, we use 1×1 convolutional layer, 3×3 dilated convolutional layer with r=6, 3×3 dilated convolutional layer with r=12, 3×3 dilated convolutional layer with r=18, and image pooling operation to perform image pooling on f. X Perform multi-scale feature extraction. Sampling multi-scale features to the same scale, thereby capturing farmland features at different scales. M For f M , which is expressed as follows:

[0061]

[0062] Among them, F, P and C represent convolution, image pooling and feature combination operations respectively. The upper and lower scripts of the convolution kernel ω represent the dilation rate r and the convolution kernel size respectively.

[0063] Step 2), the decoder of the farmland coarse segmentation network uses the output features of the encoder and the shallow features in the convolutional network to decode and obtain the segmentation result to retain the detailed information of the farmland;

[0064] In order to alleviate the problem of detail information loss caused by downsampling operations, the decoder designed in the present invention can combine deep features containing rich spatial context information with shallow features containing more detail information, and retain the detail information of the farmland in a step-by-step upsampling manner.

[0065] The decoder of the farmland coarse segmentation network is as follows Figure 4 As shown. The decoder obtains the multi-scale features f M Compared with the shallow features f in the backbone network XceptionN Fusion feature f MN Obtained through the following formula:

[0066] f MN =C(F(ω 1×1 ,U(f M )), F(ω 1×1 , f N ))(3)

[0067] Among them, U represents 4 times upsampling using bilinear interpolation.

[0068] Then, a convolutional layer of size 1×1 is used to fused the features f MN The number of channels is mapped to the number of categories. After the channels are mapped to the number of categories, bilinear interpolation is used to upsample to the same size as the input image block to obtain the segmentation result y of the farmland coarse segmentation network. Fi .

[0069] y Fi =U(F(ω 3×3 , f MN ))(4)

[0070] Step 3) Use the coarse segmentation results to locate the pixels belonging to farmland in the remote sensing image, and make secondary judgments on these pixels through the farmland fine segmentation network composed of convolutional neural networks, so as to eliminate the pixels that mistakenly classify ridges and other roads as farmland in the coarse segmentation results;

[0071] By observing remote sensing images, we can see that adjacent farmlands are usually separated by roads such as ridges. Since these roads are very small, it is difficult for the coarse segmentation network to fully identify them, and thus they are mistakenly segmented as farmlands. In order to solve this problem, the present invention designs a farmland fine segmentation network. The network judges the farmland coarse segmentation results twice and eliminates road pixels that are mistakenly classified as farmland.

[0072] The farmland segmentation network is as follows Figure 5 As shown. First, use the rough segmentation result y Fi , locate x i The present invention cuts out a 9×9 area with the pixel as the center. As the input of the fine segmentation network. Through the three convolutional layers, the road information can be extracted to obtain the road features f of the area R :

[0073]

[0074] f R Expanded into a one-dimensional vector, input into a fully connected layer with 2 hidden nodes, so that f RMapped into a two-dimensional vector. The Sigmoid activation function is used on this vector to obtain the probability that the central pixel belongs to the road and farmland. The judgment result d is obtained by the following formula R :

[0075] d R =A(σ(L(f(f R ), 2)))(6)

[0076] Among them, A, σ, L and f represent the argmax function, sigmoid function, fully connected layer and flatten operation respectively.

[0077] For the rough segmentation result y Fi All pixels marked as farmland in the image are judged by the farmland segmentation network to obtain the judgment matrix D R , thus obtaining the fine segmentation result y i .y i The expression is as follows:

[0078] y i =D R ·y Fi (7)

[0079] Step 4) The fine segmentation results of the input image blocks are spliced ​​in a mean superposition manner to complete the segmentation of the original large image.

[0080] The fine segmentation result y of the image block i The stitching is performed by single-row stitching, single-column stitching and multi-row and multi-column stitching. For the image blocks at the edge of the original image, if there are left and right overlapping areas when the fine segmentation results are stitched, single-row stitching is performed; for the image blocks at non-edge positions of the original image, if there are left and right overlapping areas when the fine segmentation results are stitched, single-column stitching is performed; for the image blocks at non-edge positions of the original image, if there are left and right overlapping areas when the fine segmentation results are stitched, multi-row and multi-column stitching is performed to obtain the remote sensing image segmentation result Y.

[0081] The overlapping situations are mainly divided into the following three types: single row stitching, single column stitching, and multi-row and multi-column stitching. The following is a description of the calculation of the overlapping parts in the three cases in combination with Figure 6(a), Figure 6(b), and Figure 6(c). In Figure 6(a), Figure 6(b), and Figure 6(c), the shaded part represents the overlapping part of the image blocks during the stitching process. The darker the color, the more overlaps there are, and the number represents the number of overlaps. In the single row stitching of the three image blocks A, B, and C shown in Figure 6(a), image block B has two overlapping areas with image blocks A and C, and the number of overlaps is l = 2. For example, the overlapping area L 1 , the superposition length is 192, and its final probability value is:

[0082]

[0083] The upper and lower scripts A and B are used to indicate the horizontal and vertical coordinate ranges of the image block, respectively.

[0084] The same is true for the single-column splicing of the three image blocks shown in Figure 6(b); the nine image blocks shown in Figure 6(c) are spliced ​​in multiple rows and columns, with different overlapping areas overlapping at different times. The final value is also obtained by means of mean superposition in the above manner.

[0085] Embodiment The technical solution provided by the present invention is applied to the sentinel remote sensing image data of Jinhu County, Huai'an City, Jiangsu Province. The spatial resolution of the image is 10 meters and the size is 4600×4436. The image consists of 4 channels, and the channel order is: red, green, blue, and near infrared. The image will be segmented into 2 types of targets: farmland and background (non-farmland).

[0086] The present invention utilizes multi-scale features to solve the problem of uneven farmland scales, and uses a multi-stage segmentation strategy to obtain detailed information of roads such as ridges, so as to obtain accurate farmland segmentation results.

[0087] First, the encoder of the farmland coarse segmentation network uses a convolutional neural network to extract sentinel image features, and uses dilated convolutions of different scales to extract multi-scale features to solve the problem of different scales of farmland; then, in order to retain the detailed information of the farmland, the decoder of the farmland coarse segmentation network uses the output features of the encoder and the shallow feature decoding in the convolutional neural network to obtain the segmentation result; finally, the farmland fine segmentation network makes a secondary judgment on the results of the farmland coarse segmentation network to avoid misclassifying roads as farmland.

[0088] The classification process of this embodiment is as follows:

[0089] 1. The original sentinel image is cropped into image blocks and input into the encoder of the farmland coarse segmentation network in sequence. The encoder uses a convolutional neural network to extract sentinel image features and uses dilated convolutions of different scales to extract multi-scale features to solve the problem of different farmland scales:

[0090] Since the sentinel image itself is very large in scale, it is difficult to process it with current GPU resources. Therefore, the present invention uses a 256×256 sliding window with a step length of 64 to cut the original sentinel image into image blocks.

[0091] The farmland rough segmentation network encoder of the present invention uses Xception with dilated convolution as the backbone network of the model to perform preliminary feature extraction; a 1×1 convolution layer, a 3×3 dilated convolution layer with a dilation rate of 6, a 3×3 dilated convolution layer with a dilation rate of 12, a 3×3 dilated convolution layer with a dilation rate of 18, and an image pooling operation are used to extract multi-scale information for the extracted features, thereby capturing farmland features at different scales;

[0092] 2. The decoder of the farmland coarse segmentation network uses the output features of the encoder and the shallow features in the convolutional network to decode the segmentation results to retain the detailed information of the farmland:

[0093] The farmland coarse segmentation network decoder of the present invention fuses the obtained multi-scale features with the shallow features in the convolutional network, uses a 1×1 convolutional layer to map the number of channels of the output features into the number of categories, and uses bilinear interpolation to perform 4 times upsampling to obtain the segmentation result of the farmland coarse segmentation network.

[0094] In order to effectively train the farmland coarse segmentation network, the Gaofen Image Dataset (GID) dataset is selected as the experimental training data. GID is a large dataset for land use and land cover classification. It contains 150 high-quality Gaofen-2 (GF-2) images from more than 60 different cities in China, with an image size of 6908×7300 pixels. Multispectral provides images in blue, green, red, and near-infrared bands.

[0095] 3. Use the coarse segmentation results to locate the pixels of farmland in the sentinel image, and make secondary judgments on these pixels through the farmland fine segmentation network composed of convolutional neural networks, so as to eliminate the pixels that mistakenly segment the ridges and other roads as farmland in the coarse segmentation results:

[0096] The farmland fine segmentation network consists of three convolutional layers and one fully connected layer. Each convolutional layer consists of a 3×3 convolution kernel, which is used to extract features from the image. The number of hidden nodes in the fully connected layer is set to 2, aiming to map the convolutional features into a two-dimensional vector. The vector value represents the probability that the input image belongs to the road and farmland. The farmland fine segmentation network makes a secondary judgment on the results of the farmland coarse segmentation network to avoid misclassifying the road as farmland.

[0097] As shown in Figure 7(a) and Figure 7(b), in order to segment the road more accurately, the present invention manually annotates the road information of the Jinhu Sentinel image to obtain the road information of the entire image, and uses this information as training data for the farmland fine segmentation network. For a certain pixel point in the road training data, a 9×9 small image block is cut out with the pixel point as the center. The label of the center pixel point approximately represents the label of the entire small image block. The image block with the center pixel point as the road is defined as a positive sample, and the image block with the center pixel point as a non-road is defined as a negative sample. According to the positive and negative sample ratio of 1:1, the training data is obtained to train the farmland fine segmentation network.

[0098] 4. The fine segmentation results of the input image blocks are spliced ​​by means of mean superposition to complete the segmentation of the original large image:

[0099] According to the cropping method in step 1, all fine segmentation result image blocks are spliced, and the prediction values ​​of the overlapping parts are replaced by the mean value. The sentinel image segmentation result is obtained.

[0100] The Jinhu County Sentinel Image Experiment is shown in Figure 8(a), Figure 8(b), and Figure 8(c). After the image preprocessing operation, the image is first passed through the farmland coarse segmentation network to perform preliminary farmland segmentation on the preprocessed image. On this basis, the farmland fine segmentation network is used to make a secondary judgment on the preliminary result and refine the segmentation result. From the experimental results, it can be seen that the accuracy of the result after fusing the coarse segmentation network and the fine segmentation network is significantly improved.

[0101] In order to further verify the generalization performance of the proposed model, a test experiment was conducted on the sentinel images of Jiangsu Province. The experimental results of Jiangsu Province images are shown in Figures 9(a), 9(b), and 9(c). Similar to the results of Jinhu County, the accuracy of the results after fusing the coarse segmentation network and the fine segmentation network is significantly improved.

[0102] The present invention also provides a farmland automatic segmentation system, comprising:

[0103] The cropping module is used to obtain the original remote sensing image, crop the original remote sensing image into image blocks, and input them into the encoder module of the farmland coarse segmentation network in sequence to obtain farmland features at different scales;

[0104] A coarse segmentation module, used for decoding the farmland features at different scales and shallow features in the convolutional network to obtain a coarse segmentation result;

[0105] A fine segmentation module is used to locate the pixels belonging to the farmland in the original remote sensing image by using the coarse segmentation result, and to perform a secondary judgment on the pixels through a farmland fine segmentation network composed of a convolutional neural network, thereby eliminating the pixels that are mistakenly classified as farmland in the coarse segmentation result, and obtaining a fine segmentation result;

[0106] The stitching module is used to stitch the fine segmentation results of all input image blocks in a mean superposition manner to complete the segmentation of the original large image.

[0107] The present invention also provides a computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by a computing device, the computing device performs any of the methods described.

[0108] The present invention also provides a computing device, comprising:

[0109] One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the methods described.

[0110] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0111] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0112] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0114] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for automatic segmentation of farmland, characterized in that: include: Obtain the original remote sensing image, crop the original remote sensing image into image blocks, and input them into the encoder module of the farmland coarse segmentation network in sequence to obtain farmland features at different scales; Obtaining a coarse segmentation result according to the farmland features at different scales and shallow feature decoding in the convolutional network; The pixels belonging to the farmland in the original remote sensing image are located by using the coarse segmentation result, and the pixels are secondarily judged by a farmland fine segmentation network composed of a convolutional neural network, and the pixels mistakenly classified as farmland in the coarse segmentation result are eliminated to obtain a fine segmentation result; The fine segmentation results of all input image blocks are spliced ​​by means of mean superposition to complete the segmentation of the original large image.

2. The automatic farmland segmentation method according to claim 1, characterized in that: The original remote sensing image is cut into image blocks, which are sequentially input into the encoder module of the farmland coarse segmentation network to obtain farmland features at different scales, including: Crop the original remote sensing image X to obtain the cropped image block x i , i∈{1, 2, …, n}, n represents the number of image blocks obtained by cropping; Image Block x i As the input of the encoder, feature extraction is performed through the Xception network with hole convolution to obtain the shallow feature f in the network N And the output feature f X , and then use 1×1 convolution layer, 3×3 dilated convolution layer with dilation rate r=6, 3×3 dilated convolution layer with dilation rate r=12, 3×3 dilated convolution layer with dilation rate r=18, and image pooling operation to perform image pooling on f X Perform multi-scale feature extraction to capture farmland features at different scales M , expressed as: Among them, F, P and C represent convolution, image pooling and feature combination operations respectively, and the upper and lower scripts of the convolution kernel ω represent the expansion rate r and the convolution kernel size respectively.

3. The automatic farmland segmentation method according to claim 2, characterized in that: The coarse segmentation result is obtained according to the farmland features at different scales and shallow feature decoding in the convolutional network, including: The farmland characteristics f at different scales are obtained M Compared with the shallow features f in the backbone network Xception N Fusion is performed to obtain the fusion feature f MN , expressed as: f MN =C(F(ω 1×1 ,U(f M )),F(ω 1×1 ,f N ))(3) Among them, U represents 4 times upsampling using bilinear interpolation; Use a 1×1 convolutional layer to fuse the features f MN The number of channels is mapped to the number of categories; After the channels are mapped to the number of categories, bilinear interpolation is used to upsample to the same size as the input image block to obtain the coarse segmentation result y of the farmland coarse segmentation network. Fi , expressed as: y Fi =U(F(ω 1×1 ,f MN ))(4)。 4. The automatic farmland segmentation method according to claim 3, characterized in that: The pixels belonging to farmland in the remote sensing image are located by using the coarse segmentation result, and the pixels are secondarily judged by a farmland fine segmentation network composed of a convolutional neural network, and the pixels mistakenly classified as farmland in the coarse segmentation result are eliminated to obtain a fine segmentation result, including: Using the rough segmentation result y Fi , locate x i The pixel points predicted as farmland are judged by the farmland fine segmentation network, and each judgment result d R , according to each judgment result d R Get the judgment matrix D R , and then according to the judgment matrix D R And the rough segmentation result y Fi Get the fine segmentation result y i , expressed as: y i =D R ·y Fi (7) Each judgment result d R ,include: Crop a fixed area with the pixel as the center As the input of the fine segmentation network, the road information is extracted through three convolutional layers to obtain the road features f in the area R : f R Expand it into a one-dimensional vector and input it into a fully connected layer with 2 hidden nodes. R Mapped into a two-dimensional vector, the Sigmoid activation function is used on the two-dimensional vector to obtain the probability that the central pixel belongs to the road and farmland. The judgment result d is obtained by the following formula R : d R =A(σ(L(f(f R ),2)))(6) Among them, A, σ, L and f represent the argmax function, sigmoid function, fully connected layer and flatten operation respectively.

5. The automatic farmland segmentation method according to claim 4, characterized in that: The fine segmentation results of all input image blocks are spliced ​​in a mean superposition manner to complete the segmentation of the original large image, including: The fine segmentation result y of the image block i The stitching is performed by single-row stitching, single-column stitching and multi-row and multi-column stitching. For the image blocks at the edge of the original image, if there are left and right overlapping areas when the fine segmentation results are stitched, single-row stitching is performed. For the image blocks at non-edge positions of the original image, if there are left and right overlapping areas when the fine segmentation results are stitched, multi-row and multi-column stitching is performed. The predicted value of the overlapping area is replaced by the mean to obtain the remote sensing image segmentation result Y.

6. A farmland automatic segmentation system, characterized in that: include: The cropping module is used to obtain the original remote sensing image, crop the original remote sensing image into image blocks, and input them into the encoder module of the farmland coarse segmentation network in sequence to obtain farmland features at different scales; A coarse segmentation module, used for decoding the farmland features at different scales and shallow features in the convolutional network to obtain a coarse segmentation result; A fine segmentation module is used to locate the pixels belonging to the farmland in the original remote sensing image by using the coarse segmentation result, and to perform a secondary judgment on the pixels through a farmland fine segmentation network composed of a convolutional neural network, thereby eliminating the pixels that are mistakenly classified as farmland in the coarse segmentation result, and obtaining a fine segmentation result; The stitching module is used to stitch the fine segmentation results of all input image blocks in a mean superposition manner to complete the segmentation of the original large image.

7. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions which, when executed by a computing device, cause the computing device to perform any one of the methods according to claims 1 to 5 .

8. A computing device, characterized in that include, One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any one of the methods according to claims 1 to 5.

Images