Natural field type detection method based on multi-task learning

Abstract

This invention discloses a natural field shape detection method based on multi-task learning, comprising the following steps: Step S1, generating ridge polygonal patches using boundary information of cultivated land patches; Step S2, acquiring training remote sensing samples with ridge boundaries; Step S3, constructing a semantic segmentation model; Step S4, constructing a topological network model, and identifying inflection point information using the segmented ridges; Step S5, inputting images collected from different regions, segmenting cultivated land and ridge polygonal patches using the multi-task combined semantic segmentation model and topological network model, embedding the ridge polygonal patches into the cultivated land, and obtaining the final collinear cultivated land detection patch result. The method of this invention utilizes the ability of semantic segmentation to divide images into cultivated land, ridges, and background, without requiring temporal imagery; by further processing the feature maps of the ridges, the gaps between cultivated land areas are obtained, and the connection relationships between ridges are learned using a topological network, thus further subdividing the natural field shape.

Description

Technical Field

[0001] The present invention belongs to the technical field of cultivated land plot information extraction and remote sensing image processing, and specifically relates to a natural field type detection method based on multi-task learning. Background Art

[0002] At present, farmland detection not only needs to identify farmland, but also needs to have precise boundaries, which has important application value in the agricultural field. Accurate farmland boundaries are more conducive to the construction of smart agriculture, such as land use planning and management, land resource detection and evaluation, and precision agriculture and automated agriculture. Existing farmland identification solutions are mainly divided into the following categories:

[0003] 1. Farmland segmentation scheme based on time series characteristics

[0004] In publication number CN114973012A, a method for identifying cultivated land based on low- and medium-resolution remote sensing data is proposed. The method uses EVI multi-time series data; a ground object sample library is constructed through a calculation scheme in the remote sensing field, and a mixed pixel linear spectral decomposition model is used to segment cultivated land, thereby improving the accuracy of cultivated land identification; in publication number CN111666914A, a method for identifying cultivated land based on the distance between curves is proposed. The method first obtains the reference vegetation index NDVI time series curve of cultivated land crops and the NDVI time series curve in the current area; the two curves are adjusted and distance-weighted to obtain the target curve distance; and the pixels corresponding to the target curve are then divided into cultivated land pixels.

[0005] 2. Farmland segmentation solution based on semantic segmentation

[0006] In publication number CN115861837A, a method for identifying cultivated land based on the ARes-UNet++ network is proposed. By making a data set based on the GF-2 remote sensing image and constructing the ARes-Unet++ network, cultivated land with different textures and spectral characteristics can be effectively identified. In publication number CN116994025A, a method for improving the accuracy of cultivated land identification using image super-resolution technology is proposed. The method includes using low-resolution remote sensing images as input to an image super-resolution model to obtain high-resolution images; using high-resolution super-resolution remote sensing images as input to a cultivated land identification classification model to output cultivated land identification results. Summary of the invention

[0007] The purpose of the present invention is to provide a natural field type detection method based on multi-task learning to solve the problem that in the prior art proposed in the background technology, by acquiring multi-time series images of the same area, the data collection cost is relatively expensive, and if interfered by clouds, the image quality will be seriously reduced, affecting the overall cultivated land identification ability.

[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0009] The natural field type detection method based on multi-task learning includes the following steps:

[0010] Step S1, collecting remote sensing images within a set period and cultivated land patches of the remote sensing images within the period, and generating ridge line patches using boundary information of the cultivated land patches;

[0011] Step S2, preprocessing the cultivated land patches and remote sensing images to obtain training remote sensing samples with field ridge boundaries;

[0012] Step S3, constructing a semantic segmentation model for segmenting the background, cultivated land and ridges in the remote sensing image;

[0013] Step S4, constructing a topological network model, using the field ridges segmented in step S3 to identify the field ridge breakpoint information, and matching the breakpoints with the best connecting lines;

[0014] Step S5, input the remote sensing images collected from different regions, use the semantic segmentation model and the topological network model to segment the cultivated land and the ridge line patches, embed the ridge line patches into the cultivated land, and obtain the final collinear cultivated land detection patch results.

[0015] According to the above technical solution, in step S1, the field ridge line map is generated by using the boundary information of the cultivated land map as follows:

[0016] By collecting satellite and past production data, we can obtain remote sensing images of the area to be measured and the cultivated land patch data of the same period, and obtain the boundary data of the cultivated land; align the geographic coordinates of the remote sensing images, cultivated land patch data and cultivated land boundary data to generate ridge line patches.

[0017] According to the above technical solution, in step S2, obtaining the training remote sensing samples with ridge boundaries is specifically: cutting the generated ridge line pattern generated in step S1 into the same size to produce the training remote sensing samples.

[0018] According to the above technical solution, in step S3, constructing a semantic segmentation model is specifically as follows:

[0019] In the first step, a linear transformation layer is used to increase the channel by two times, and then it is equally divided into two parts, namely Global features and Local features, the formula is as follows:

[0020]

[0021] Where Linear (f) represents the linear change function, and f is the input image;

[0022] Among them, the learning of global features is completed by the visual transformer, while the learning of local features is completed by a layer of 7×7 deep convolution. The formula for the change of the two parts of the features is as follows:

[0023]

[0024] Where Multiply(Q, K) represents the matrix multiplication operation; DWConv(f l ) represents deep separation convolution; softmax is a normalization function, Multiply represents the multiplication of two matrices, softmax(Multiply(Q, K)) represents the calculation of the similarity between two features, Multiply(Softmax(Multiply(Q,K)), V) represents the calculation of the weighted result of V in the similarity matrix to obtain the final multi-head self-attention result; Q, K, and V of global feature learning are all the same as same;

[0025] Finally, the global features and local features are fused and sent to the multi-layer perceptron, and then added to the original features to obtain features with global and local representation capabilities. The feature calculation formula is as follows:

[0026]

[0027] In the formula, F represents the feature map after the formula change; Sum represents summation; Norm represents normalization, f represents the input image, and MLP represents multi-layer perceptron, which is only used as a feedforward network.

[0028] According to the above technical solution, in step S4, constructing a topological network model is specifically as follows:

[0029] The topological network model includes a multi-layer perceptron, two sliding window self-attention mechanisms, and a point classification layer;

[0030] After obtaining the relationship between each point and other points, the lines with low connection probability are filtered through the threshold, and finally ridges are formed. The linear structure of the ridges is used to cut the cultivated land patches to obtain the final locally collinear cultivated land.

[0031] According to the above technical solution, in step S5, the multi-task combined semantic segmentation model and topological network model are used to segment the cultivated land and the ridge line patches, and the ridge line patches are embedded in the cultivated land, and the final collinear cultivated land detection patch result is specifically as follows:

[0032] The multi-task combination semantic segmentation model includes semantic segmentation and auxiliary detection semantic segmentation losses. Both semantic segmentation and auxiliary detection semantic segmentation losses use cross entropy loss:

[0033]

[0034] Where, L ce represents the cross entropy loss; Indicates the mean of N samples; N indicates the number of samples; n indicates the current calculation sample; k indicates a certain category; K indicates the number of categories; represents the true label; Indicates the prediction result;

[0035] The key point topology connection loss is the binary cross entropy loss:

[0036]

[0037] Where, L bce represents the binary cross entropy loss, y i represents the true label, p i Represents the prediction result; i represents a sample.

[0038] According to the above technical solution, the overall loss is calculated based on the cross entropy loss and the binary cross entropy loss, specifically:

[0039]

[0040] Among them, L ce represents the cross entropy loss, L bce represents the binary cross entropy loss; α is the loss weight value for segmenting cultivated land, ridges and background, β is the auxiliary loss weight value, and γ is the topological connection prediction loss weight value.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] Through the method of the present invention, the image is divided into cultivated land, ridges and background by utilizing the ability of semantic segmentation, without the need for time series images; by further processing the feature map of the ridges, the gaps between the cultivated land are obtained, and the connection relationship between the ridges is learned using a topological network to subdivide the natural field type. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is the overall processing flow chart of the present invention;

[0044] Figure 2 It is a schematic diagram of a decoder module of the present invention;

[0045] Figure 3 This is a diagram showing the model verification results for hilly areas of the present invention;

[0046] Figure 4 This is a diagram showing the verification results of the model in plain areas of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Embodiment 1

[0049] like Figure 1 As shown, the natural field type detection method based on multi-task learning includes the following steps:

[0050] Step S1, collecting remote sensing images within a set period and cultivated land patches of the remote sensing images of the period, and generating ridge line patches using boundary information of the cultivated land patches;

[0051] Step S2, preprocessing the cultivated land patches and remote sensing images to obtain training remote sensing samples with field ridge boundaries;

[0052] Step S3, constructing a semantic segmentation model to distinguish the background, cultivated land and ridges in the remote sensing image;

[0053] Step S4, constructing a topological network model, using the field ridges segmented in step S3 to identify the inflection point information, and matching the inflection points with the best connection lines;

[0054] Step S5, input the remote sensing images collected from different regions, use the multi-task combination of semantic segmentation model and topological network model to segment the cultivated land and ridge line patches, embed the ridge line patches into the cultivated land, and obtain the final collinear cultivated land detection patch results.

[0055] Through the method of the present invention, the image is divided into cultivated land, ridges and background by utilizing the ability of semantic segmentation, without the need for time series images; by further processing the feature map of the ridges, the gaps between the cultivated land are obtained, and the connection relationship between the ridges is learned using a topological network to subdivide the natural field type.

[0056] Embodiment 2

[0057] This embodiment is a further refinement of the first embodiment. This embodiment provides a specific implementation method.

[0058] Dataset creation

[0059] Through satellite and past production data, remote sensing images and corresponding cultivated land patch data of a certain area in Sichuan were obtained, and the boundary data of cultivated land was obtained by using the surface to fold line of ArcGIS software; the geographic coordinates of the above three types of data (images, cultivated land patches, and field ridge boundary data) were aligned, and they were cropped to the same size to make training remote sensing samples, in which the field ridge boundary width was enlarged by a certain proportion, and the field ridge fold line data generated a fold line connection topological relationship diagram and a fold point marking diagram in a single image.

[0060] Semantic segmentation model construction. The present invention designs a semantic segmentation model with a UNet-like structure, in which the pre-trained SAM is used in the feature extraction stage. The input image is , after multiple SAM encoders, sub-feature graphs F1, F2, F3, and F4 are obtained, where the larger the subscript, the higher the processed computing module.

[0061]

[0062] The decoder consists of a global feature learning module and a local feature learning module. The overall structure is as follows: Figure 2 shown.

[0063] Global context is crucial for semantic segmentation of complex remote sensing scenes, but local information is crucial for farmland boundary extraction. In order to strengthen the learning of the two parts of features, a global-local attention is proposed, and two parallel branches are constructed to extract global and local contexts respectively. Local feature extraction uses a 7×7 deep convolution scheme to extract context; global feature extraction uses a visual transformer module to extract feature maps. A single-head self-attention scheme is adopted to learn global feature information, while a window-based single-head self-attention is adopted for other feature maps to capture global context information.

[0064] The overall module is a residual structure. In the first step, a linear change layer is used to increase the channel by two times, and then it is equally divided into two parts, namely Global features and Local features, the formula is as follows:

[0065]

[0066] In the formula, Linear(f) represents the linear change function, and f is the input image.

[0067] The learning of global features is done through the visual transformer, while the learning of local features is done by a layer of 7×7 deep convolution. The formula for the change of the two features is as follows:

[0068]

[0069] Where Multiply(Q, K) represents the matrix multiplication operation; DWConv(f l ) represents deep separation convolution; softmax is a normalization function, Multiply represents the multiplication of two matrices, softmax(Multiply(Q, K)) represents the calculation of the similarity between two features, Multiply(Softmax(Multiply(Q,K)), V) represents the calculation of the weighted result of V in the similarity matrix to obtain the final multi-head self-attention result; Q, K, and V of global feature learning are all the same as same;

[0070] Finally, the global features and local features are fused and sent to the multi-layer perceptron, and then added to the original features to obtain features with global-local representation capabilities. The feature calculation formula is as follows:

[0071]

[0072] In the formula, F represents the feature map after the formula change; Sum represents summation; Norm represents normalization, f represents the input image, and MLP represents multi-layer perceptron.

[0073] Topological network construction: The ridges are composed of multiple linear patches, while the segmented ridges are composed of strips, which causes the ridges to erode the cultivated land area. At the same time, there are some breakpoints in the ridge detection. The main function of the topological network proposed in this invention is to achieve the refinement of the ridges and connect the local breakpoints. The ridges are refined by skeletonization to obtain the ridge features with a width of 1, and then the embedding method is used to obtain the ridge features with a width of 1. The interpolated feature map assigns a feature vector to each point in the segmentation pattern of the ridge. The topological network learns through sliding window self-attention and assigns each ridge vertex the probability of connecting to other points.

[0074] The overall composition of the topological network consists of a multi-layer perceptron, two sliding window self-attention mechanisms and a point classification layer. After obtaining the relationship between each point and other points, the lines with low connection probability are filtered through the threshold, and finally the ridges are formed. The linear structure of the ridges is used to cut the cultivated land patches to obtain the final locally collinear cultivated land.

[0075] Multi-task learning balance: The present invention involves the learning of multiple tasks including auxiliary semantic segmentation, semantic segmentation and topological connection. In order to balance the calculation of multiple losses, the present invention performs multiple rounds of iterations under the parameter setting of equal loss distribution, and then redistributes the loss ratio of each task through the early training weights, and uses the dynamic loss weights to optimize the overall loss of the model, so that the overall performance of the model is optimized.

[0076] The multi-task combination semantic segmentation model includes semantic segmentation and auxiliary detection semantic segmentation losses. Both semantic segmentation and auxiliary detection semantic segmentation losses use cross entropy loss:

[0077]

[0078] Where, L ce represents the cross entropy loss; Indicates the mean of N samples; N indicates the number of samples; n indicates the current calculation sample; k indicates a certain category; K indicates the number of categories; represents the true label; Indicates the prediction result;

[0079] The key point topology connection loss is the binary cross entropy loss:

[0080]

[0081] Where, L bce represents the binary cross entropy loss, y i represents the true label, p i represents the prediction result, and i represents a sample.

[0082] The overall loss is:

[0083]

[0084] Among them, L ce represents the cross entropy loss, L bce represents the binary cross entropy loss; α is the loss weight value for segmenting cultivated land, ridges and background, β is the auxiliary loss weight value, and γ is the topological connection prediction loss weight value.

[0085] Multi-region model verification: The model training data set is the Sichuan hilly area. In order to verify the performance of the model, the surrounding hilly areas are selected. The inference results are converted into a patch file using remote sensing geoprocessing tools. The black frame represents the prediction boundary, and the inside of the frame is the predicted cultivated land. The results are shown in the following figure 3. Figure 4 shown.

[0086] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0087] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A natural field type detection method based on multi-task learning, characterized by: The following steps are involved: Step S1, collecting remote sensing images within a set period and cultivated land patches of the remote sensing images within the period, and generating ridge line patches using boundary information of the cultivated land patches; Step S2, preprocessing the cultivated land patches and remote sensing images to obtain training remote sensing samples with field ridge boundaries; Step S3, constructing a semantic segmentation model for segmenting the background, cultivated land and ridges in the remote sensing image; the semantic segmentation model is composed of a linear change layer that doubles the channel, and then equally cuts it into two parts, namely global features and local features; the learning of global features is completed by a visual transformer, while the learning of local features is completed by deep convolution; finally, the global features and local features are fused and sent to a multi-layer perceptron, and then added to the original features to obtain features with global and local representation capabilities; Step S4, constructing a topological network model, using the field ridges segmented in step S3 to identify the field ridge breakpoint information, and matching the breakpoints with the best connection lines; the topological network model includes a multi-layer perceptron, two sliding window self-attention mechanisms, and a point classification layer; after obtaining the relationship between each point and other points, the lines with low connection probability are filtered through the threshold, and finally the field ridges are formed, and the linear structure of the field ridges is used to cut the cultivated land patches to obtain the final locally collinear cultivated land; Step S5, input the remote sensing images collected from different regions, use the semantic segmentation model and the topological network model to segment the cultivated land and the ridge line patches, embed the ridge line patches into the cultivated land, and obtain the final collinear cultivated land detection patch results.

2. The natural field type detection method based on multi-task learning according to claim 1, characterized in that: In step S1, the boundary information of the cultivated land patch is used to generate the field ridge line patch as follows: By collecting satellite and past production data, we can obtain remote sensing images of the area to be measured and the cultivated land patch data of the same period, and obtain the boundary data of the cultivated land; align the geographic coordinates of the remote sensing images, cultivated land patch data and cultivated land boundary data to generate ridge line patches.

3. The natural field type detection method based on multi-task learning according to claim 1, characterized in that: In step S2, the training remote sensing samples with ridge boundaries are obtained by cutting the ridge line patches generated in step S1 into the same size to prepare the training remote sensing samples.

4. The natural field type detection method based on multi-task learning according to claim 1, characterized in that: In step S3, constructing a semantic segmentation model is specifically as follows: The channel is doubled by a linear change layer and then divided into two equal parts, namely Global features and Local features, the formula is as follows: Where Linear (f) represents the linear change function, and f is the input image; Among them, the learning of global features is completed by visual transformers, while the learning of local features is completed by deep convolution. The formulas for the changes of the two parts of features are as follows: Where Multiply(Q, K) represents matrix multiplication operation; DWConv(f l ) represents deep separation convolution; softmax is a normalization function, Multiply represents the multiplication of two matrices, softmax(Multiply(Q, K)) represents the calculation of the similarity between two features, Multiply(Softmax(Multiply(Q, K)), V) represents the calculation of the weighted result of V in the similarity matrix to obtain the final multi-head self-attention result; Q, K, and V of global feature learning are all the same as same; Finally, the global features and local features are fused and sent to the multi-layer perceptron, and then added to the original features to obtain features with global and local representation capabilities. The feature calculation formula is as follows: In the formula, F represents the feature map after the formula change; Sum represents summation; Norm represents normalization, f represents the input image, and MLP represents multi-layer perceptron.

5. The natural field type detection method based on multi-task learning according to claim 1, characterized in that: In step S5, the multi-task combined semantic segmentation model and topological network model are used to segment the cultivated land and the ridge line patches, and the ridge line patches are embedded into the cultivated land to obtain the final collinear cultivated land detection patch result: The multi-task combination semantic segmentation model includes semantic segmentation and auxiliary detection semantic segmentation losses. Both semantic segmentation and auxiliary detection semantic segmentation losses use cross entropy loss: Where, L ce represents the cross entropy loss; Indicates the mean of N samples; N indicates the number of samples; n indicates the current calculation sample, k indicates a certain category; K indicates the number of categories; represents the true label; Indicates the prediction result; The key point topology connection loss is the binary cross entropy loss: Where, L bce represents the binary cross entropy loss, y i represents the true label, p i Represents the prediction result, and i represents a sample.

6. The natural field type detection method based on multi-task learning according to claim 5, characterized in that: The overall loss is calculated based on the cross entropy loss and the binary cross entropy loss, specifically: Among them, L ce represents the cross entropy loss, L bce represents the binary cross entropy loss; α is the loss weight value for segmenting cultivated land, ridges and background, β is the auxiliary loss weight value, and γ is the topological connection prediction loss weight value.

Images