Rural homestead remote sensing image recognition method

Abstract

The invention discloses a rural homestead remote sensing image recognition method, which comprises the steps of performing super-pixel segmentation on a rural homestead remote sensing image to obtain a plurality of super-pixel nodes, and performing calculation to generate an initial adjacent matrix; constructing a spectral feature vector according to color information of a plurality of pixel points in each superpixel node; taking the average pixel value of the superpixel node in the 3 * 3 neighborhood as a first spatial feature; a Sobel operator is used to calculate a second spatial feature of each superpixel node; splicing the first spatial features and the corresponding second spatial features to obtain spatial feature vectors; inputting an initial feature vector obtained by fusing the spectral feature vector and the spatial feature vector and the initial adjacency matrix into a Mamb-GNN network architecture to obtain a target fusion feature matrix; and analyzing and processing the target fusion feature matrix to obtain a binary mask image of the homestead. According to the method, the small target recognition rate can be improved, and the recognition precision of the multi-scale homestead in a complex rural scene is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing and remote sensing ground object recognition, and particularly relates to a rural homestead remote sensing image recognition method. BACKGROUND

[0002] The rural homestead is a core element of rural land management and planning, and accurate identification of the rural homestead is crucial for homestead right registration and village spatial layout optimization. In the prior art, the rural homestead in the rural homestead remote sensing image is identified by combining satellite remote sensing, unmanned aerial vehicle aerial photography, field investigation and evidence, and geographic information spatial analysis.

[0003] However, the prior art has obvious bottlenecks in extraction: small homesteads (such as single houses) are easily submerged by other ground objects due to weak features, and the boundaries of similar ground objects such as farmland and roads are easily confused, which can easily cause misclassification, and the feature extraction of different scale homesteads (from small courtyards to large homestead groups) is not balanced, and the recognition effect is poor in the scene of densely distributed homesteads, which leads to the difficulty of the overall extraction precision to meet the actual needs of rural fine management. SUMMARY

[0004] Therefore, the embodiments of the present application provide a rural homestead remote sensing image recognition method, which can improve the small target recognition rate and effectively improve the recognition accuracy of multi-scale homesteads in complex rural scenes, and meet the actual needs of rural fine management.

[0005] The present application provides a monocular vision camera calibration and downward angle determination method, comprising: The rural homestead remote sensing image is superpixel segmented to obtain a plurality of superpixel nodes; each superpixel node includes a plurality of pixel points in the rural homestead remote sensing image; a spatial position distance between adjacent superpixel nodes is calculated, and an initial adjacency matrix is generated; the initial adjacency matrix is a sparse symmetric matrix, and the dimension of the initial adjacency matrix is the same as the total number of the superpixel nodes; a spectral feature vector of each superpixel node is constructed according to color information of the plurality of pixel points in the superpixel node; an average pixel value of each superpixel node in a 3*3 neighborhood is calculated, and the average pixel value is taken as a first spatial feature of the corresponding superpixel node; and a convolution operation is performed on a pixel matrix corresponding to the rural homestead remote sensing image by using a Sobel operator, and a second spatial feature of each superpixel node is calculated in combination with a pixel point corresponding to each superpixel node in the pixel matrix; the first spatial feature and the corresponding second spatial feature are spliced to obtain a spatial feature vector corresponding to each superpixel node; an initial feature vector obtained by fusing the spectral feature vector and the corresponding spatial feature vector, and the initial adjacency matrix are input into a Mamba-GNN network architecture to obtain a target fusion feature matrix; the Mamba-GNN network architecture includes a plurality of graph convolution-Mamba layers connected in sequence, and a Mamba state space model is embedded in each graph convolution-Mamba layer; a binary mask image of the homestead is obtained by analyzing and processing the target fusion feature matrix.

[0006] Optionally, the color information includes values of the pixel points in R, G, B and NIR bands; constructing the spectral feature vector of each superpixel node according to the color information of the plurality of pixel points in the superpixel node includes: calculating an average value of all pixel points in a specific superpixel node in the R band, an average value in the G band, an average value in the B band and an average value in the NIR band respectively; the specific superpixel node is any one of the plurality of superpixel nodes; constructing the spectral feature vector of the specific superpixel node according to the average values; the spectral feature vector can be represented as , wherein, The average value of all pixel points in the specific superpixel node in the R band, the average value in the G band, the average value in the B band and the average value in the NIR band are represented by R, G, B and NIR respectively.

[0007] Optionally, the pixel matrix corresponding to the rural homestead remote sensing image is convoluted by using a Sobel operator, and the second spatial feature of each superpixel node is calculated in combination with the pixel point corresponding to each superpixel node in the pixel matrix, including: the pixel matrix corresponding to the rural homestead remote sensing image is convoluted by using a Sobel operator, to obtain a first spatial feature matrix in a horizontal direction and a second spatial feature matrix in a vertical direction; the dimensions of the first spatial feature matrix and the second spatial feature matrix are the same as the dimension of the pixel matrix; the first pixel feature of each superpixel node corresponding to each pixel point is determined from the first spatial feature matrix, and the second pixel feature of each specific superpixel node corresponding to each pixel point is determined from the second spatial feature matrix; the second spatial feature of each superpixel node is calculated according to the first pixel feature and the corresponding second pixel feature.

[0008] Optionally, the initial feature vector obtained by fusing the spectral feature vector and the corresponding spatial feature vector, and the initial adjacency matrix are input into the Mamba-GNN network architecture to obtain the target fusion feature matrix, including: the spectral feature vector and the corresponding spatial feature vector are fused by a multilayer perceptron (MLP) to obtain the initial feature vector corresponding to each superpixel node; the initial feature vectors are sorted according to the center pixel point coordinates of each superpixel node to obtain an initial feature vector sequence; the initial feature vector sequence is constructed into an initial feature matrix, and the initial feature matrix and the initial adjacency matrix are input into a first graph convolution-Mamba layer and sequentially pass through a second graph convolution-Mamba layer to an Nth graph convolution-Mamba layer to obtain N node feature matrices and N dynamic weight parameters; each graph convolution-Mamba layer generates a node feature matrix and corresponding dynamic weight parameters, and the input of the current graph convolution-Mamba layer includes the node feature matrix output by the last graph convolution-Mamba layer and the initial adjacency matrix; N is the total number of graph convolution-Mamba layers; the N node feature matrices are weighted and fused by using the N dynamic weight parameters to obtain the target fusion feature matrix.

[0009] Optionally, each graph convolution-Mamba layer comprises a graph convolution module and a Mamba state space model, and the process that each graph convolution-Mamba layer generates a node feature matrix and corresponding dynamic weight parameters comprises: performing convolution operation on a specific feature matrix, an initial adjacency matrix and a weight matrix by using the graph convolution module to obtain a convolved feature matrix; the specific feature matrix is an initial feature matrix or a node feature matrix output by a previous graph convolution-Mamba layer; performing dynamic feature extraction on the convolved feature matrix and a node size of each superpixel node by using the Mamba state space model to generate the node feature matrix; and determining the dynamic weight parameters corresponding to the node feature matrix according to the node size, and a nonlinear mapping relationship between a reference node size and a reference dynamic weight parameter constructed in a Mamba-GNN network architecture training process; wherein the node size of the superpixel node is determined based on neighborhood pixels in the initial feature vector sequence; the reference node size and the reference dynamic weight parameter are inversely related.

[0010] Optionally, performing dynamic feature extraction on the convolved feature matrix and the node size of each superpixel node by using the Mamba state space model to generate the node feature matrix comprises: inputting the node size into a gating neural network of the Mamba state space model to obtain a state transition selective parameter corresponding to the corresponding superpixel node; and generating the node feature matrix according to the state transition selective parameter, the convolved feature matrix, and a selective state transition equation and an observation equation provided by the Mamba state space model.

[0011] Optionally, the selective state transition equation can be expressed as: wherein, is a time step, respectively represent accumulated context information of the first t superpixel nodes, and new context information formed after the accumulated context information of the first t superpixel nodes is updated, represents a state transition selective parameter corresponding to the tth superpixel node , respectively represent a state transition matrix and an input matrix.

[0012] Optionally, the analysis and processing of the target fusion feature matrix to obtain the binary mask image of the homestead comprises: mapping each feature in the target fusion feature matrix to a corresponding homestead probability by using a fully connected layer to obtain a homestead probability map; and sequentially performing morphological operation and connected region analysis processing on the homestead probability map to obtain the binary mask image.

[0013] In the embodiment of the present application, the rural homestead remote sensing image is superpixel segmented to obtain a plurality of superpixel nodes, the spatial position distance between adjacent superpixel nodes is calculated, and an initial adjacency matrix capable of reflecting the spatial distribution characteristics of ground objects is generated to provide data support for accurate rural homestead detection; according to the color information of the plurality of pixel points in each superpixel node, a spectral feature vector of each superpixel node is constructed; the average pixel value of each superpixel node in a 3*3 neighborhood is calculated, and the average pixel value is taken as the first spatial feature of the corresponding superpixel node; and the Sobel operator is used to perform convolution operation on the pixel matrix corresponding to the rural homestead remote sensing image, and the second spatial feature of each superpixel node is calculated in combination with the corresponding pixel points of each superpixel node in the pixel matrix; the first spatial feature and the corresponding second spatial feature are spliced to obtain the spatial feature vector corresponding to each superpixel node; the initial feature vector obtained by fusing the spectral feature vector and the corresponding spatial feature vector, and the initial adjacency matrix are input into the Mamba-GNN network architecture to obtain a target fusion feature matrix; the target fusion feature matrix is analyzed and processed to obtain a binary mask image of the homestead. In this way, by linking and fusing the spectral feature vector and the corresponding spatial feature vector, the spectral feature and the spatial feature can be deeply integrated and improved, and further based on the dynamic scale perception mechanism of the Mamba state space model embedded in the graph convolution-Mamba layer, the Mamba-GNN network architecture can adaptively adjust the attention degree to different scale features, improve the small target recognition rate, and each graph convolution-Mamba layer works cooperatively to realize deep extraction and fusion of multi-scale features, so as to finally obtain a more accurate binary mask image of the homestead, effectively improve the recognition accuracy of multi-scale homestead in complex rural scenes, and meet the actual needs of rural fine management. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. Figure 1 A flowchart of a rural homestead remote sensing image recognition method provided in the embodiment of the present application. DETAILED DESCRIPTION

[0015] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. The following embodiments are used to describe the present application, but not to limit the scope of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of the present application.

[0016] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0017] It should be noted that the terms "first\second" involved in the embodiments of the present application are only to distinguish similar objects, and do not represent the specific order of the objects. It can be understood that "first\second" can be interchanged with specific order or sequence as allowed, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0018] Those skilled in the art can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as generally understood by those skilled in the art to which the embodiments of the present application belong. It should also be understood that terms such as those defined in a general dictionary should be understood as having a meaning consistent with that in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as such.

[0019] As shown in FIG. 1, a flowchart of a rural homestead remote sensing image recognition method provided by the embodiments of the present application is shown. The method comprises: Figure 1 S101, performing superpixel segmentation on the rural homestead remote sensing image to obtain a plurality of superpixel nodes. S101, performing superpixel segmentation on the rural homestead remote sensing image to obtain a plurality of superpixel nodes.

[0020] It should be noted that each superpixel node (or superpixel region) includes a plurality of pixel points in the rural homestead remote sensing image, and each superpixel node is composed of adjacent pixel points with similar color, texture and other visual features.

[0021] In some embodiments, the rural homestead remote sensing image is a high-resolution (e.g., resolution ≤ 0.5 meters) four-band (including red R, green G, blue B, and near-infrared NIR) remote sensing image. By performing superpixel segmentation on the rural homestead remote sensing image, a plurality of superpixel nodes can be obtained. The superpixel nodes can be set according to actual needs. For example, the number of superpixel nodes can be 500-1000. By segmenting the rural homestead remote sensing image into a plurality of superpixel nodes, the image can be represented by a higher-level unit than the pixels in the original rural homestead remote sensing image, thereby simplifying the subsequent image recognition task.

[0022] S102, calculate the spatial position distance between adjacent superpixel nodes, and generate an initial adjacency matrix.

[0023] It should be noted that the initial adjacency matrix is a sparse symmetric matrix, and the dimension of the initial adjacency matrix is the same as the total number of superpixel nodes.

[0024] In some embodiments, the superpixel nodes having an adjacent relationship can be determined first (if two superpixels are adjacent in space, they share a boundary and have pixel points with the same or similar pixel values), and then the average value of the position coordinates of each pixel point in these superpixel nodes is calculated. The average value of the position coordinates is used as the center point coordinates of the superpixel node. Then, the Euclidean distance between each center point coordinate is calculated, a reference adjacency matrix is constructed based on the calculated Euclidean distances, and the reference adjacency matrix is normalized to obtain a sparse initial adjacency matrix.

[0025] S103, construct a spectral feature vector of each superpixel node according to the color information of the plurality of pixel points in the superpixel node.

[0026] In some embodiments, the four-band color values of each pixel point in the superpixel node can be combined, and a spectral feature vector of the superpixel node can be constructed based on the result of the combination operation.

[0027] S104, calculate the average pixel value of each superpixel node in a 3x3 neighborhood, and use the average pixel value as the first spatial feature of the corresponding superpixel node; and perform convolution operation on the pixel matrix corresponding to the rural homestead remote sensing image using the Sobel operator, and combine the pixel points corresponding to each superpixel node in the pixel matrix to calculate the second spatial feature of each superpixel node.

[0028] In some embodiments, the pixel value of each superpixel node can be calculated based on the pixel value of each pixel point in the superpixel node, and then the average pixel value of the superpixel node in the 3*3 neighborhood is calculated, and the average pixel value is taken as the first spatial feature of the superpixel node. The first spatial feature can be taken as part of the spatial feature vector of the superpixel node. Through this processing process, the features of other superpixel nodes adjacent to the superpixel node are included in the first spatial feature, establishing a connection between the spatial features.

[0029] In an embodiment, the Sobel operator includes a convolution kernel in the horizontal direction and a convolution kernel in the vertical direction. The Sobel operator is used to perform convolution operation on the pixel matrix corresponding to the rural homestead remote sensing image, and two convolved pixel matrices are obtained. The convolved pixel matrix can highlight the change of pixel value in the horizontal direction and the change of pixel value in the vertical direction, enhance the information of horizontal and vertical edges, and improve the pixel expression of edges or textures. Further, the second spatial feature of the superpixel node can be calculated by combining the feature values of each pixel point in the superpixel node in the convolved pixel matrix.

[0030] S105, the first spatial feature and the corresponding second spatial feature are spliced to obtain the respective spatial feature vector corresponding to each superpixel node.

[0031] In some embodiments, the first spatial feature and the second spatial feature of each superpixel node can be spliced respectively. In the process of splicing the first spatial feature and the second spatial feature, the first spatial feature can be taken as the first part of the spatial feature vector, and the second spatial feature can be taken as the second part of the spatial feature vector.

[0032] S106, the initial feature vector obtained by the linkage fusion of the spectral feature vector and the corresponding spatial feature vector, and the initial adjacency matrix are input into the Mamba-GNN network architecture to obtain the target fusion feature matrix.

[0033] It should be noted that the Mamba-GNN network architecture includes a plurality of graph convolution-Mamba layers connected in sequence, and each graph convolution-Mamba layer is embedded with a Mamba state space model.

[0034] In some embodiments, the spectral feature vector and the corresponding spatial feature vector of each superpixel node can be linked and fused to obtain a fused feature vector. The fused feature vector is 64-dimensional. Then, the plurality of fused feature vectors and the initial adjacency matrix are input into the Mamba-GNN network architecture for multi-scale feature extraction and dynamic scale fusion to obtain the target fusion feature matrix.

[0035] S107, analyze and process the target fusion feature matrix to obtain a binary mask image of the homestead.

[0036] In some embodiments, the analysis and processing of the target fusion feature matrix can be converting the feature values in the target fusion feature matrix into corresponding homestead probability values, obtaining a homestead probability map, and performing morphological post-processing on the homestead probability map, and finally obtaining a binary mask image of the homestead.

[0037] In the embodiments of the present application, the rural homestead remote sensing image is superpixel segmented to obtain a plurality of superpixel nodes, the spatial position distance between adjacent superpixel nodes is calculated, an initial adjacency matrix reflecting the spatial distribution characteristics of ground objects is generated, and data support is provided for accurate rural homestead detection; the spectral feature vector of each superpixel node is constructed according to the color information of the plurality of pixel points in each superpixel node; the average pixel value of each superpixel node in a 3x3 neighborhood is calculated, and the average pixel value is taken as the first spatial feature of the corresponding superpixel node; and the Sobel operator is used to perform convolution operation on the pixel matrix corresponding to the rural homestead remote sensing image, and the second spatial feature of each superpixel node is calculated in combination with the corresponding pixel points of each superpixel node in the pixel matrix; the first spatial feature and the corresponding second spatial feature are spliced to obtain the spatial feature vector corresponding to each superpixel node; the initial feature vector obtained by fusing the spectral feature vector and the corresponding spatial feature vector, and the initial adjacency matrix are input into the Mamba-GNN network architecture to obtain a target fusion feature matrix; the target fusion feature matrix is analyzed and processed to obtain a binary mask image of the homestead. In this way, by linking and fusing the spectral feature vector and the corresponding spatial feature vector, the spectral feature and the spatial feature can be deeply integrated and improved, and further based on the dynamic scale perception mechanism of the Mamba state space model embedded in the graph convolution-Mamba layer, the Mamba-GNN network architecture can adaptively adjust the emphasis on different scale features, improve the small target recognition rate, and each graph convolution-Mamba layer works cooperatively to realize deep extraction and fusion of multi-scale features, finally obtain a more accurate binary mask image of the homestead, effectively improve the recognition accuracy of multi-scale homestead in complex rural scenes, and meet the actual needs of rural fine management.

[0038] In some embodiments of the present application, the color information includes the values of the pixel points in the R, G, B and NIR bands, based on which, the construction of the spectral feature vector of each superpixel node according to the color information of the plurality of pixel points in each superpixel node in step S103 can be realized by the following steps S1031 to S1032.

[0039] S1031, calculate the average value of all pixel points in a specific superpixel node on the R band, the average value on the G band, the average value on the B band, and the average value on the NIR band, respectively.

[0040] wherein the specific superpixel node is any one of the plurality of superpixel nodes.

[0041] In some embodiments, the values of each pixel point in the rural homestead remote sensing image on the R, G, B and NIR bands can be obtained first, i.e., the R value, G value, B value and NIR value of the pixel point, and then the average value of all pixel points in each superpixel node on the R band, the average value on the G band, the average value on the B band and the average value on the NIR band are calculated in combination with the R value, G value, B value and NIR value of each pixel point.

[0042] S1032, constructing a spectral feature vector of the specific superpixel node according to the average values.

[0043] In some embodiments, the spectral feature vector can be represented as wherein, respectively represent the average value of all pixel points in the specific superpixel node on the R band, the average value on the G band, the average value on the B band and the average value on the NIR band. That is, the first element in the spectral feature vector is obtained based on the difference between the value on the R band and the value on the NIR band of all pixel points in the superpixel node; the second element is obtained based on the ratio between the value on the G band and the value on the B band of all nodes in the superpixel node; and the third element is obtained based on the average value of the value on the A band, the value on the G band and the value on the B band of all nodes in the superpixel node.

[0044] It should be noted that, is the difference between the red band and the near-infrared band, and the reflectivity of cement roof and other building materials is higher in the near-infrared band, while the reflectivity of farmland covered with vegetation is lower in this band. The difference between the two can effectively highlight the difference in spectrum between the homestead roof and the farmland, and strengthen the feature expression of the building area; is the ratio of the green band to the blue band, and different ground objects have different absorption and reflection characteristics on different visible light bands. This ratio helps to further distinguish different ground object types, such as artificial objects such as roads and roofs, and natural objects, which will show different numerical characteristics in the green-blue band reflectivity ratio; is the average value of the red, green and blue visible light bands, so as to obtain the brightness information of the image, reflecting the overall reflection intensity of the ground object, which helps to distinguish the light and dark degree of different ground objects as a whole, such as cement roof with high brightness and shadow area with low brightness.

[0045] It can be understood that by calculating the average value of all pixel points in a specific superpixel node in the R band, the average value in the G band, the average value in the B band and the average value in the NIR band respectively, the spectral feature vector of the specific superpixel node is constructed according to the average value, and the vector is obtained by different band operation combinations, which can highlight the spectral difference between the homestead roof and the farmland and the road, so that in the subsequent identification process, the Mamba-GNN network architecture can better distinguish the homestead and the non-homestead.

[0046] In some embodiments of the present application, the second spatial feature of each superpixel node in step S104 can be calculated by performing convolution operation on the pixel matrix corresponding to the rural homestead remote sensing image using the Sobel operator, and combining the pixel points corresponding to each superpixel node in the pixel matrix.

[0047] S1041, the Sobel operator is used to perform convolution operation on the pixel matrix corresponding to the rural homestead remote sensing image, to obtain a first spatial feature matrix in the horizontal direction and a second spatial feature matrix in the vertical direction.

[0048] It should be noted that the dimensions of the first spatial feature matrix and the second spatial feature matrix are the same as the dimensions of the pixel matrix.

[0049] In some embodiments, the Sobel operator can be used to calculate the gradient of the pixel points of the rural homestead remote sensing image in the horizontal direction and the vertical direction respectively, and the pixel matrix corresponding to the rural homestead remote sensing image is subjected to convolution operation to obtain a first spatial feature matrix in the horizontal direction and a second spatial feature matrix in the vertical direction. Through the convolution operation, the change rate of the pixel points of the rural homestead remote sensing image in different directions can be obtained.

[0050] In some embodiments, the Sobel operator provides a convolution kernel in the horizontal direction and a convolution kernel in the vertical direction, and the convolution kernel in the horizontal direction can be used to perform convolution on the pixel matrix corresponding to the rural homestead remote sensing image to obtain the first spatial feature matrix, and the convolution kernel in the vertical direction can be used to perform convolution on the pixel matrix corresponding to the rural homestead remote sensing image to obtain the second spatial feature matrix.

[0051] For example, the convolution operation of the Sobel operator on the pixel matrix corresponding to the rural homestead remote sensing image can be represented by the following formula (1): (1); Wherein, , respectively represent the convolution kernel in the horizontal direction and the convolution kernel in the vertical direction, respectively represent the first spatial feature matrix and the second spatial feature matrix, a pixel matrix corresponding to a remote sensing image of a rural homestead is represented.

[0052] It can be understood that the horizontal direction convolution kernel enhances the horizontal edge information by weighted summation in the horizontal direction of the target pixel of the remote sensing image, the middle column coefficient is 0, the two side column coefficients are symmetrical and one positive and one negative, and the purpose is to highlight the change of the pixel value in the horizontal direction. When there is a horizontal edge in the image, the pixels on both sides of the edge will produce a large difference after convolution, so as to highlight the edge. The vertical direction convolution kernel is similar to the horizontal direction, which enhances the vertical edge information by weighted summation in the vertical direction, the middle row coefficient is 0, the upper and lower row coefficients are symmetrical and the signs are opposite, so that the pixels on both sides of the vertical edge produce a significant difference after convolution.

[0053] S1042, determine the first pixel feature of each pixel point corresponding to each superpixel node from the first spatial feature matrix, and determine the second pixel feature of each pixel point corresponding to each specific superpixel node from the second spatial feature matrix.

[0054] In some embodiments, the first pixel feature of the pixel point corresponding to the superpixel node can be determined from the first spatial feature matrix, and the second pixel feature of the pixel point corresponding to each specific superpixel node can be determined from the second spatial feature matrix according to the coordinate position information of each pixel point in the superpixel node, that is, the pixel feature corresponding to all pixel points in the superpixel node in the first spatial matrix (the first pixel feature) and the pixel feature corresponding to all pixel points in the superpixel node in the second spatial feature matrix (the second pixel feature) are obtained.

[0055] S1043, according to the first pixel feature and the corresponding second pixel feature, calculate the second spatial feature of each superpixel node.

[0056] In some embodiments, the average value of the first pixel feature of the pixel point corresponding to the superpixel node and the average value of the second pixel feature of the pixel point corresponding to the superpixel node can be calculated first, and then the second spatial feature (gradient value) of the superpixel node can be calculated based on the average value of the first pixel feature and the average value of the second pixel feature. For example, the gradient value can be calculated by the following formula (2) : (2) ; wherein, respectively represent the average value of the first pixel feature and the average value of the second pixel feature.

[0057] In other embodiments, the gradient value of each pixel point can be calculated first according to the first pixel feature and the second pixel feature of the pixel point, and then the average value of the gradient values of all pixel points in the superpixel node is taken as the second spatial feature of the superpixel node.

[0058] In some embodiments of the present application, the initial feature vector obtained after the linkage fusion of the spectral feature vector and the corresponding spatial feature vector in step S105, and the initial adjacency matrix are input into the Mamba-GNN network architecture to obtain the target fusion feature matrix, which can be realized through the following steps S1051 to S1054.

[0059] S1051, linkage fusion of the spectral feature vector and the corresponding spatial feature vector is performed through a multi-layer perceptron (MLP) to obtain an initial feature vector corresponding to each superpixel node.

[0060] In some embodiments, the spectral feature vector (dimension 3) and the corresponding spatial feature vector (dimension 2) of the superpixel node can be vertically spliced according to the dimensions to form a combined feature vector, and then linkage fusion of the two types of feature vectors is realized through a multi-layer perceptron (MLP), and finally converted into a 64-dimensional initial feature vector.

[0061] It can be understood that, through linkage fusion of the spectral feature vector and the corresponding spatial feature vector, the spectral feature and the spatial feature are not isolated from each other, but are mutually interpreted and enhanced, the implicit and task strongly related collaborative discriminant pattern between the spectral and spatial features is fused, and new derivative discriminant information (features) is generated, thereby laying a solid and efficient foundation for subsequent identification of rural homestead based on the superpixel node features.

[0062] S1052, according to the center pixel point coordinates of each superpixel node, the initial feature vectors are sorted to obtain an initial feature vector sequence.

[0063] In some embodiments, the center pixel point coordinates can represent the corresponding position of the superpixel node in the rural homestead remote sensing image, and therefore, based on the center pixel point coordinates of each superpixel node, the initial feature vectors are sorted, which means that spatial position information is implicitly injected into the superpixel node features, so that the subsequent Mamba spatial state model can model the long-distance interaction of the superpixel nodes in the image space when scanning the initial feature vector sequence, rather than only the adjacency relationship in the graph structure.

[0064] S1053, the initial feature vector sequence is constructed into an initial feature matrix, and the initial feature matrix and the initial adjacency matrix are input into a first graph convolution-Mamba layer, and sequentially pass through a second graph convolution-Mamba layer to an Nth graph convolution-Mamba layer to obtain N node feature matrices and N dynamic weight parameters.

[0065] wherein each graph convolution-Mamba layer generates a node feature matrix and a corresponding dynamic weight parameter, and the input of the current graph convolution-Mamba layer includes the node feature matrix output by the last graph convolution-Mamba layer and the initial adjacency matrix; N is the total number of graph convolution-Mamba layers.

[0066] In some embodiments, each initial feature vector can be taken as a row of the initial feature matrix in the order of the initial feature vectors in the initial feature vector sequence, so as to obtain an initial feature matrix of N x 64 dimensions.

[0067] In some embodiments, the N graph convolution-Mamba layers are sequentially connected, and the node feature matrix output by the previous graph convolution-Mamba layer can be taken as the input of the next graph convolution-Mamba layer, and the computer mechanism of the Mamba state space model can be used to realize dynamic scale perception of the node features. Meanwhile, each graph convolution-Mamba layer can also generate a dynamic weight parameter used for feature fusion according to the scale of the superpixel node, so as to facilitate multi-scale fusion.

[0068] S1054, weighting fusion of the N node feature matrices is performed by using the N dynamic weight parameters to obtain a target fusion feature matrix.

[0069] In some embodiments, the node feature matrix and the corresponding dynamic weight can be multiplied to obtain a weighted feature matrix, and then the weighted feature matrices are added to obtain the target fusion feature matrix.

[0070] It can be understood that through the cooperative work of the graph convolution-Mamba layers in the Mamba-GNN network architecture, deep extraction and fusion of multi-scale features can be realized, and the problem of insufficient recognition accuracy of multi-scale homestead in complex rural scenes in the traditional method is effectively solved.

[0071] In some embodiments of the present application, each graph convolution-Mamba layer includes a graph convolution module and a Mamba state space model, and the process of generating a node feature matrix and a corresponding dynamic weight parameter by each graph convolution-Mamba layer can be realized by the following steps S201 to S202.

[0072] S201, performing convolution operation on a specific feature matrix, an initial adjacency matrix and a weight matrix by using a graph convolution module to obtain a convolved feature matrix.

[0073] It should be noted that the specific feature matrix is the initial feature matrix or the node feature matrix output by the previous graph convolution-Mamba layer. In the case where the current graph convolution-Mamba layer is the first graph convolution-Mamba layer in the Mamba-GNN network architecture, the specific feature matrix is the initial feature matrix; in the case where the current graph convolution-Mamba layer is any one of the second graph convolution-Mamba layer to the Nth graph convolution-Mamba layer in the Mamba-GNN network architecture, the specific feature matrix is the node feature matrix output by the previous graph convolution-Mamba layer.

[0074] In some embodiments, the weight matrix is a trainable weight matrix of the graph convolution-Mamba layer, which can be updated through backpropagation during the training process of the Mamba-GNN network architecture, and the classification loss value is calculated according to the supervision signal of the homestead detection task until the classification loss value is less than or equal to the classification loss threshold, so as to obtain the trained weight matrix.

[0075] Exemplarily, the convolutional feature matrix can be calculated by the following formula (3): (3) ; Wherein, represents the convolutional feature matrix output by the i-th graph convolution-Mamba layer, represents the initial adjacency matrix, represents the specific feature matrix input into the i-th graph convolution-Mamba layer, represents the weight matrix corresponding to the i-th graph convolution-Mamba layer. S202, using the Mamba state space model to perform dynamic feature extraction on the convolutional feature matrix and the node size of each superpixel node, and generating a node feature matrix; and determining the dynamic weight parameter corresponding to the node feature matrix according to the node size, and the nonlinear mapping relationship between the reference node size and the reference dynamic weight parameter constructed in the training process of the Mamba-GNN network architecture.

[0076] Wherein, the node size of the superpixel node is determined based on the neighborhood pixels in the initial feature vector sequence; the reference node size and the reference dynamic weight parameter are inversely related.

[0077] Wherein, the node size of the superpixel node is determined based on the neighborhood pixels in the initial feature vector sequence; the reference node size and the reference dynamic weight parameter are inversely related.

[0078] ​​In some embodiments, the node size is used to represent the relative scale of the region represented by the superpixel node, and the node size can be in the range of (0, 1). If the node size is close to 1, it means that the corresponding superpixel node is a large-scale region, such as an independent large farm. If the node size is close to 0, it means that the corresponding superpixel node is a small-scale region, such as a dense small farm.

[0079] In some embodiments, the node size of the superpixel node can be calculated based on the geometric score, the feature score and the context score of the superpixel node. The geometric score can be calculated based on the physical shape of the superpixel node. The feature score can be calculated based on the initial feature vector sequence or the initial feature matrix. The context score can be calculated based on the neighborhood relationship of the superpixel node.

[0080] For example, the ratio of the number of pixels contained in the superpixel node to the total number of pixels of the rural residential land remote sensing image can be taken as the basic area of the superpixel node. The shape index (including circularity and compactness) of the superpixel node can be calculated. The weighted sum of the basic area, the circularity and the compactness can be taken as the geometric score of the superpixel node, and the weights corresponding to the basic area, the circularity and the compactness can be set to 0.6, 0.2 and 0.2 respectively.

[0081] For example, the brightness of the superpixel node can be calculated according to the 3rd dimension (D3) of the 64-dimensional initial feature vector of the superpixel node (D1, D2, D3, D4, D5, D6, D7, D8, D9, D10, D11, D12, D13, D14, D15, D16, D17, D18, D19, D20, D21, D22, D23, D24, D25, D26, D27, D28, D29, D30, D31, D32, D33, D34, D35, D36, D37, D38, D39, D40, D41, D42, D43, D44, D45, D46, D47, D48, D49, D50, D51, D52, D53, D54, D55, D56, D57, D58, D59, D60, D61, D62, D63, D64). The texture complexity of the superpixel node can be calculated based on the 5th dimension (D5) of the 64-dimensional initial feature vector (D1, D2, D3, D4, D5, D6, D7, D8, D9, D10, D11, D12, D13, D14, D15, D16, D17, D18, D19, D20, D21, D22, D23, D24, D25, D26, D27, D28, D29, D30, D31, D32, D33, D34, D35, D36, D37, D38, D39, D40, D41, D42, D43, D44, D45, D46, D47, D48, D49, D50, D51, D52, D53, D54, D55, D56, D57, D58, D59, D60, D61, D62, D63, D64). The vegetation suppression index of the superpixel node can be calculated according to the 1st dimension (D1) of the initial feature vector (D1, D2, D3, D4, D5, D6, D7, D8, D9, D10, D11, D12, D13, D14, D15, D16, D17, D18, D19, D20, D21, D22, D23, D24, D25, D26, D27, D28, D29, D30, D31, D32, D33, D34, D35, D36, D37, D38, D39, D40, D41, D42, D43, D44, D45, D46, D47, D48, D49, D50, D51, D52, D53, D54, D55, D56, D57, D58, D59, D60, D61, D62, D63, D64). The weighted sum of the brightness, the texture complexity and the vegetation suppression index can be taken as the feature score of the superpixel node, and the weights corresponding to the brightness, the texture complexity and the vegetation suppression index can be set to 0.6, 0.2 and 0.2 respectively.

[0082] For example, the direct neighbor set of each superpixel node can be obtained from the initial adjacency matrix. The ratio of the basic area of the superpixel node to the average basic area of the neighbor superpixel nodes can be taken as the area contrast of the superpixel node. The feature similarity can be calculated according to the cosine distance between the superpixel node and each neighbor node. The ratio of the number of neighbor sets of the superpixel node to the maximum number of neighbors in all superpixel nodes can be taken as the density of the superpixel node. The weighted sum of the area contrast, the feature similarity and the density can be taken as the context score of the superpixel node, and the weights of the area contrast, the feature similarity and the density can be set to 0.5, 0.3 and 0.2 respectively.

[0083] In some embodiments, the Mamba state space model can serve as a dynamic feature processor to dynamically extract the feature matrix after convolution based on the node size of the superpixel node to obtain a node feature matrix; meanwhile, the Mamba state space model can serve as a multi-scale fusion controller to generate a dynamic weight parameter for feature fusion.

[0084] In some embodiments, the step S202 of dynamically extracting the feature matrix after convolution and the node size of each superpixel node by using the Mamba state space model to generate the node feature matrix can be implemented by the following steps S2021 to S2022: S2021, input the node size to the gating neural network of the Mamba state space model to obtain the state transition selectivity parameter corresponding to the superpixel node.

[0085] In some embodiments, the node size of each superpixel node can be determined first, and then the activation function of the gating neural network, such as the SiLU (Sigmoid Linear Unit) function, can be used to perform nonlinear transformation on each node size to obtain the state transition selectivity parameter of the corresponding superpixel node. The parameters of the activation function of the gating neural network can be determined in the Mamba-GNN network architecture training process.

[0086] S2022, generate the node feature matrix according to the state transition selectivity parameter, the feature matrix after convolution, and the selective state transition equation and observation equation provided by the Mamba state space model.

[0087] In some embodiments, the selective state transition equation provided by the Mamba state space model is shown in formula (4): (4); wherein, is the update step length, represents the cumulative context information of the first t superpixel nodes, represents the new context information formed after updating the cumulative context information of the first t superpixel nodes, represents the state transition selectivity parameter corresponding to the t-th superpixel node, represents the feature vector of the t-th superpixel node (corresponding to a row in the feature matrix after convolution), represents the state transition matrix, which is used to control the decay or growth of the state . represents the input matrix, which is used to control how to be projected and integrated into .

[0088] The observation equation provided by the Mamba state space model is shown in formula (5): (5); wherein, represents the feature representation output by the Mamba state space model after processing the t-th superpixel node, represents an observation matrix, used to project to the output space, and determine which features of contribute more to the output. represents a pass-through matrix, which can be set to 0 in the present application.

[0089] In some embodiments, the recursive formula (6) actually used for calculation can be obtained by discretizing the selective state transition equation and the observation equation using the zero-order hold (ZOH) or Euler method; (6); wherein, is obtained by modulating the continuous-time A matrix through and and then through a discretization operation, and ; is a dynamic vector that changes over time, which is generated in real time by a small neural network according to the input : , is a discretization step (a learnable parameter). Both depend on the state transition selective parameter generated by the node size, so as to realize a dynamic system that depends on the input.

[0090] For the t-th superpixel node, the Mamba state space model can calculate the discretization parameter at the current time according to the state transition selective parameter , update the state: , and then recombine the output sequence into a node feature matrix.

[0091] The Mamba state space model can realize differentiated extraction of different scale features by dynamically adjusting the state transition parameters A and B. In the shallow graph convolution-Mamba, the Mamba state space model processes high-resolution local features, i.e., small node sizes, and enhances the detail capture ability through a larger ; in the deep graph convolution-Mamba, the Mamba state space model processes low-resolution global features, i.e., large node sizes, and enhances the detail capture ability through a smaller The value of the focus space distribution can finally be automatically optimized according to the node size to extract features, and the recognition ability for small targets in dense areas and independent large targets can be significantly improved.

[0092] In some embodiments, the Mamba state space model learns a nonlinear mapping relationship between the reference node size and the reference dynamic weight parameter in the training stage respectively represent the dynamic weight parameter and the node size, so that the dynamic weight parameter corresponding to different node sizes can be adaptively generated through the mapping relationship between the reference node size and the reference dynamic scale weight (reflecting the importance of features). Among them, is a small network (such as MLP), which can be a feedforward neural network, and is trained synchronously in the process of training the Mamba-GNN network architecture. The loss function of the Mamba-GNN network architecture is represented by a classification loss, and in the process of direction propagation, the gradient is transmitted to , so as to adjust its parameters, so that the generated can help the Mamba-GNN network architecture to better fuse features, thereby improving the classification accuracy.

[0093] Need to be explained, and are inversely related. For small residential plots (such as dense farm house areas), is smaller, the value generated by the Mamba state space model is larger, so that the model pays more attention to local detailed features; for large residential plots (such as independent farmhouses), is larger, the value generated by the Mamba state space model is smaller, so that the model pays more attention to global spatial distribution features. This dynamic adjustment mechanism enables the model to adaptively allocate appropriate feature extraction weights for residential plots of different sizes, thereby improving the recognition ability of the model for residential plots.

[0094] In some embodiments, each graph convolution-Mamba layer focuses on processing features of a specific scale. The first graph convolution-Mamba layer may focus on pixel-level detailed information, and subsequent graph convolution-Mamba layers gradually focus on larger range spatial structures.

[0095] ​​Exemplarily, in the case that the Mamba-GNN network architecture includes three graph convolution-Mamba layers, the first graph convolution-Mamba layer performs fine-grained feature extraction, inputs the initial feature vector sequence, processes high-resolution local features by using the Mamba state space model, focuses on capturing small features such as edges and textures, uses convolution operations combined with the state update rule of the Mamba state space model to enhance local feature expression, and outputs a fine-grained feature map from the first graph convolution-Mamba layer, which contains rich local information; the second graph convolution-Mamba layer performs medium-scale feature extraction, inputs the fine-grained feature map output by the previous layer, further processes feature combinations in a larger range by using the Mamba state space model, and adjusts the state transition parameters of the Mamba state space model to make it more inclined to capture medium-scale patterns such as house outlines or small building groups, and outputs a medium-scale feature map from the second graph convolution-Mamba layer, which contains more macro structural information; the third graph convolution-Mamba layer performs coarse-grained feature extraction, inputs the medium-scale feature map output by the previous layer, further expands the receptive field by using the Mamba state space model to capture larger spatial layouts, and dynamically adjusts the state transition selectivity parameters and dynamic weight parameters of the Mamba state space model to better adapt to the identification needs of large-scale structures, and finally outputs a coarse-grained feature map reflecting the overall layout and large-scale structure of the entire region. In each graph convolution-Mamba layer, the parameters (state transition selectivity parameters and dynamic weight parameters) in the Mamba state space model are dynamically adjusted according to the feature scale (node size) of the current layer to ensure that the model can adaptively pay attention to information of different scales.

[0096] In some embodiments of the present application, the analysis and processing of the target fusion feature matrix in step S107 to obtain the binary mask image of the homestead can be realized through the following steps S1071 to S1072.

[0097] S1071, map each feature in the target fusion feature matrix to a corresponding homestead probability by using a fully connected layer to obtain a homestead probability map.

[0098] In some embodiments, the target fusion feature matrix is a In some embodiments, the target fusion feature matrix is a

[0099] S1072, sequentially performing morphological operation and connected region analysis processing on the homestead probability map to obtain a binary mask image.

[0100] In some embodiments, the homestead probability map generally has noise (isolated misjudgment pixels) and holes (parts of the homestead are misjudged as non-homestead), so the accuracy of the homestead probability map can be improved by morphological operation and connected region operation.

[0101] In some embodiments, small noise points (very small homestead misjudgment in non-homestead) can be removed by opening operation, and holes in the homestead can be filled by closing operation, so that the homestead area is more complete. Further, all connected regions (such as 4-connected or 8-connected) in the homestead probability map are identified, and then the area (number of pixels) of each connected region is calculated. A minimum area threshold (such as 10 pixels) can be set to filter very small regions. After morphological operation and connected region analysis, the processed homestead probability map needs to be converted into a binary image. A threshold (such as 0.5) can be set, and pixels with a probability greater than or equal to the threshold are regarded as homestead (1), and pixels with a probability less than the threshold are regarded as non-homestead (0), and finally a binary mask image of the homestead is obtained.

[0102] In addition, key precision indicators such as IoU (intersection over union) and small target recognition rate can also be calculated from the homestead probability map or the binary mask image, providing directly applicable vector results and performance verification data for precise extraction and quantitative evaluation of rural homestead.

[0103] Next, the application of the method provided by the present application in an actual scene will be described in conjunction with an embodiment.

[0104] A four-band remote sensing image (covering small single-family homestead, large homestead group, surrounding farmland, and road) with a resolution of 0.5 meters collected in a certain southern rural area A in China is selected. The image is divided into N (about 500-1000) superpixel regions using superpixel segmentation (such as SLIC); each superpixel is regarded as a graph node; if two superpixels are adjacent in space (share a boundary), they are connected in the adjacency matrix; the adjacency matrix E ∈ {0, 1}^{N×N} is a sparse symmetric matrix, and the Mamba-GNN network architecture is built.

[0105] The spectral feature vector is calculated from the four-band data (using the spectral difference between homestead and farmland), and the spatial feature vector of the neighborhood mean and gradient is obtained by 3x3 neighborhood statistics. The initial features of the graph nodes carrying the mean spectrum and spatial statistics of the pixels in the superpixel are generated by inputting the spectral and spatial features into the MLP.

[0106] Taking the initial features and the graph structure of the nodes as inputs, the Mamba state space model is used to obtain the living dynamic weight of the node according to the size of the homestead The Mamba state space model is embedded in each layer of the graph neural network, and small, medium and large scale features are extracted layer by layer, and then The multi-scale features are adaptively weighted and fused to obtain comprehensive homestead features.

[0107] Based on the multi-scale fusion features, the homestead and non-homestead are classified and predicted through a full connection layer, and then the small noise is removed through a morphological opening operation and the holes are filled through a morphological closing operation, and a binary mask image is output, and the IoU and small homestead recognition rate are calculated to evaluate the effect.

[0108] The identification method of the rural homestead remote sensing image provided by the application realizes the deep fusion of spectral features and spatial features, adaptively processes homesteads of different sizes through a dynamic scale perception mechanism, significantly improves the small target recognition rate in dense areas, and greatly improves the processing efficiency with a calculation complexity of only 1 / 3 of the Transformer. At the same time, the spectral feature vector and the spatial feature vector are linked to solve the spectral confusion problem between the homestead and similar ground objects (such as farmland), and provide high-precision and high-efficiency technical support for digital management of rural homesteads, and effectively promote fine management of land resources.

[0109] It should be noted that, according to the needs of implementation, each step described in the embodiments of the application can be split into more steps, or two or more steps or part of the operations of the steps can be combined into new steps to achieve the purpose of the embodiments of the application.

[0110] The above method according to the embodiments of the application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk or a magneto-optical disk, or downloaded through a network and stored in a remote recording medium or a non-transitory machine-readable medium and then stored in a local recording medium, so that the method described herein can be processed by such software on a recording medium using a general-purpose computer, a special-purpose processor or programmable or special-purpose hardware such as an ASIC or an FPGA. It can be understood that the computer, processor, microprocessor controller or programmable hardware includes a storage component (for example, RAM, ROM, flash memory, etc.) that can store or receive software or computer code, which, when accessed and executed by the computer, processor or hardware, implements the method described herein. In addition, when a general-purpose computer accesses the code for implementing the method shown herein, the execution of the code will convert the general-purpose computer into a special-purpose computer for executing the method shown herein. ​

[0111] Those skilled in the art can understand that the units and method steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software manner depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered as beyond the scope of the embodiments of the present application.

[0112] The above implementation is only used to illustrate the embodiments of the present application, and is not intended to limit the embodiments of the present application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions belong to the scope of the embodiments of the present application, and the patent protection scope of the embodiments of the present application should be defined by the claims.

Claims

1. A method for identifying rural homesteads from remote sensing images, characterized in that, include: Superpixel segmentation was performed on remote sensing images of rural homesteads to obtain multiple superpixel nodes; Each superpixel node includes multiple pixels in the remote sensing image of the rural homestead; Calculate the spatial distance between adjacent superpixel nodes and generate an initial adjacency matrix; The initial adjacency matrix is ​​a sparse symmetric matrix, and the dimension of the initial adjacency matrix is ​​the same as the total number of superpixel nodes; Based on the color information of multiple pixels in each superpixel node, construct the spectral feature vector of each superpixel node; Calculate the average pixel value of each superpixel node in a 3×3 neighborhood, and use the average pixel value as the first spatial feature of the corresponding superpixel node; The Sobel operator is used to perform a convolution operation on the pixel matrix corresponding to the remote sensing image of the rural homestead, and the second spatial feature of each superpixel node is calculated by combining the corresponding pixel of each superpixel node in the pixel matrix. The first spatial feature and the corresponding second spatial feature are concatenated to obtain the spatial feature vector corresponding to each superpixel node. The initial feature vector obtained by fusing the spectral feature vector and the corresponding spatial feature vector, along with the initial adjacency matrix, are input into the Mamba-GNN network architecture to obtain the target fused feature matrix. The Mamba-GNN network architecture includes multiple graph convolutional-Mamba layers connected in sequence, with each graph convolutional-Mamba layer embedding a Mamba state space model. The target fusion feature matrix is ​​analyzed and processed to obtain a binary mask image of the homestead.

2. The method according to claim 1, characterized in that, The color information includes the pixel values ​​in the R, G, B, and NIR bands; the construction of the spectral feature vector of each superpixel node based on the color information of multiple pixels in each superpixel node includes: Calculate the average values ​​of all pixels in the R-band, G-band, B-band, and NIR band for a specific superpixel node; the specific superpixel node can be any one of the plurality of superpixel nodes; The spectral feature vector of the specific superpixel node is constructed based on the average value; the spectral feature vector can be represented as... ,in, These represent the average values ​​of all pixels in the specific superpixel node in the R-band, G-band, B-band, and NIR band, respectively.

3. The method according to claim 1, characterized in that, The step of performing a convolution operation on the pixel matrix corresponding to the remote sensing image of the rural homestead using the Sobel operator, and calculating the second spatial feature of each superpixel node by combining the corresponding pixel points of each superpixel node in the pixel matrix, includes: The Sobel operator is used to perform a convolution operation on the pixel matrix corresponding to the remote sensing image of the rural homestead to obtain a first spatial feature matrix in the horizontal direction and a second spatial feature matrix in the vertical direction; the dimensions of the first spatial feature matrix and the second spatial feature matrix are the same as the dimensions of the pixel matrix. The first pixel feature of each superpixel node corresponding to its corresponding pixel is determined from the first spatial feature matrix, and the second pixel feature of each specific superpixel node corresponding to its corresponding pixel is determined from the second spatial feature matrix. The second spatial feature of each superpixel node is calculated based on the first pixel feature and the corresponding second pixel feature.

4. The method according to claim 1, characterized in that, The initial feature vector obtained by fusing the spectral feature vector and the corresponding spatial feature vector, along with the initial adjacency matrix, is input into the Mamba-GNN network architecture to obtain the target fused feature matrix, including: The spectral feature vector and the corresponding spatial feature vector are fused by a multilayer perceptron (MLP) to obtain the initial feature vector corresponding to each superpixel node. Based on the coordinates of the center pixel of each superpixel node, the initial feature vectors are sorted to obtain the initial feature vector sequence. The initial feature vector sequence is used to construct an initial feature matrix. This initial feature matrix and the initial adjacency matrix are then input into the first graph convolutional Mamba layer, and sequentially passed through the second to the Nth graph convolutional Mamba layer to obtain N node feature matrices and N dynamic weight parameters. Each graph convolutional Mamba layer generates a node feature matrix and corresponding dynamic weight parameters, and the input to the current graph convolutional Mamba layer includes the node feature matrix output from the previous graph convolutional Mamba layer and the initial adjacency matrix. N is the total number of graph convolutional Mamba layers. The target fused feature matrix is ​​obtained by weighting and fusing the N node feature matrices using the N dynamic weight parameters.

5. The method according to claim 4, characterized in that, Each graph convolutional-Mamba layer includes a graph convolutional module and a Mamba state-space model. The process of generating the node feature matrix and corresponding dynamic weight parameters in each graph convolutional-Mamba layer includes: The graph convolution module is used to perform convolution operations on a specific feature matrix, the initial adjacency matrix, and the weight matrix to obtain a convolutional feature matrix; the specific feature matrix is ​​the initial feature matrix or the node feature matrix output by the previous graph convolution-Mamba layer; The Mamba state space model is used to dynamically extract features from the convolutional feature matrix and the node size of each superpixel node to generate the node feature matrix; and the dynamic weight parameters corresponding to the node feature matrix are determined based on the node size and the nonlinear mapping relationship between the reference node size and the reference dynamic weight parameters constructed during the training of the Mamba-GNN network architecture. The node size of the superpixel node is determined based on the neighboring pixels in the initial feature vector sequence; the reference node size is inversely correlated with the reference dynamic weight parameter.

6. The method according to claim 5, characterized in that, The step of dynamically extracting features from the convolutional feature matrix and the node size of each superpixel node using the Mamba state-space model to generate the node feature matrix includes: The node size is input into the gated neural network of the Mamba state space model to obtain the state transition selectivity parameters corresponding to the superpixel node. The node feature matrix is ​​generated based on the state transition selectivity parameter, the convolutional feature matrix, and the selective state transition equation and observation equation provided by the Mamba state space model.

7. The method according to claim 6, wherein the selective state transition equation can be expressed as: ； in, To update the step size, These represent the accumulated context information of the first t superpixel nodes and the new context information formed after updating the accumulated context information of the first t superpixel nodes, respectively. This represents the state transition selectivity parameter corresponding to the t-th superpixel node. These represent the state transition matrix and the input matrix, respectively.

8. The method according to any one of claims 1 to 7, characterized in that, The step of analyzing and processing the target fusion feature matrix to obtain a binary mask image of the homestead includes: A fully connected layer is used to map each feature in the target fusion feature matrix to the corresponding homestead probability, resulting in a homestead probability map; The probability map of the homestead is processed sequentially with morphological operations and connected component analysis to obtain the binary mask image.

Images