Forage dominance identification method based on convolutional neural network

Abstract

The invention relates to the technical field of agricultural remote sensing and artificial intelligence, and discloses a pasture dominance identification method based on a convolutional neural network, and the method comprises the steps: obtaining synchronous multi-modal data of a pasture region to be identified; performing feature extraction and fusion on the synchronous multi-modal data to obtain a fusion feature vector; segmenting the high-resolution RGB image into a plurality of sub-regions, abstracting the sub-regions as nodes, and combining with the fusion feature vector to construct a heterogeneous spatial relation graph; reasoning the heterogeneous spatial relation graph by using a graph attention network guided by structure prior, and outputting a category label for each node; and finally, according to the category label of each node and the corresponding sub-region area, calculating the dominance degree of different types of pasture. By fusing multi-source data and utilizing the graph attention network guided by the structure prior, the problems that a single data source is insufficient in feature and the spatial structure is neglected are solved, and high-precision quantitative evaluation of the dominance degree of pasture is realized.

Description

Technical Field

[0001] This invention relates to the fields of agricultural remote sensing and artificial intelligence technology, specifically a method for identifying forage dominance based on convolutional neural networks. Background Technology

[0002] Accurate identification of forage dominance is a key step in achieving precision livestock management and assessing pasture productivity. With the development of remote sensing technology and artificial intelligence, automated analysis of forage growth using image data acquired by drones or satellites has become an important technical means.

[0003] In existing technologies, convolutional neural network-based methods are commonly used to process visible light or hyperspectral images acquired by drones. Image recognition and segmentation techniques are then used to qualitatively identify the distribution areas of different forage grasses, thereby providing a preliminary assessment of their growth and species composition.

[0004] However, relying solely on single optical or spectral data has limitations in distinguishing forage grasses with similar growth states or spectral confusion. Furthermore, traditional convolutional neural networks, when performing classification, typically analyze the features of pixels or image regions in isolation, failing to effectively utilize the physical structure information and spatial proximity relationships of forage grasses in three-dimensional space. This results in insufficient recognition accuracy and makes it difficult to further transform qualitative region identification into a precise quantitative assessment of forage grass dominance.

[0005] Therefore, this invention proposes a forage dominance identification method based on convolutional neural networks to address the shortcomings of existing technologies. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a method for identifying forage dominance based on convolutional neural networks. This method mainly solves the problem that traditional technologies rely solely on single remote sensing data and ignore the spatial structural relationships of forage, resulting in insufficient accuracy in forage species identification and difficulty in achieving precise quantitative assessment of dominance.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for identifying forage dominance based on convolutional neural networks, comprising the following steps:

[0008] S1. Acquire synchronous multimodal data of the pasture area to be identified, wherein the synchronous multimodal data includes high-resolution RGB images, hyperspectral images, and lidar point cloud data;

[0009] S2. Perform feature extraction and fusion on the synchronous multimodal data to obtain the fused feature vector for each spatial location;

[0010] S3. Based on the preset segmentation algorithm and node type definition rules, the high-resolution RGB image is constructed into a heterogeneous spatial relationship graph, wherein the nodes of the heterogeneous spatial relationship graph correspond to the segmented image sub-regions, and each node is assigned an initial feature vector and node type according to the fused feature vector;

[0011] S4. Using a graph attention network guided by structural priors, reason about the heterogeneous spatial relationship graph and output a final category label for each node;

[0012] S5. Calculate the forage dominance of different types of forage based on the final category labels of all nodes and the area of ​​the image sub-regions corresponding to all nodes.

[0013] Preferably, in step S2, the step of extracting and fusing features from the synchronous multimodal data to obtain a fused feature vector for each spatial location includes:

[0014] The high-resolution RGB image is processed using a first convolutional neural network to extract visual features at each spatial location;

[0015] The hyperspectral image is processed using a second convolutional neural network to extract spectral features at each spatial location;

[0016] The lidar point cloud data is processed using a point cloud processing network to extract the three-dimensional structural features of each spatial location.

[0017] The visual features, spectral features, and three-dimensional structural features at the same spatial location are spliced ​​together along the feature dimension to generate a fused feature vector.

[0018] Preferably, in step S3, the step of constructing a heterogeneous spatial relationship graph from the high-resolution RGB image based on a preset segmentation algorithm and node type definition rules includes:

[0019] The high-resolution RGB image is divided into several image sub-regions using a superpixel segmentation algorithm.

[0020] Each of the image sub-regions is identified as a node, and the average height and average normalized vegetation index of each of the image sub-regions are calculated. A node type is assigned to each node according to a preset threshold, thereby forming a heterogeneous node set.

[0021] Based on the spatial adjacency relationship between the image sub-regions, edges are established between the corresponding heterogeneous nodes to construct a heterogeneous spatial relationship graph.

[0022] Preferably, in step S4, the step of using a structure-prior-guided graph attention network to reason about the heterogeneous spatial relationship graph and outputting a final category label for each node includes:

[0023] The feature vector of each node is iteratively updated through a network structure containing multiple attention layers; wherein, in each layer, the updated feature vector of any target node is formed by weighted aggregation of the feature vectors of the target node's neighboring nodes, and the weights of the weighted aggregation are the attention coefficients calculated through the attention mechanism guided by structural priors.

[0024] After all attention layers have completed inference, the final feature vector of each node is input into a classifier to output the final class label.

[0025] Preferably, the structure-prior-guided attention mechanism is used to calculate the attention coefficient between any target node and any of its neighboring nodes, and the attention mechanism specifically includes:

[0026] Calculate the feature similarity attention score between the target node and its neighboring nodes;

[0027] Extract the structural prior relationship between the target node and the neighboring nodes, and calculate the structural modulation factor based on the structural prior relationship;

[0028] The feature similarity attention score is combined with the structure modulation factor to generate the final attention score, and the final attention score is normalized to obtain the final attention coefficient.

[0029] Preferably, the final attention score The calculation process satisfies the following formula:

[0030]

[0031] In the formula, l is the network layer index; i is the index of the target node; and j is the index of the neighboring node. Attention score for feature similarity; This is the structure modulation factor.

[0032] Preferably, the structural prior relationship is characterized by the average height difference and spatial distance between the image sub-regions corresponding to the target node and the neighboring nodes.

[0033] Preferably, during inference, the graph attention network guided by the prior structure performs feature transformation on nodes of different node types using a linear transformation matrix associated with that type. The feature transformation process satisfies the following formula:

[0034]

[0035] In the formula, For node v i Features after transformation at layer l; The input features for the feature transformation; φ(v) i ) is node v i The node type; For node type φ(v) i The associated learnable weight matrix.

[0036] Preferably, the updated feature vector of the target node is obtained by summing the transformed features of each neighbor node of the target node by the corresponding attention coefficient, and then processing it through a nonlinear activation function.

[0037] Preferably, in step S5, the step of calculating the forage dominance of different types of forage grasses based on the category labels of all nodes and the area of ​​the image sub-regions corresponding to all nodes includes:

[0038] For the k-th type of forage grass, count all nodes whose final category label is the k-th type of forage grass, and sum the areas of the image sub-regions corresponding to the nodes to obtain the total coverage area A of the k-th type of forage grass. k ;

[0039] The total vegetation cover area A is obtained by summing the areas of the image sub-regions corresponding to the nodes and counting all nodes whose final category labels belong to any vegetation category. veg ;

[0040] The total coverage area A k With the total vegetation coverage area A veg The ratio of the two grasses is used as the forage dominance of the kth grass.

[0041] This invention provides a method for identifying forage dominance based on convolutional neural networks. It has the following beneficial effects:

[0042] 1. This invention extracts and fuses features from high-resolution RGB images, hyperspectral images, and lidar point cloud data, and constructs a heterogeneous spatial relationship graph using the fused features. This solves the problem of insufficient feature dimensions and inability to accurately distinguish forage grasses in different growth states caused by relying on a single data source in existing technologies. It provides a multi-dimensional feature foundation for subsequent analysis based on convolutional neural networks, thereby enabling accurate identification of forage grass dominance.

[0043] 2. This invention utilizes a graph attention network guided by structural priors to combine the feature similarity between nodes with structural prior relationships such as average height difference and spatial distance during the reasoning process. This solves the problem that traditional graph networks ignore physical spatial constraints and easily confuse regions with similar features but different physical attributes. As a result, the classification decision based on convolutional neural networks is more in line with the growth pattern of forage grass, thereby improving the accuracy of forage grass dominance identification.

[0044] 3. This invention constructs an image into a set of nodes by using a superpixel segmentation algorithm and transforms the recognition task into reasoning about the node categories. This solves the problems of large computational load and sensitivity to local noise in traditional pixel-level classification. By statistically analyzing the area of ​​the image sub-regions corresponding to the final classified nodes, it provides an objective basis for quantitatively calculating the dominance of different types of forage grasses and completes the process from qualitative recognition to quantitative analysis based on convolutional neural networks. Attached Figure Description

[0045] Figure 1 This is a flowchart of the forage dominance identification method based on convolutional neural networks of the present invention;

[0046] Figure 2 This is a flowchart of step S1 of the present invention;

[0047] Figure 3 This is a flowchart of step S2 of the present invention;

[0048] Figure 4 This is a flowchart of step S3 of the present invention;

[0049] Figure 5 This is a flowchart of step S4 of the present invention;

[0050] Figure 6 This is a flowchart of step S5 of the present invention. Detailed Implementation

[0051] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] Please see the appendix Figures 1-6 This invention provides a method for identifying forage dominance based on convolutional neural networks, comprising the following steps:

[0053] First, perform step S1: acquire synchronous multimodal data of the pasture area to be identified. The synchronous multimodal data includes high-resolution RGB images, hyperspectral images, and LiDAR point cloud data.

[0054] Then, step S2 is performed: feature extraction and fusion are performed on the synchronous multimodal data to obtain the fused feature vector for each spatial location;

[0055] Next, step S3 is executed: based on the preset segmentation algorithm and node type definition rules, the high-resolution RGB image is constructed into a heterogeneous spatial relationship graph, where the nodes of the heterogeneous spatial relationship graph correspond to the segmented image sub-regions, and each node is assigned an initial feature vector and node type according to the fused feature vector;

[0056] Then, step S4 is executed: using a graph attention network guided by structural priors to reason about the heterogeneous spatial relationship graph and output a final category label for each node;

[0057] Finally, perform step S5: Calculate the forage dominance of different types of forage based on the final category labels of all nodes and the area of ​​the corresponding image sub-regions of all nodes.

[0058] See attached document Figure 1 and Figure 2 In one specific embodiment, the data acquisition process for identifying pasture dominance in step S1 is completed on an unmanned aerial vehicle (UAV)-based data acquisition platform. The platform integrates a high-resolution RGB camera, a hyperspectral imager, and a lidar sensor. Following a preset flight path, the platform performs low-altitude scanning of the pasture area to be identified, simultaneously acquiring three modalities of data for the area.

[0059] The high-resolution RGB image was captured by the platform's onboard high-resolution RGB camera. This high-resolution RGB image is a digital image containing three channels: red, green, and blue. It provides rich visual texture and spatial structure information for subsequent convolutional neural networks, forming the basis for analyzing the macroscopic morphology of pasture communities.

[0060] The hyperspectral image was acquired by the hyperspectral imager onboard the platform. This hyperspectral image is a data cube, in which reflectance information of hundreds of continuous, narrow-band spectral channels ranging from visible light to near-infrared wavelengths is recorded at each pixel in the spatial dimension. These detailed spectral features are the core basis for accurately distinguishing different types of forage grasses.

[0061] The lidar point cloud data is collected by lidar sensors mounted on the platform. This lidar point cloud data is a set of points returned after laser pulses scan the ground, containing massive amounts of three-dimensional spatial coordinates (X, Y, Z) and reflection intensity information. This data directly provides accurate elevation and three-dimensional structural information of the pasture canopy, which is crucial for analyzing the vertical distribution of vegetation.

[0062] To achieve spatial consistency of synchronized multimodal data, the raw data needs to be preprocessed. This preprocessing includes geometric correction and spatial registration. Using inertial measurement unit (IMU) and global positioning system (GPS) data carried by the UAV platform, orthorectification and georegistration are first performed on the high-resolution RGB images, hyperspectral images, and lidar point cloud data, respectively, unifying them into the same geographic coordinate system.

[0063] Subsequently, using the orthorectified high-resolution RGB image as a reference, the hyperspectral image and LiDAR point cloud data were registered to the same spatial resolution and pixel grid as the RGB image through methods such as resampling. This step ensures strict alignment of the three data types at any spatial location, guaranteeing the effectiveness of subsequent feature fusion.

[0064] In addition, the preprocessing includes radiometric calibration and atmospheric correction of the hyperspectral images to eliminate the influence of sensor errors and atmospheric conditions on spectral information, obtaining accurate surface reflectance data for ground features. Simultaneously, the lidar point cloud data is filtered to remove noise points and outliers. Finally, a spatially synchronized, high-quality synchronous multimodal dataset is output, laying the foundation for subsequent feature extraction and pasture identification.

[0065] See attached document Figure 1 and Figure 3 After acquiring synchronous multimodal data, the method of the present invention further performs step S2 to generate deep fusion features for subsequent forage identification.

[0066] Specifically, for the preprocessed high-resolution RGB image, a first convolutional neural network is used to extract visual features at each spatial location. In one embodiment, the first convolutional neural network is a deep residual network (ResNet) pre-trained on a large natural image dataset. The network abstracts the color, texture, and local shape information of the image layer by layer through multiple convolution, pooling, and non-linear activation operations, and finally extracts high-dimensional visual features for each spatial location in the image on the deep feature map of the network.

[0067] For the preprocessed hyperspectral images, a second convolutional neural network (CNN) is used to extract spectral features at each spatial location. Given that hyperspectral data contains both spatial and spectral dimensions, the second CNN can be a three-dimensional convolutional neural network (3D-CNN). The 3D-CNN utilizes three-dimensional convolutional kernels to extract features in the joint spectral-spatial domain, thereby capturing the fine spectral absorption and reflectance patterns that distinguish different types of forage grasses. After processing, the 3D-CNN extracts spectral features for each spatial location that characterize the differences in vegetation biochemical components.

[0068] For the preprocessed lidar point cloud data, a point cloud processing network is employed to extract the 3D structural features of each spatial location. This network can be a PointNet++ network. First, the point cloud data is divided into a 2D grid aligned with image pixels. Then, the point set within each grid cell is hierarchically and abstractly processed to effectively learn the local geometric structure of the point set. Finally, the network extracts 3D structural features representing information such as the average elevation, vegetation density, and canopy undulation of the local area for each spatial location.

[0069] Since the aforementioned data preprocessing steps have ensured that the data from the three modalities are strictly aligned spatially, corresponding features extracted by the three independent networks can be obtained simultaneously for any geographic location within the pasture area. Subsequently, the visual features, spectral features, and 3D structural features of the same spatial location are concatenated along the feature dimension to generate a unified fused feature vector. This process can be represented by the following formula:

[0070]

[0071] In the formula, F fused (x,y) represents the fused feature vector at spatial location (x,y); F rgb (x, y represent the feature vectors corresponding to the visual features at spatial location (x, y); F hsi (x,y) represents the eigenvector corresponding to the spectral features at the spatial location (x,y); F lidar (x,y) represents the feature vector corresponding to the three-dimensional structural features at the spatial location (x,y); The symbol represents the vector concatenation operation.

[0072] In this way, a comprehensive and information-rich feature representation is generated for each spatial location in the pasture area, providing a foundation for constructing graph structures and achieving high-precision pasture dominance analysis.

[0073] See attached document Figure 1 and Figure 4 After generating the fused feature vector for each spatial location, the method of the present invention transforms the pixel-level feature representation into a more structured graph representation. This process corresponds to step S3, which lays the foundation for subsequent graph-based forage recognition.

[0074] In one specific embodiment, step S3 first performs image segmentation on the high-resolution RGB image. Image segmentation is achieved using a preset segmentation algorithm, which is a superpixel segmentation algorithm, such as Simple Linear Iterative Clustering (SLIC). This algorithm segments the high-resolution RGB image into several image sub-regions that are compact in color and space.

[0075] After segmentation, each image sub-region is identified as a node. Next, to form a heterogeneous node set, each node needs to be assigned a node type. This assignment process is based on the node's physical properties, specifically including: calculating the average height and average normalized vegetation index of the image sub-region corresponding to each node, and assigning a node type according to a preset threshold.

[0076] For any node v i and its corresponding image sub-region S i Its average height H i The calculation method is as follows:

[0077] Filter out all spatial locations that fall into image sub-region S i The set of lidar point clouds P within the range i ={p1,p2,...,p k};

[0078] Extract the height value z(p) of each point in the point cloud set. j ), and calculate the arithmetic mean of these height values. This process can be expressed by the following formula:

[0079]

[0080] In the formula, k is the point cloud set P i The total number of points in z(p); j ) represents the height value of the j-th point.

[0081] For the same node v i and its corresponding image sub-region S i Its average normalized vegetation index N i The calculation method is as follows:

[0082] Using the reflectance of the near-infrared and red bands in hyperspectral data, the sub-region S is calculated. i The Normalized Difference Vegetation Index (NDVI)(x,y) for each pixel location (x,y) is calculated using the following formula:

[0083]

[0084] In the formula, ρ NIR (x,y) and ρ R (x,y) represent the reflectance values ​​of the near-infrared band and the red band at pixel position (x,y), respectively.

[0085] Pair of sub-image sub-regions S i The average normalized vegetation index N is obtained by averaging the NDVI values ​​of all pixels within the area. i .

[0086] After calculating H for each node i and N i After setting the value, a node type φ(v) is assigned to each node based on the preset height threshold and vegetation index threshold. i ).

[0087] In this embodiment, the preset threshold is specifically described: height threshold T h1 Set to 0.5 meters to distinguish between regular pasture grasses and taller shrubs or weeds; height threshold T h2 The threshold is set to 0.1 meters to distinguish between pasture with a certain canopy height and surface or very sparse vegetation; the normalized vegetation index threshold T... n The value is set to 0.2, which is an empirical value commonly used to distinguish between vegetation and non-vegetation (such as bare soil and withered grass). Based on these precise thresholds, the node type definition rules are as follows:

[0088] If H i >0.5 and N i If the value is greater than 0.2, then the node type φ(v) i ) is defined as tall vegetation;

[0089] If 0.1 <H i ≤0.5 and N i If the value is greater than 0.2, then the node type φ(v) i ) is defined as low vegetation;

[0090] If N i If ≤0.2, then the node type φ(v) i () is defined as non-vegetation.

[0091] Because this rule allows nodes to be assigned different types, all nodes together form a heterogeneous set of nodes that can reflect the vertical structure and vegetation cover heterogeneity of pasture areas.

[0092] Simultaneously, based on the fused feature vector obtained in step S2, an initial feature vector is assigned to each node. For node v i Its initial eigenvector By its corresponding image sub-region S i The fused feature vector F of all spatial locations within the area fused It is formed by aggregating 9x,y). One specific aggregation method is to take the average value:

[0093]

[0094] In the formula, |S i |Represents subregion S i The total number of spatial locations included.

[0095] Finally, based on the spatial adjacency relationships between image sub-regions, edges are established between corresponding heterogeneous nodes. If any two image sub-regions S... i and S j If they share a boundary, then in their corresponding node v i and v j An edge is established between each element. After all edges are established, a complete heterogeneous spatial relationship graph is constructed. This graph structure provides the foundation for subsequent reasoning using graph attention networks, and ultimately for accurate identification and dominance analysis of forage species.

[0096] See attached document Figure 1 and Figure 5 After constructing the heterogeneous spatial relationship graph, the method of the present invention further executes step S4, which uses a graph attention network guided by structural prior to reason about the heterogeneous spatial relationship graph, in order to aggregate neighborhood information and update the feature representation of the nodes, and finally outputs a final category label for each node, thereby providing a basis for the identification of pasture dominance.

[0097] In one specific embodiment, the process of reasoning about heterogeneous spatial relationship graphs using a structure-prior-guided graph attention network is achieved by iteratively updating the feature vector of each node in the graph through a network structure containing multiple attention layers. After all attention layers have completed their reasoning, the final feature vector of each node is input into a classifier to output the final class label.

[0098] Specifically, in the l-th layer of the graph attention network, for any target node v in the graph... i Its characteristics are from Updated to The process is explained in detail below:

[0099] Node type-related feature transformations.

[0100] Because the graph contains heterogeneous nodes, in order to capture the unique feature patterns of different types of nodes (e.g., tall vegetation, short vegetation, and no vegetation as defined above), the network performs feature transformation on nodes of different types during inference using independent, learnable linear transformation matrices associated with that type. For node v i The feature transformation process satisfies the following formula:

[0101]

[0102] In the formula, l is the network layer index; i is the index of the target node; For node v i The input features of the feature transformation at layer l; φ(v i ) is node v iNode type; W φ (v i ) (l) For node type φ(v) i The associated learnable weight matrix; For node v i Features after transformation at layer l.

[0103] Calculation of attention coefficients guided by structural priors.

[0104] Next, the network computes the value of any target node v through a structure-prior-guided attention mechanism. i With any of its neighboring nodes v j The attention coefficient between them. The process is as follows:

[0105] Calculate the feature similarity attention score between the target node and its neighboring nodes. This step aims to measure the relevance between two nodes at the feature level, and its calculation formula is as follows:

[0106]

[0107] In the formula, j is the index of the neighboring node; Attention score for feature similarity; Let be a learnable attention weight vector in the l-th layer; This indicates a vector concatenation operation.

[0108] Then, the structural prior relationship between the target node and its neighboring nodes is extracted, and the structural modulation factor is calculated based on this relationship. In this embodiment, the structural prior relationship is characterized by the average height difference and spatial distance between their corresponding image sub-regions. The average height difference ΔH ij =|H i -H j | Reflects the differences in the vertical structure of vegetation, spatial distance D ij =||c i -c j ||2 reflects their geographical proximity, where c i and c j These are the centroid coordinates of the two sub-regions. These two structural prior relations are input into a multilayer perceptron (MLP) network to adaptively calculate the structure modulation factor.

[0109] Next, the feature similarity attention score is combined with the structure modulation factor to generate an attention score. In this embodiment, the two are combined element-wise by multiplication, and the calculation process satisfies the following formula:

[0110]

[0111] In the formula, l is the network layer index; i is the index of the target node; and j is the index of the neighboring node. Attention score for feature similarity; It is the structure modulation factor; A score is given for attention.

[0112] Finally, the Softmax function is used to evaluate the target node v. i Normalize the final attention scores of all its neighboring nodes to obtain the final attention coefficient:

[0113]

[0114] In the formula, This represents the final attention coefficient. Representative node v i The set of all neighboring nodes.

[0115] Iterative update of node feature vectors.

[0116] After obtaining the final attention coefficients, the updated feature vector of the target node is formed by a weighted aggregation of the transformed features of its neighboring nodes. Specifically, this update process involves multiplying the transformed features of each neighboring node of the target node by its corresponding attention coefficient, summing the results, and finally processing them through a non-linear activation function (e.g., ELU) σ.

[0117]

[0118] In the formula, For node v i The updated feature vector at layer l.

[0119] The final output of the category labels.

[0120] By repeating steps 1 to 3 above, i.e., stacking multiple attention layers, the network can capture a wider range of neighborhood structure information, thereby learning more discriminative node features. After all attention layers (assuming a total of L layers) have completed inference, the final feature vector of each node is... The input is fed into a classifier (e.g., a fully connected layer followed by a softmax function) to output the probability that the node belongs to each predefined pasture category. The probability with the highest value is the final category label L(v) for that node. i In this way, the method of the present invention can assign an accurate category to each sub-region within the pasture area, providing reliable technical support for achieving high-precision pasture dominance analysis.

[0121] See attached document Figure 1 and Figure 6After assigning a final category label to each node using a graph attention network, the method of this invention executes step S5 to quantify the dominance of different types of forage grasses. This step is a crucial step in achieving accurate assessment of forage growth and identification of dominance.

[0122] In one specific embodiment, the calculation process is based on the category labels of all nodes and the area of ​​the image sub-regions corresponding to all nodes. The area of ​​each image sub-region can be calculated by counting the total number of pixels it contains and combining this with the ground sampling distance (GSD) of the image. The specific calculation steps are as follows:

[0123] Calculate the total coverage area of ​​a specific type of forage grass.

[0124] For the k-th type of forage grass (e.g., white clover or perennial ryegrass), firstly, among all nodes in the heterogeneous spatial relationship graph, count all nodes whose final category label is the k-th type of forage grass. Then, sum the areas of the image sub-regions corresponding to each of these nodes to obtain the total coverage area A of the k-th type of forage grass. k This process can be represented by the following formula:

[0125]

[0126] In the formula, k represents a specific type of forage grass; A k Let V be the total coverage area of ​​the k-th type of forage grass; V is the set of all nodes in the heterogeneous spatial relationship graph; v i Let L(v) be a node in set V; i ) is node v i The final category label obtained through reasoning in step S4; Area(v i For node v i The area of ​​the corresponding image sub-region.

[0127] Calculate the total vegetation cover area.

[0128] Next, all nodes whose final category label belongs to any vegetation category are counted. This includes all nodes identified as specific forage species, but excludes nodes identified as "non-vegetation". The areas of the image sub-regions of all these vegetation nodes are summed to obtain the total vegetation cover area A. veg This process can be represented by the following formula:

[0129]

[0130] In the formula, A veg C represents the total vegetation cover area. vegThis represents a set of all category labels belonging to vegetation (for example, the set includes labels such as "white clover" and "perennial ryegrass", but does not include the "non-vegetation" label).

[0131] Calculate the dominance of pasture.

[0132] Finally, the ratio of the total cover area of ​​a specific type of forage grass to the total vegetation cover area is taken as the dominance of that forage grass. For the k-th forage grass, its dominance D is... k The calculation formula is:

[0133]

[0134] In the formula, D k Let represent the forage dominance of the k-th forage species.

[0135] Through the above steps, a clear dominance index can be calculated for each type of forage grass in a mixed forage planting area. This index intuitively reflects the proportion of different forage grasses in the total vegetation, providing reliable data support for ranchers to conduct scientific forage management, reseeding decisions, and assess ranch productivity. It is the final result of this invention's use of deep learning technology, especially convolutional neural networks, to achieve high-precision forage dominance analysis.

[0136] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1.A method for identifying forage dominance based on a convolutional neural network, characterized in that, The method comprises the following steps: S1, acquiring synchronous multi-modal data of a to-be-identified pasture area, wherein the synchronous multi-modal data comprises a high-resolution RGB image, a hyperspectral image, and laser radar point cloud data; S2, performing feature extraction and fusion on the synchronous multi-modal data to obtain a fusion feature vector of each spatial position; S3, constructing the high-resolution RGB image into a heterogeneous spatial relation graph based on a preset segmentation algorithm and node type definition rule, wherein the nodes of the heterogeneous spatial relation graph correspond to segmented image sub-regions, and each node is assigned an initial feature vector and a node type according to the fusion feature vector; S4, using a structure-prior-guided graph attention network to reason the heterogeneous spatial relation graph to output a final class label for each node; S5, calculating the pasture dominance of different types of pasture according to the final class labels of all nodes and the areas of the image sub-regions corresponding to all nodes. 2.The method according to claim 1, wherein, In step S2, the step of performing feature extraction and fusion on the synchronous multi-modal data to obtain a fusion feature vector of each spatial position comprises: using a first convolutional neural network to process the high-resolution RGB image to extract visual features of each spatial position; using a second convolutional neural network to process the hyperspectral image to extract spectral features of each spatial position; using a point cloud processing network to process the laser radar point cloud data to extract three-dimensional structure features of each spatial position; splicing the visual features, spectral features, and three-dimensional structure features of the same spatial position in the feature dimension to generate a fusion feature vector. 3.The method according to claim 1, wherein, In step S3, the step of constructing the high-resolution RGB image into a heterogeneous spatial relation graph based on a preset segmentation algorithm and node type definition rule comprises: using a superpixel segmentation algorithm to segment the high-resolution RGB image into a plurality of image sub-regions; determining each image sub-region as a node, calculating the average height and average normalized vegetation index of each image sub-region, and assigning a node type to each node according to a preset threshold to form a heterogeneous node set; establishing edges between corresponding heterogeneous nodes according to the spatial adjacency relationship between the image sub-regions to construct a heterogeneous spatial relation graph. 4.The method according to claim 1, wherein, In step S4, the step of using a structure-prior-guided graph attention network to reason the heterogeneous spatial relation graph to output a final class label for each node comprises: iteratively updating the feature vector of each node through a network structure comprising multiple attention layers; wherein in each layer, the updated feature vector of any target node is formed by weighted aggregation of the feature vectors of the neighbor nodes of the target node, and the weight of the weighted aggregation is an attention coefficient calculated by a structure-prior-guided attention mechanism; after all the attention layers are reasoned, inputting the final feature vector of each node into a classifier to output a final class label. 5.The method of claim 4, wherein, The structure-prior guided attention mechanism is used to calculate an attention coefficient between any target node and any neighbor node of the target node, and specifically includes: calculating a feature similarity attention score between the target node and the neighbor node; extracting a structure-prior relationship between the target node and the neighbor node, and calculating a structure modulation factor based on the structure-prior relationship; combining the feature similarity attention score and the structure modulation factor to generate a final attention score, and normalizing the final attention score to obtain a final attention coefficient. 6.The method of claim 5, wherein, the final attention score The calculation process of the final attention score satisfies the following formula: In the formula, l is the network layer index; i is the index of the target node; j is the index of the neighbor node; is a feature similarity attention score; is a structure modulation factor. 7.The method according to claim 5, wherein, The structure-prior relationship is characterized by an average height difference and a spatial distance between image sub-regions corresponding to the target node and the neighbor node. 8.The method according to claim 4, wherein, In the inference process of the structure-prior guided graph attention network, a linear transformation matrix associated with the type of different node types is used for feature transformation of the nodes, and the feature transformation process satisfies the following formula: wherein is a node v i transformed features at the l-th layer; is an input feature for the feature transformation φ(v i ) is a node type of node v i ; is a learnable weight matrix associated with the node type φ(v i ). 9.The method of claim 4, wherein, The updated feature vector of the target node is obtained by summing the transformed features of each neighbor node of the target node multiplied by the corresponding attention coefficient and then passing through a nonlinear activation function. 10.The method of claim 1, wherein, In step S5, the step of calculating the forage dominance of different types of forage according to the class labels of all nodes and the areas of the image sub-regions corresponding to all nodes includes: For the kth kind of pasture, all nodes with the final class label of the kth kind of pasture are counted, and the image sub-region area corresponding to the nodes is accumulated to obtain the total coverage area A of the kth kind of pasture k ; The statistical final class label belongs to all nodes of any vegetation class, and the image sub-area area corresponding to the node is accumulated to obtain the total vegetation coverage area A veg ; The total coverage area A k The ratio of the total vegetation coverage area A veg to the total coverage area A as the forage dominance of the kth forage.

Images