Grassland ecological monitoring method and system based on image recognition

Abstract

The invention relates to the technical field of ecological monitoring, and particularly discloses a grassland ecological monitoring method and system based on image recognition, and the method comprises the steps: obtaining a high-resolution RGB image of a grassland region through an unmanned aerial vehicle, carrying out the geometric correction and geographical registration of the image, and building the mapping correlation between image pixels and an actual spatial position; and performing block cutting processing and semantic segmentation on the image while reserving the geographic metadata so as to separate each independent plant individual in the image, and generating a plant instance ROI image set. Then, multi-scale features of a plant are further extracted to consider global morphology and local texture of the plant, and context perception enhancement in multiple directions is carried out on multi-scale feature representation of a plant image, so that discriminability of spatial structure features such as leaf edges and pulse sequence directions is enhanced; and therefore, accurate identification and positioning of the invasion species can be realized. According to the method, the efficiency of ecological monitoring of the grassland can be effectively improved, and early warning and accurate tracking of invasive species are realized.

Description

Technical Field

[0001] The present application relates to the field of ecological monitoring technology, and more specifically, to a grassland ecological monitoring method and system based on image recognition. Background Art

[0002] As an important terrestrial ecosystem, grasslands are not only a key barrier to maintaining regional ecological balance, regulating climate, and conserving water resources, but also the foundation for animal husbandry development and biodiversity protection. However, against the backdrop of global climate change and intensified human activities, grasslands face multiple threats such as degradation, desertification, and biological invasions. The spread of invasive species, in particular, has severely damaged the original ecological structure and functions, posing a severe challenge to agricultural and animal husbandry production and ecological security. Traditional grassland ecological monitoring relies mainly on manual field surveys. This method is not only time-consuming, labor-intensive, and costly, but also has a limited monitoring range, making it difficult to achieve large-scale, high-frequency dynamic monitoring, and even more unable to provide timely warnings and accurately locate the early expansion of invasive species.

[0003] In recent years, grassland ecological monitoring methods based on image recognition technology have provided new solutions for grassland ecological monitoring due to their advantages of non-contact, high efficiency, and large-scale coverage. Currently, existing solutions mostly use satellite remote sensing or drone image data. For example, some studies analyze macro-indicators such as vegetation indices (such as NDVI) to assess the overall growth and coverage of grasslands, or combine traditional machine learning methods to classify remote sensing images to distinguish different vegetation types. However, these methods have significant shortcomings in terms of refined monitoring: First, the spatial resolution of traditional remote sensing imagery is often insufficient for individual plant-level identification, especially for early-stage, small-scale invasive species identification. Second, even when using high-resolution drone imagery, existing solutions often rely on a direct end-to-end classification architecture. While this improves computational efficiency, it lacks the ability to accurately locate invasive species because it ignores the geospatial information of individual plants. More importantly, when processing grassland vegetation images, traditional convolutional neural networks are limited by their local receptive field characteristics, and lack directional sensitivity in their perception of plant edge texture and microstructural features, which in turn affects the ability to finely identify invasive species and makes it prone to missed detections and misjudgments.

[0004] Therefore, an optimized grassland ecological monitoring method and system based on image recognition is expected. Summary of the Invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a grassland ecological monitoring method and system based on image recognition, which uses a drone to obtain high-resolution RGB images of a grassland area, and performs geometric correction and geographic registration on the images to establish a mapping association between image pixels and actual spatial positions. Then, while retaining geographic metadata, the image is sliced ​​and semantically segmented to separate the individual plants in the image and generate a set of plant instance ROI images. Subsequently, the multi-scale features of the plants are further extracted to take into account their global morphology and local texture, and the multi-scale feature representation of the plant images is enhanced by context perception in multiple directions to enhance the discrimination of spatial structural features such as leaf edges and vein sequence directions, thereby achieving accurate identification and positioning of invasive species. This method can effectively improve the efficiency of grassland ecological monitoring and achieve early warning and precise tracking of invasive species.

[0006] Accordingly, according to one aspect of the present application, a grassland ecological monitoring method based on image recognition is provided, which includes:

[0007] Obtain RGB images of the target grassland area collected by the UAV;

[0008] Performing image preprocessing on the RGB image of the target grassland area to obtain an image block of the target grassland local area with geographic coordinates;

[0009] Inputting the target grassland local area image block with geographic coordinates into a plant instance semantic segmentation model to obtain a set of plant instance ROI images;

[0010] Extracting a plant instance ROI image corresponding to a first plant from the set of plant instance ROI images;

[0011] Perform multi-directional enhanced perception of image features on the plant instance ROI image corresponding to the first plant to determine whether the first plant is an invasive species.

[0012] According to another aspect of the present application, a grassland ecological monitoring system based on image recognition is provided, which includes:

[0013] Grassland RGB image acquisition module, used to obtain RGB images of the target grassland area collected by the UAV;

[0014] An image preprocessing module, configured to perform image preprocessing on the RGB image of the target grassland area to obtain an image block of a local area of ​​the target grassland with geographic coordinates;

[0015] A plant instance semantic segmentation module is used to input the target grassland local area image block with geographic coordinates into a plant instance semantic segmentation model to obtain a set of plant instance ROI images;

[0016] a plant image extraction module to be detected, configured to extract a plant instance ROI image corresponding to a first plant from the set of plant instance ROI images;

[0017] The invasive species identification module is used to perform multi-directional enhanced perception of image features on the plant instance ROI image corresponding to the first plant to determine whether the first plant is an invasive species.

[0018] Compared with the existing technology, the grassland ecological monitoring method and system based on image recognition provided by this application uses drones to obtain high-resolution RGB images of grassland areas, and performs geometric correction and geographic registration on them to establish a mapping association between image pixels and actual spatial positions. Then, while retaining geographic metadata, the image is sliced ​​and semantically segmented to separate each independent plant individual in the image and generate a set of plant instance ROI images. Subsequently, the multi-scale features of the plant are further extracted to take into account its global morphology and local texture, and the multi-scale feature representation of the plant image is enhanced by context perception in multiple directions to enhance the discrimination of spatial structural features such as leaf edges and vein sequence directions, thereby achieving accurate identification and positioning of invasive species. This method can effectively improve the efficiency of grassland ecological monitoring and achieve early warning and precise tracking of invasive species. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0020] Figure 1 Flowchart of a grassland ecological monitoring method based on image recognition according to an embodiment of the present application.

[0021] Figure 2 Schematic diagram of data flow of a grassland ecological monitoring method based on image recognition according to an embodiment of the present application.

[0022] Figure 3 This is a flowchart of step S2 in the grassland ecological monitoring method based on image recognition according to an embodiment of the present application.

[0023] Figure 4 This is a flowchart of step S5 in the grassland ecological monitoring method based on image recognition according to an embodiment of the present application.

[0024] Figure 5This is a flowchart of step S52 in the grassland ecological monitoring method based on image recognition according to an embodiment of the present application.

[0025] Figure 6 This is a block diagram of a grassland ecological monitoring system based on image recognition according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0027] Figure 1 Flowchart of a grassland ecological monitoring method based on image recognition according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of grassland ecological monitoring method based on image recognition according to the embodiment of the present application. Figure 1 and Figure 2 As shown, the grassland ecological monitoring method based on image recognition according to the embodiment of the present application includes the following steps: S1, acquiring an RGB image of a target grassland area collected by a drone; S2, performing image preprocessing on the RGB image of the target grassland area to obtain a target grassland local area image block with geographic coordinates; S3, inputting the target grassland local area image block with geographic coordinates into a plant instance semantic segmentation model to obtain a set of plant instance ROI images; S4, extracting a plant instance ROI image corresponding to a first plant from the set of plant instance ROI images; S5, performing multi-directional enhanced perception of image features on the plant instance ROI image corresponding to the first plant to determine whether the first plant is an invasive species.

[0028] In the above-mentioned grassland ecological monitoring method based on image recognition, step S1 obtains an RGB image of the target grassland area collected by an unmanned aerial vehicle. It should be understood that due to the shortcomings of traditional grassland ecological monitoring methods, such as manual field surveys, there are low efficiency, high cost, limited monitoring range, and difficulty in achieving high-frequency dynamic monitoring, especially for the early and small-scale occurrence of invasive species, it is difficult to effectively capture them, and the spatial resolution of satellite remote sensing images is often insufficient in the identification of individual plant levels. Therefore, in order to obtain high-resolution raw data that can accurately display the individual details of grassland vegetation, this application utilizes the advantages of unmanned aerial vehicle remote sensing with low flight altitude, maneuverability, and the ability to carry high-resolution sensors. By operating the unmanned aerial vehicle equipped with an RGB (red, green, and blue) camera to take aerial photos of the target grassland area, RGB images covering the target grassland area are obtained. In the specific implementation process, first, the flight route, altitude, flight speed of the unmanned aerial vehicle, and the ground resolution and overlap (heading overlap and lateral overlap) of the image are pre-planned to ensure that the collected images can cover the entire target area and can support image splicing and three-dimensional reconstruction in the subsequent grassland ecological monitoring and management process through sufficient overlap. Next, the drone flies autonomously or remotely according to the set parameters, and the onboard RGB camera takes continuous photos at a certain interval to obtain a series of digital images containing rich color and texture information. This can efficiently and economically obtain high-spatial-resolution RGB images of the target grassland area, clearly recording the individual morphology, color, texture and other surface characteristics of grassland vegetation, and thus providing high-quality original image data for subsequent individual plant identification and invasive species judgment.

[0029] In the above-mentioned grassland ecological monitoring method based on image recognition, the step S2 is to perform image preprocessing on the RGB image of the target grassland area to obtain an image block of the target grassland local area with geographic coordinates. Figure 3 FIG. 1 is a flow chart of step S2 in the grassland ecological monitoring method based on image recognition according to an embodiment of the present application. Figure 3 As shown, step S2 includes: S21, performing geometric correction and geographic registration on the RGB image of the target grassland area to obtain the geographic coordinates of each pixel in the RGB image of the target grassland area; S22, performing radiometric calibration and color standardization on the RGB image of the target grassland area with geographic coordinates, and then performing image slicing to obtain the image block of the local area of ​​the target grassland with geographic coordinates.

[0030] Specifically, the S21 performs geometric correction and geographic registration on the RGB image of the target grassland area to obtain the geographic coordinates of each pixel in the RGB image of the target grassland area. It should be understood that since the drone is affected by various factors such as airflow, changes in its own posture (pitch, roll, yaw) and camera lens distortion during flight, the original RGB image collected has different degrees of geometric deformation, and there is a lack of direct and accurate correspondence between the pixels on the image and their actual geographical locations on the earth's surface, which makes it difficult to locate invasive species. Therefore, the present application is further based on the basic principles of photogrammetry and remote sensing image processing, and performs geometric correction and geographic registration on the original RGB image to eliminate the geometric distortion of the image, and accurately assigns each pixel in the image to its geographic coordinates in the real world (such as longitude and latitude or coordinates in a specific projection coordinate system), thereby realizing accurate linking of image information and geographic spatial position, and providing a spatial basis for the subsequent precise positioning of identified plants (especially invasive species). In the specific implementation process, several ground control points (GCPs) must be deployed throughout the survey area and their 3D geographic coordinates precisely measured using high-precision GPS / GNSS equipment. Alternatively, a drone can be equipped with a high-precision RTK (Real-Time Kinematic Differential) or PPK (Post-Processed Kinematic Differential) module to obtain highly accurate camera position (POS) data at the time of exposure. Next, using specialized drone data processing software (such as Pix4Dmapper or Agisoft Metashape), the original RGB imagery, GCP data (if available), and POS data (if available) are imported. Aerial triangulation (analytic aerial triangulation or bundle block adjustment) is used to calculate the precise internal and external orientation elements of the camera and the 3D coordinates of the object points. This is then combined with a digital elevation model (DEM, which can be generated from the imagery itself or imported externally) to eliminate image point displacement caused by terrain undulations. Ultimately, a geometrically de-distorted RGB orthophoto with geographic coordinates is generated. In this way, each pixel in the image corresponds to a clear geographic coordinate, which not only ensures the accuracy of distance and area measurement on the image, but more importantly, any plant subsequently identified from the image can have its position directly converted into actual geographic coordinates, thus providing the core spatial information foundation for the precise positioning and management of invasive species.

[0031] Specifically, in S22, after performing radiation calibration and color standardization on the RGB image of the target grassland area with geographic coordinates, the image is sliced ​​to obtain the image block of the local area of ​​the target grassland with geographic coordinates. It should be understood that when the drone collects images at different times (such as different times of the day, different seasons) and under different weather conditions (such as sunny days and cloudy days), environmental factors such as light intensity, solar altitude angle, and atmospheric conditions will have significant differences. At the same time, the response characteristics of the camera sensor itself will also vary over time or batches, which will lead to inconsistencies in the brightness, contrast, and color performance of the acquired RGB images, which will directly affect the stability and generalization ability of subsequent plant identification and classification based on visual features such as image color and texture. In addition, it is not feasible to directly input the entire large-scale orthophoto (which may reach GB level) into the deep learning model for processing due to video memory limitations and computational efficiency issues. Therefore, this application is further based on remote sensing image preprocessing technology and data segmentation strategy. By first performing radiometric calibration and color standardization on large-scale geo-referenced images, and then performing image segmentation, the radiometric distortion and color deviation caused by differences in illumination and sensors are eliminated or reduced, ensuring that the image data has better consistency and comparability in spectral characteristics, while reducing the input data size of subsequent deep learning models, reducing the complexity of model processing, and improving computational efficiency. Specifically, radiometric calibration aims to convert the dimensionless DN value (Digital Number) recorded by the image sensor into a radiometric brightness value or surface reflectance value with actual physical meaning. This can be achieved by utilizing sensor factory calibration parameters, calibration plate data collected synchronously on-site (such as a standard gray plate), and empirical linear transformation. Color standardization further adjusts the overall hue, brightness, and contrast of the image through image enhancement algorithms (such as histogram matching, color constancy algorithm, white balance adjustment, etc.), so that it exhibits a more consistent visual effect between images collected in different batches and under different lighting conditions. After image preprocessing, the standardized large-format orthophoto image is cut into a series of smaller image blocks according to a preset size (e.g., 256x256 pixels, 512x512 pixels, etc., depending on the input requirements of the subsequent semantic segmentation model) and a certain overlap rate (e.g., 10%-30%, to avoid plants being split at the edges of the blocks and to ensure that each plant is completely contained in at least one image block). Importantly, during the block cutting process, each image block inherits its geographic coordinate information in the original orthophoto image. In this way, not only are image data with more consistent and standardized color and radiation characteristics obtained, which improves the robustness of subsequent model analysis, but also, through block cutting, the big data is decomposed into small units suitable for deep learning model processing, while retaining the geographic location information of each small image block, laying the foundation for the precise positioning and spatial analysis of subsequent recognition results.

[0032] In the above-mentioned grassland ecological monitoring method based on image recognition, in step S3, the target grassland local area image block with geographic coordinates is input into the plant instance semantic segmentation model to obtain a set of plant instance ROI images. In a specific example of the present application, the plant instance semantic segmentation model is a trained Mask R-CNN model. It should be understood that in a complex grassland environment, different plants are often closely adjacent and overlap with each other, and the background (such as soil, dead grass, stones, etc.) is complex and diverse. It is difficult for traditional end-to-end image classification methods to accurately distinguish and locate individual plant instances. Therefore, in order to achieve accurate identification and boundary delineation of each individual plant in the image, thereby providing clear and independent objects for subsequent fine feature extraction and invasive species judgment for individual plants, the present application adopts a trained Mask R-CNN model as a plant instance semantic segmentation model, and obtains a set of plant instance ROI images by performing pixel-by-pixel classification and segmentation mask generation on the grassland local area image block with geographic coordinates. Specifically, Mask R-CNN is a powerful instance segmentation framework that adds a parallel mask prediction branch to object detection networks (such as Faster R-CNN). This allows for accurate segmentation of target objects while simultaneously generating high-quality segmentation masks for each instance. During training, a large number of images containing various grassland plant species (including both invasive and native species) are collected and carefully annotated manually (i.e., a precise pixel-level outline mask is created for each plant instance). The Mask R-CNN model is then trained using this annotated dataset through supervised learning, learning to simultaneously perform three tasks from the input image: 1) identify the presence of a plant (object classification); 2) locate the bounding box of each plant (object detection); and 3) generate a pixel-level segmentation mask for each identified plant (instance segmentation). After training, the obtained image patches of the target grassland local area, along with the geographic coordinates, are fed into the trained Mask R-CNN model one by one. The model performs a forward propagation on each image block, obtaining a series of binary plant instance segmentation masks containing bounding box coordinates. Pixels with a mask value of 1 correspond to the foreground region of the plant instance, while pixels with a mask value of 0 correspond to the background region. By overlaying the mask with the original image according to the bounding box coordinates, the ROI (Region of Interest) image of each plant instance can be extracted, resulting in a collection of plant instance ROI images. In this way, each individual plant can be separated from the complex grassland background while retaining its complete shape and outline information, laying the foundation for subsequent single plant analysis.

[0033] In the above-mentioned grassland ecological monitoring method based on image recognition, step S4 extracts the plant instance ROI image corresponding to the first plant from the set of plant instance ROI images. That is, in order to be able to perform independent and in-depth analysis on each plant instance in the set of plant instance ROI images, the present application implements traversal recognition of plant instance ROI images by selecting plant instance ROI images (defined as the plant instance ROI image corresponding to the first plant) from the set of plant instance ROI images one by one as the current processing unit. During implementation, the ROI images of individual plants can be extracted one by one from the set of multiple plant instance ROIs in sequence (for example, according to the order of appearance in the image, or according to the confidence level of the model output) or according to a preset strategy. In each iteration or processing cycle, the currently selected plant instance ROI image is regarded as the plant instance ROI image corresponding to the first plant. It should be understood that the "first plant" here does not refer to a specific plant, but rather refers to the plant object of interest in the current processing batch. Once the invasive species analysis process for the "first plant" is complete, the system then extracts the next plant instance ROI image from the collection, making it the new "first plant," and repeats this cycle until all plant instance ROIs in the collection have been processed. This ensures that subsequent invasive species identification can focus on a single, independent plant object, and the entire processing process systematically covers all plants in the image block.

[0034] In the above-mentioned grassland ecological monitoring method based on image recognition, the step S5 is to perform multi-directional enhanced perception of image features on the plant instance ROI image corresponding to the first plant to determine whether the first plant is an invasive species. Figure 4 FIG. 5 is a flow chart of step S5 in the grassland ecological monitoring method based on image recognition according to an embodiment of the present application. Figure 4 As shown, the step S5 includes: S51, extracting multi-scale image features from the plant instance ROI image corresponding to the first plant to obtain a first plant multi-scale image feature coding map; S52, performing feature multi-directional enhanced perception on the first plant multi-scale image feature coding map to obtain a first plant multi-scale image feature enhanced coding map; S53, performing feature classification on the first plant multi-scale image feature enhanced coding map to determine whether the first plant is an invasive species.

[0035] Specifically, the step S51 extracts multi-scale image features from the plant instance ROI image corresponding to the first plant to obtain a first plant multi-scale image feature encoding map. In a specific example of the present application, multi-scale image feature extraction based on the FPN model is performed on the plant instance ROI image corresponding to the first plant to obtain the first plant multi-scale image feature encoding map. It should be understood that the morphological characteristics of plants, such as overall canopy structure, branching pattern, leaf shape, leaf size, texture details (such as veins, hairs, edge serrations), etc., often present different discriminant information at different observation scales. A single-scale feature extraction network may find it difficult to simultaneously capture these global overview and local refinement features. Therefore, in order to comprehensively and effectively capture discriminative visual information of different spatial scales from the ROI image of a single plant and provide high-quality feature input for subsequent invasive species identification, this application is based on the Feature Pyramid Network (FPN) architecture in deep learning. By performing bottom-up multi-scale feature extraction and top-down multi-scale feature fusion on the plant instance ROI image corresponding to the first plant, it is possible to generate a multi-scale feature representation containing rich detail information and deep semantic information. Specifically, the FPN model is built on the standard convolutional neural network backbone, and is composed of a feature pyramid of multiple convolutional layers of different scales, wherein convolutional layers of different depths will naturally produce feature maps of different spatial resolutions and different semantic levels: shallow feature maps have high spatial resolution but weak semantic information, and are rich in low-level details such as textures and edges; deep feature maps have low spatial resolution but strong semantic information, and are rich in category-related abstract concepts. The FPN model introduces a top-down path and lateral connections to progressively upsample high-level semantic information and fuse it with shallow, high-resolution feature maps (e.g., element-wise addition or concatenation). This ensures that the feature maps output by each layer of the pyramid possess both strong semantic information and good spatial positioning capabilities. Based on this, by inputting the plant instance ROI image corresponding to the first plant into the FPN model for image feature extraction, the resulting first plant multi-scale image feature encoding map can simultaneously represent macroscopic features such as the plant's overall outline and canopy structure, as well as microscopic details such as leaf edges and surface texture. Compared to single-scale feature representation, this significantly improves the ability to distinguish between native and invasive species.

[0036] Specifically, in step S52, the first plant multi-scale image feature encoding map is subjected to multi-directional feature enhancement perception to obtain a first plant multi-scale image feature enhancement encoding map. It should be understood that, considering that the local receptive field characteristics of the standard convolutional neural network (CNN) limit the model's perception of long-distance spatial structures, and the isotropic response of the traditional convolution kernel may have certain limitations in perceiving and capturing subtle structural features with specific directions (such as the vein direction of plant leaves, the specific angle of edge serrations, the growth direction of stems, etc.), resulting in insufficient feature differentiation. In this regard, in order to further enhance the discrimination of spatial structural features such as leaf edges and vein direction, thereby improving the accuracy and robustness of the final invasive species identification, this application proposes a multi-directional enhanced perception algorithm. By performing context-related feature extraction and attention aggregation in multiple directions on each pixel position feature in the first plant multi-scale image feature coding map, each pixel position feature can fully integrate the context information in multiple directions around it, thereby enhancing the perception ability of specific directional subtle structural features, and obtaining the first plant multi-scale image feature enhancement coding map with more keen perception and representation capabilities for the key directional features of the plant (such as texture direction, edge angle), providing a more refined and powerful feature basis for subsequent invasive species classification and identification.

[0037] Figure 5 FIG. 5 is a flow chart of step S52 in the grassland ecological monitoring method based on image recognition according to an embodiment of the present application. Figure 5 As shown, the step S52 includes: S521, extracting the channel feature vector at the (h, w)th pixel position from the first plant multi-scale image feature coding map as the first plant multi-scale image feature vector to be enhanced; S522, performing multi-directional context information perception on the first plant multi-scale image feature vector to be enhanced to obtain a first-direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, a second-direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, and a third-direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced; S523, performing direction-sensitive attention aggregation on the first-direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, the second-direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, and the third-direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced to obtain a first plant multi-scale image feature enhancement coding vector, wherein the first plant multi-scale image feature enhancement coding vector is the channel feature vector at the (h, w)th pixel position of the first plant multi-scale image feature enhancement coding map.

[0038] Specifically, the step S521 can be expressed as follows:

[0039] F∈R H×W×C

[0040] v h,w ∈R C

[0041] Wherein, F is the first plant multi-scale image feature coding map, H, W, and C are the height, width, and number of channels of the first plant multi-scale image feature coding map, respectively, and v h,w represents the feature vector to be enhanced of the first plant multi-scale image, that is, the channel feature vector of the (h, w)th pixel position in the feature coding map of the first plant multi-scale image.

[0042] That is, in order to achieve fine-grained feature enhancement of the first plant multi-scale image feature coding map, the present application extracts the channel feature vector of the (h, w)th pixel position as the enhancement target, and subsequently enriches and enhances the feature expression of the pixel position by associating the perception fusion of the multi-directional contextual information around the pixel position. Specifically, the channel feature vector of the first plant multi-scale image feature coding map is generated by the multi-level feature fusion of the FPN network, and its channel dimension carries cross-scale semantic information from the overall morphology of the plant to the micro-texture. By extracting its channel feature vector, it is possible to focus on the local feature representation of the current spatial position, providing a basis for subsequent multi-directional enhanced perception.

[0043] Specifically, step S522 includes: first, in the first plant multi-scale image feature coding map, with the first plant multi-scale image feature vector to be enhanced as the center, feature sampling is performed along the first direction, the second direction, and the third direction to obtain a set of first-direction plant multi-scale image feature context vectors, a set of second-direction plant multi-scale image feature context vectors, and a set of third-direction plant multi-scale image feature context vectors, wherein the first plant multi-scale image feature vector to be enhanced is located at the center position of the set of the first-direction plant multi-scale image feature context vectors, the set of the second-direction plant multi-scale image feature context vectors, and the set of the third-direction plant multi-scale image feature context vectors, and can be expressed by the formula:

[0044]

[0045] N=2k+1

[0046] Among them, d i represents the i-th direction, In the direction d i The step size vector used when sampling on , represents the set of plant multi-scale image feature context vectors in the first direction, the set of plant multi-scale image feature context vectors in the second direction, and the set of plant multi-scale image feature context vectors in the third direction. k represents the sampling radius, that is, the channel feature vectors of k pixel positions are sampled from the center pixel to both sides in each direction. If the coordinates are out of bounds, they are padded with zero values. Representing a collection The nth context vector in the set The number of context vectors in .

[0047] Here, since the discriminative features of grassland vegetation often extend along a specific spatial direction, the fixed sampling pattern of the traditional convolution kernel cannot adaptively capture such anisotropic structures. Therefore, in order to explicitly model the direction-sensitive long-distance dependencies in the first plant multi-scale image feature encoding map, this application is based on a spatial sequence sampling strategy, with the target pixel (h, w) as the center, and performs ordered feature sampling on the feature map along three preset directions: horizontal (0°), vertical (90°), and diagonal (45°). In a specific implementation, with the target pixel (h, w) as the center, symmetrical sampling is performed along each direction with a fixed step size (such as 1 pixel per step), and channel feature vectors with a distance of 1 to K pixels from the center point are collected on both sides of the target pixel to form a set of 2K+1 channel feature vectors. In this way, the model can capture the evolution law of spatial structure from local to remote in each direction (such as the continuity of leaf veins or the periodicity of texture), and provide structured input for subsequent context modeling.

[0048] Next, the set of the first-directional plant multi-scale image feature context vectors, the set of the second-directional plant multi-scale image feature context vectors, and the set of the third-directional plant multi-scale image feature context vectors are respectively input into a directional context perception device based on a converter structure to obtain a first-directional context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, a second-directional context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, and a third-directional context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, which can be expressed as follows:

[0049]

[0050] Among them, TransformerEncoder represents the directional context perception based on the converter structure, is the set of direction context perception pairs based on the converter structure The set of contextual latent state features obtained after context-aware encoding, represents a first-direction context-aware implicit coding vector of the feature to be enhanced in the first plant multi-scale image, a second-direction context-aware implicit coding vector of the feature to be enhanced in the first plant multi-scale image, and a third-direction context-aware implicit coding vector of the feature to be enhanced in the first plant multi-scale image, Representing a collection The k+1th contextual hidden state feature in . That is, extract the set The contextual latent state features at the center position are used as the first direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, the second direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, and the third direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced.

[0051] Here, considering that the traditional recurrent neural network (RNN) has a gradient vanishing problem when processing long sequences, and it is difficult for a fixed convolution kernel to dynamically adjust the attention weights of key features in the sequence. Therefore, in order to efficiently model the contextual dependencies within the directional sequence, this application designs a directional context sensor based on the self-attention mechanism of Transformer. In a specific implementation, a set of plant multi-scale image feature context vectors in each direction is input into an encoding module comprising a self-attention layer and a feedforward neural network. For example, after the horizontal direction sequence is positionally encoded, the correlation weights between the feature vectors in the sequence are calculated by self-attention, and the context information with the greatest relevance to the position feature of the center pixel (h, w) is dynamically aggregated (such as the indicative effect of the distal leaf tip morphology on the current leaf vein direction), thereby obtaining a context-aware implicit coding vector based on the target pixel (h, w) position feature to be enhanced and integrating the global contextual association information in the corresponding direction, forming the first direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, the second direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, and the third direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced.

[0052] Specifically, step S523 includes: first, performing multi-layer feature modulation based on multi-directional sensitivity on the first direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, the second direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, and the third direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced to obtain the first direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image feature to be enhanced, the second direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image feature to be enhanced, and the third direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image feature to be enhanced, which can be expressed by the formula:

[0053]

[0054] in, and Direction d i and direction d j The embedding representation vector, φ(d i ,d j ) is the direction d i Relative to direction d j The attention score, exp(·) represents the exponential operation with e as the base, α i,j Indicates direction d i Relative to Attention weights, and j = 1, 2, 3, Indicates multi-directional modulation That is, the j-th direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced is modulated in multiple directions.

[0055] It should be understood that, since the context information in different directions also has context association, and the contribution to the enhancement of the current pixel features may change dynamically with the scene (for example, the diagonal direction feature has a higher discriminant value for spiral veins than the horizontal direction). Therefore, in order to adaptively enhance the feature response of the key direction, this application introduces a direction-sensitive multi-layer feature modulation mechanism. Specifically, for the context-aware implicit coding vector of each direction, the sensitivity attention weights of the three directions are calculated in turn through the direction identification embedding layer (encoding the direction label as a vector) and the feature interaction layer (feature interaction with the context-aware implicit coding vector obtained after context aggregation perception), and three-layer feature modulation is performed on it to obtain the context-aware implicit coding vector after multi-layer modulation. Finally, the context-aware implicit coding vector in each direction is modulated in multiple directions respectively to form the first direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image to be enhanced feature, the second direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image to be enhanced feature, and the third direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image to be enhanced feature. By stacking feature modulation, the contribution weights of each directional feature can be gradually refined, thereby highlighting directional patterns that are crucial for invasive species identification.

[0056] Finally, the first-direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image feature to be enhanced, the second-direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image feature to be enhanced, and the third-direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image feature to be enhanced are fused to obtain the first plant multi-scale image feature enhancement coding vector. That is, in order to integrate the complementarity of multi-directional context information, the present application further performs feature fusion on the context-aware implicit coding vectors in the three directions after multi-directional modulation to form a global enhanced expression of cross-directional complementary features to obtain the first plant multi-scale image feature enhancement coding vector. In a specific example of the present application, the first-direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image feature to be enhanced, the second-direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image feature to be enhanced, and the third-direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image feature to be enhanced are fused in a position-based summation manner to obtain the first plant multi-scale image feature enhancement coding vector, which can be expressed by the formula:

[0057]

[0058] in, Represents the first plant multi-scale image feature enhanced coding vector.

[0059] Based on this, by performing the above operation on each pixel position in the original first plant multi-scale image feature coding map, the global image features are enhanced from local to overall, and the first plant multi-scale image feature enhanced coding map is obtained.

[0060] In particular, in a preferred embodiment of the present application, the first direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image to be enhanced feature, the second direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image to be enhanced feature, and the third direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image to be enhanced feature are weighted fused based on heterogeneous response coupling correction to obtain the first plant multi-scale image feature enhanced coding vector. Here, considering that the context-aware coding vectors of different directions have anisotropic field polarization effect when fused (i.e., the difference in feature response intensity in different directions is too large), which leads to information aliasing in the feature coupling process (for example, horizontal direction features excessively suppress the effective information in the diagonal direction), thereby reducing the discriminability of the first plant multi-scale image feature enhanced coding vector. Therefore, in order to balance the contribution of multi-directional features and suppress the polarization effect, the present application proposes a heterogeneous response coupling correction mechanism. By mathematically modeling the mutual constraint relationship between the features in each direction, the fusion weight is dynamically adjusted so that the final enhanced feature can retain directional specificity and reflect cross-directional structural correlation.

[0061] Specifically, before summing the first direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image to be enhanced, the second direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image to be enhanced, and the third direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image to be enhanced by position, the context-aware implicit coding vector of each direction after multi-directional modulation is calculated. Compute its normalized response with respect to the other two directions:

[0062]

[0063] in, represents the k-th direction context-aware implicit coding vector of the first plant multi-scale image to be enhanced, where k = 1, 2, 3 and k ≠ j. express The high-order anisotropic bilinear response representation vector of ;

[0064] Then, under the covariant partial derivative representation of heterogeneous analysis, we can respectively right Find the partial derivatives:

[0065]

[0066] in, express Here, due to the context-aware implicit coding vectors in the other two directions in the denominator, It is essentially symmetrical, so its partial derivatives are the same, which means that the calculation of partial derivatives makes an anisotropic correction to the overall field polarization response.

[0067] In this way, and After point multiplication weighted correction, calculate:

[0068]

[0069] Based on this, the multi-directional field coupling effect under the direction-dependent polarization intensity is effectively improved, and the aliasing fusion effect of the first direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image to be enhanced feature, the second direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image to be enhanced feature, and the third direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image to be enhanced feature is improved, thereby improving the first plant multi-scale image feature enhancement coding vector. accuracy of expression.

[0070] Specifically, the step S53 performs feature classification on the first plant multi-scale image feature enhancement coding map to determine whether the first plant is an invasive species. Specifically, the present application is based on the classification principle in supervised learning, and outputs a classification result of whether the plant is an invasive species by sending the first plant multi-scale image feature enhancement coding map into a pre-trained classifier for feature pattern analysis and discrimination. In the training stage, the classifier uses plant samples containing a large number of known categories (i.e., those that have been clearly identified as invasive species or native species) for training and learning. These samples have also undergone the aforementioned image preprocessing, instance segmentation, multi-scale feature extraction, and multi-directional feature enhancement perception processes to obtain corresponding feature coding maps as training inputs, so that the classifier learns the mapping relationship from the visual feature patterns of specific plants (such as canopy shape, leaf contours, texture features, etc.) to the category to which they belong (invasive species or native species). In actual monitoring applications, when the first, unlabeled, multi-scale image feature enhancement code is fed into a trained classifier, the classifier automatically identifies and categorizes plants as invasive by calculating the probability distribution of the plant belonging to a predefined category (e.g., "invasive species" / "non-invasive species") and selecting the category with the highest probability as the final classification result. Furthermore, by combining the location of the plant's ROI in the original image and the image's georeferencing information, once an invasive species is identified, the plant's specific geographic location on the grassland can be precisely determined, providing critical technical and data support for subsequent grassland ecological management, the development of invasive species removal strategies, and the evaluation of their effectiveness.

[0071] In summary, according to the embodiment of the present application, a grassland ecological monitoring method based on image recognition is explained, which uses a drone to obtain high-resolution RGB images of a grassland area, and performs geometric correction and geographic registration on the images to establish a mapping association between image pixels and actual spatial positions. Then, while retaining geographic metadata, the image is sliced ​​and semantically segmented to separate the individual plants in the image and generate a set of plant instance ROI images. Subsequently, the multi-scale features of the plants are further extracted to take into account their global morphology and local texture, and the multi-scale feature representation of the plant images is enhanced by context perception in multiple directions to enhance the discrimination of spatial structural features such as leaf edges and vein sequence directions, thereby achieving accurate identification and positioning of invasive species. This method can effectively improve the efficiency of grassland ecological monitoring and achieve early warning and precise tracking of invasive species.

[0072] Furthermore, the present application also provides a grassland ecological monitoring system based on image recognition.

[0073] Figure 6 FIG is a block diagram of a grassland ecological monitoring system based on image recognition according to an embodiment of the present application. Figure 6 As shown, the grassland ecological monitoring system 100 based on image recognition according to the embodiment of the present application includes: a grassland RGB image acquisition module 110, used to acquire an RGB image of a target grassland area collected by a drone; an image preprocessing module 120, used to perform image preprocessing on the RGB image of the target grassland area to obtain a target grassland local area image block with geographic coordinates; a plant instance semantic segmentation module 130, used to input the target grassland local area image block with geographic coordinates into a plant instance semantic segmentation model to obtain a set of plant instance ROI images; a plant image extraction module 140 to be detected, used to extract a plant instance ROI image corresponding to a first plant from the set of plant instance ROI images; an invasive species identification module 150, used to perform multi-directional enhanced perception of image features on the plant instance ROI image corresponding to the first plant to determine whether the first plant is an invasive species.

[0074] Here, those skilled in the art will appreciate that the specific operations of each module in the grassland ecological monitoring system based on image recognition have been described in the above Figures 1 to 5 The description of the grassland ecological monitoring method based on image recognition has been introduced in detail, and therefore, its repeated description will be omitted.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A grassland ecological monitoring method based on image recognition, characterized in that: include: Obtain RGB images of the target grassland area collected by the UAV; performing image preprocessing on the RGB image of the target grassland area to obtain an image block of a local area of ​​the target grassland with geographic coordinates; Inputting the target grassland local area image block with geographic coordinates into a plant instance semantic segmentation model to obtain a set of plant instance ROI images; Extracting a plant instance ROI image corresponding to a first plant from the set of plant instance ROI images; Perform multi-directional enhanced perception of image features on the plant instance ROI image corresponding to the first plant to determine whether the first plant is an invasive species.

2. The grassland ecological monitoring method based on image recognition according to claim 1, characterized in that: Performing image preprocessing on the RGB image of the target grassland area to obtain an image block of a local area of ​​the target grassland with geographic coordinates, including: Performing geometric correction and geographic registration on the RGB image of the target grassland area to obtain the geographic coordinates of each pixel in the RGB image of the target grassland area; After radiometric calibration and color standardization are performed on the RGB image of the target grassland area with the geographic coordinates, the image is sliced ​​to obtain image blocks of the target grassland local area with the geographic coordinates.

3. The grassland ecological monitoring method based on image recognition according to claim 1, characterized in that: The plant instance semantic segmentation model is a trained Mask R-CNN model.

4. The grassland ecological monitoring method based on image recognition according to claim 1, characterized in that: Performing multi-directional enhanced perception of image features on a plant instance ROI image corresponding to the first plant to determine whether the first plant is an invasive species includes: Extracting multi-scale image features from the plant instance ROI image corresponding to the first plant to obtain a first plant multi-scale image feature encoding map; Performing feature multi-directional enhanced perception on the first plant multi-scale image feature coding map to obtain a first plant multi-scale image feature enhanced coding map; Feature classification is performed on the first plant multi-scale image feature enhanced coding map to determine whether the first plant is an invasive species.

5. The grassland ecological monitoring method based on image recognition according to claim 4 is characterized in that: Extracting multi-scale image features from the plant instance ROI image corresponding to the first plant to obtain a first plant multi-scale image feature encoding map includes: Multi-scale image feature extraction based on the FPN model is performed on the plant instance ROI image corresponding to the first plant to obtain the first plant multi-scale image feature coding map.

6. The grassland ecological monitoring method based on image recognition according to claim 5, characterized in that: Performing multi-directional enhanced feature perception on the first plant multi-scale image feature coding map to obtain a first plant multi-scale image feature enhanced coding map includes: Extracting the channel feature vector at the (h, w)th pixel position from the first plant multi-scale image feature coding map as the first plant multi-scale image feature vector to be enhanced; Performing multi-directional context information perception on the feature vector to be enhanced of the first plant multi-scale image to obtain a first-directional context-aware implicit coding vector of the feature to be enhanced of the first plant multi-scale image, a second-directional context-aware implicit coding vector of the feature to be enhanced of the first plant multi-scale image, and a third-directional context-aware implicit coding vector of the feature to be enhanced of the first plant multi-scale image; The first direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, the second direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, and the third direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced are subjected to direction-sensitivity-based attention aggregation to obtain a first plant multi-scale image feature enhanced coding vector, wherein the first plant multi-scale image feature enhanced coding vector is the channel feature vector of the (h, w)th pixel position of the first plant multi-scale image feature enhanced coding map.

7. The grassland ecological monitoring method based on image recognition according to claim 6, characterized in that: Performing multi-directional context information perception on the feature vector to be enhanced of the first plant multi-scale image to obtain a first-direction context-aware implicit coding vector of the feature to be enhanced of the first plant multi-scale image, a second-direction context-aware implicit coding vector of the feature to be enhanced of the first plant multi-scale image, and a third-direction context-aware implicit coding vector of the feature to be enhanced of the first plant multi-scale image, including: In the first plant multi-scale image feature coding map, with the first plant multi-scale image feature vector to be enhanced as the center, feature sampling is performed along the first direction, the second direction, and the third direction to obtain a set of plant multi-scale image feature context vectors in the first direction, a set of plant multi-scale image feature context vectors in the second direction, and a set of plant multi-scale image feature context vectors in the third direction, wherein the first plant multi-scale image feature vector to be enhanced is located at a center position of the set of plant multi-scale image feature context vectors in the first direction, the set of plant multi-scale image feature context vectors in the second direction, and the set of plant multi-scale image feature context vectors in the third direction; The set of the first-directional plant multi-scale image feature context vectors, the set of the second-directional plant multi-scale image feature context vectors, and the set of the third-directional plant multi-scale image feature context vectors are respectively input into the directional context perception based on the converter structure to obtain the first-directional context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, the second-directional context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, and the third-directional context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced.

8. The grassland ecological monitoring method based on image recognition according to claim 7, characterized in that: Performing direction-sensitive attention aggregation on a first direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, a second direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, and a third direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced to obtain a first plant multi-scale image feature enhanced coding vector, including: performing multi-directional sensitivity-based multi-layer feature modulation on the first direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, the second direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, and the third direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, respectively, to obtain the multi-directionally modulated first direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, the multi-directionally modulated second direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced, and the multi-directionally modulated third direction context-aware implicit coding vector of the first plant multi-scale image feature to be enhanced; The first-direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image feature to be enhanced, the second-direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image feature to be enhanced, and the third-direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image feature to be enhanced are fused to obtain the first plant multi-scale image feature enhancement coding vector.

9. The grassland ecological monitoring method based on image recognition according to claim 8, characterized in that: The method comprises fusing the first-direction context-aware implicit coding vector of the feature to be enhanced of the multi-directionally modulated first plant multi-scale image, the second-direction context-aware implicit coding vector of the feature to be enhanced of the multi-directionally modulated first plant multi-scale image, and the third-direction context-aware implicit coding vector of the feature to be enhanced of the multi-directionally modulated first plant multi-scale image to obtain the first plant multi-scale image feature enhanced coding vector, including: The first-direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image feature to be enhanced, the second-direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image feature to be enhanced, and the third-direction context-aware implicit coding vector of the multi-directionally modulated first plant multi-scale image feature to be enhanced are weightedly fused based on heterogeneous response coupling correction to obtain the first plant multi-scale image feature enhancement coding vector.

10. A grassland ecological monitoring system based on image recognition, characterized in that: include: Grassland RGB image acquisition module, used to obtain RGB images of the target grassland area collected by the UAV; An image preprocessing module, configured to perform image preprocessing on the RGB image of the target grassland area to obtain an image block of a local area of ​​the target grassland with geographic coordinates; A plant instance semantic segmentation module is used to input the target grassland local area image block with geographic coordinates into a plant instance semantic segmentation model to obtain a set of plant instance ROI images; a plant image extraction module to be detected, configured to extract a plant instance ROI image corresponding to a first plant from the set of plant instance ROI images; The invasive species identification module is used to perform multi-directional enhanced perception of image features on the plant instance ROI image corresponding to the first plant to determine whether the first plant is an invasive species.

Images