Lossless estimation method and system based on RGB image and computer readable storage medium

Abstract

The invention provides an RGB image-based lossless estimation system, which comprises an image acquisition module, an image preprocessing module, an image segmentation module, a fresh grass segmentation calculation module, a hay and soil segmentation module, a regression model prediction module and an auxiliary data processing module, and the modules are connected and communicated through data streams. Meanwhile, the invention further provides a lossless estimation method based on the RGB image and a computer readable storage medium, the cost is reduced, the efficiency is improved, remarkable advantages are shown in the aspects of precision, adaptability, eco-friendliness and the like, and the method can be widely applied to grassland ecological monitoring and management and has wide application prospects. And reliable technical support is provided for ecological system protection and sustainable development.

Description

Technical Field

[0001] The present invention belongs to the technical fields of computer vision, remote sensing technology, ecological monitoring and agricultural science, and in particular relates to a non-destructive estimation method and system based on RGB images and a computer-readable storage medium. Background Art

[0002] At present, the common technologies for estimating FVC and AGB include multispectral and hyperspectral remote sensing technology, drone remote sensing technology, and ground measurement technology. Each technology has its own advantages and disadvantages, which are mainly reflected in: (1) Multispectral and hyperspectral remote sensing technology. Multispectral remote sensing uses a small amount of wide-band spectral data and is commonly found in Landsat satellites. Hyperspectral remote sensing uses continuous narrow-band data to obtain more detailed spectral information and is commonly used in Hyperion and EnMAP satellites. Using these data, FVC and AGB are estimated by calculating vegetation indices such as NDVI (Normalized Difference Vegetation Index). Disadvantages or shortcomings: High cost: The cost of obtaining hyperspectral data is high, and special software and technical support are required for processing. Limited spatial resolution: Although hyperspectral data provides high spectral resolution, the spatial resolution is low and it is difficult to meet the needs of fine monitoring. Poor timeliness: Satellite data is limited by transit time and weather conditions and cannot achieve high-frequency real-time monitoring. Complex data processing: Hyperspectral data processing is complex and involves a large amount of computing and data storage requirements.

[0003] (2) UAV remote sensing technology. UAVs are equipped with high-resolution cameras and LiDAR (light detection and ranging) sensors, which can obtain high-precision surface data. UAV remote sensing is suitable for small-scale fine monitoring. It provides detailed vegetation structure information through multi-angle and multi-height data collection. Disadvantages or shortcomings: Limited coverage: The flight time and coverage of UAVs are limited, making it difficult to cover large areas. Complex operation: UAV operation requires professional skills, and data processing and analysis are also relatively complex. Affected by weather conditions: UAV flight and data collection are easily affected by weather conditions, such as wind speed, rainfall, etc.

[0004] (3) Ground measurement technology. Ground measurement measures vegetation cover and biomass through field sampling and the use of portable optical instruments. Common methods include fixed-point sample surveys, optical instrument measurements, etc. Disadvantages or shortcomings: Time-consuming and labor-intensive: A large amount of manpower and time are required for field sampling, which is inefficient, especially in measurements of large areas. Destructive sampling: Traditional ground measurements often use destructive sampling methods, which may cause damage to vegetation and soil, affecting the continuous monitoring of ecosystems. Insufficient data coverage: The spatial coverage of ground measurements is limited, and it is difficult to represent the vegetation conditions in a large area.

[0005] In summary, although the above-mentioned technologies can be used to estimate and obtain results, they all have their own shortcomings and deficiencies. Therefore, developing a low-cost and high-efficiency measurement technology is still a problem that needs to be solved urgently. Summary of the invention

[0006] Technical solution: In order to solve the above technical problems, the present invention provides a lossless estimation method based on RGB images, and the specific steps of the method are:

[0007] Step 1: Use an RGB camera to take grassland sample images on the ground, pre-process the collected images including resizing and data standardization, and have the user determine the grassland attributes for future use;

[0008] Step 2: When the processed RGB image is obtained, the U-net deep learning model is used to segment the sample image into two parts: the sample frame and the background image. The morphological dilation operation is performed on the segmented binary image, and the convex hull mask algorithm is used to generate the final mask.

[0009] Step 3: When the grassland image in the cropped sample plot is obtained, for the fresh grass image, the vegetation index is calculated, and the image is divided into the vegetation area and the non-vegetation area by the threshold method to obtain the vegetation pixel ratio. For the hay image, the vegetation index threshold method is used to remove the fresh grass part, and a multi-step image segmentation and classification method is used to achieve fine segmentation of soil and hay, and then the vegetation pixel ratio is calculated;

[0010] Step 4: When the vegetation pixel ratio of the image is obtained, a machine learning regression model is used, with the vegetation pixel ratio as the input feature, to preliminarily calculate the grassland vegetation coverage FVC and aboveground biomass AGB through the input features;

[0011] Step 5: After the preliminary FVC and AGB calculations, the user inputs the auxiliary data module, adds the auxiliary data as input features of the regression model, and calculates the precise FVC and AGB.

[0012] As an improvement, in step 1, the RGB image size is first adjusted from 2048×1536 pixels to 512×512 pixels, and then normalized to map the value of each pixel from [0,255] to [0,1].

[0013] As an improvement, in step 2, the specific method is to first use the U-net deep learning model to perform image segmentation and output a binary image; then perform a morphological dilation operation, a 3×3 square kernel structure element, and gradually scan the image. If there is a white foreground pixel in the area covered by the structure element, all pixels in the area are set as foreground; finally, use the convex hull masking algorithm to apply masking processing to the original RGB image, where the function used for applying masking processing is cv2.bitwise_and().

[0014] As an improvement, the convex hull mask algorithm is specifically as follows: first, contour detection is performed on a binary image generated by morphological dilation, with the foreground being white and the background being black, and the contours of all connected areas are extracted using cv2.findContours(); then, the convex hull is calculated, and the minimum convex polygon of each contour is calculated using cv2.convexHull(); finally, the mask is drawn: these convex hulls are drawn as binary masks using the cv2.drawContours() function.

[0015] As an improvement, in step 3, the specific method for distinguishing fresh grass images from dry grass images is:

[0016] Step 3.1 Vegetation index threshold method to remove fresh grass

[0017] Calculate the CIVE value of each pixel in the image using the following formula: CIVE = 0.441*R-0.811*G+0.385*B+18.787;

[0018] According to the Otsu threshold method, the threshold of CIVE is set to a, and pixels with CIVE values ​​lower than a are identified as fresh grass and removed;

[0019] Step 3.2 TurboPixels superpixel segmentation

[0020] The image was preliminarily segmented using the TurboPixels algorithm, and 500 superpixel regions were generated;

[0021] Step 3.3 Use Sobel operator and Canny edge detection

[0022] Use the Sobel operator to calculate the horizontal and vertical gradients of the image to obtain the edge information of the image; then use the Canny edge detection algorithm to further extract the edge features of the image and set the low threshold and high threshold;

[0023] Step 3.4 Watershed algorithm fine segmentation

[0024] Use the watershed algorithm to process the gradient information of the image. First, the edges detected by Sobel and Canny are used as markers for the watershed algorithm. Then, through the "water flow" process, the image is divided into different areas until all the markers meet to form a watershed.

[0025] Step 3.5 K-means clustering algorithm classification

[0026] The color features and texture features of each image area are extracted, and the image area is divided into two categories, hay and soil, using the K-means clustering algorithm. The number of initial cluster centers and the number of iterations are set to obtain the final hay and soil areas.

[0027] As a technical solution of the present invention, the present invention also provides a lossless estimation system based on RGB images, including an image acquisition module, an image preprocessing module, an image segmentation module, a fresh grass segmentation calculation module, a hay and soil segmentation module, a regression model prediction module, and an auxiliary data processing module, and the modules are connected and communicated through data streams.

[0028] As an improvement, the image acquisition module transmits the collected original image data to the image preprocessing module; the image preprocessing module preprocesses the image data and transmits it to the image segmentation module; the image segmentation module is used to segment and classify the preprocessed image data, and then transmit it to the fresh grass segmentation calculation module and the hay and soil segmentation module; the regression model prediction module constructs a regression model based on the vegetation pixel ratio data output by the fresh grass segmentation calculation module and the hay and soil segmentation module, and performs regression prediction on the vegetation pixel ratio of fresh grass or hay. The auxiliary data processing module processes and introduces auxiliary data and transmits it to the regression model prediction module.

[0029] As an improvement, auxiliary data include average vegetation height and number of plants in the sample plot.

[0030] As a technical solution of the present invention, the present invention also provides a computer-readable storage medium, which stores a computer program that can accurately predict FVC and AGB based on a lossless estimation method based on RGB images. The lossless estimation method based on RGB images is any one of the above methods.

[0031] As an improvement, the program implements the following steps when executed by the processor:

[0032] Step 1: User registration and login, the user registers and logs in to enter the program;

[0033] Step 2: Upload and preprocess images. Users upload images of fresh grass and hay. The image preprocessing module resizes the images and performs standardization of resolution and color channels. Then proceed to the next step.

[0034] Step 3: Image segmentation and calculation

[0035] The fresh grass image enters the fresh grass segmentation calculation module, and the vegetation index threshold method is used to segment the image, and the soil background is removed to distinguish the fresh grass from other parts; the threshold is set using the Otsu threshold algorithm, and the area below a specific value is identified as the fresh grass part and removed, and the vegetation pixel ratio of the fresh grass image is calculated to obtain the segmented image and calculation results; the hay image enters the hay segmentation calculation module, and the image containing soil is selected to identify and segment the hay and soil areas, and the vegetation pixel ratio of the hay image is calculated;

[0036] Step 4: Proportion of vegetation pixels and preliminary calculation of FVC and AGB

[0037] The vegetation pixel ratio obtained in step 3 is input into the regression model prediction module to obtain preliminary FVC and AGB values;

[0038] Step 5: Manual input and accurate calculation of auxiliary data

[0039] After the actual data measured on site, including the average vegetation height and the number of plants in the sample plot, were used as auxiliary data input to the independent regression model prediction modules for fresh grass image processing and hay image processing, the parameters and results of the models were adjusted to obtain accurate FVC and AGB values, respectively.

[0040] Beneficial effects: The lossless estimation method based on RGB images proposed in the present invention has significant application prospects and practical value in terms of reducing costs, improving efficiency, achieving lossless sampling and high spatial resolution. Compared with the prior art, the specific advantages are as follows:

[0041] (1) Low cost and high efficiency: Using ordinary RGB cameras to collect images reduces the cost of equipment and data acquisition, while improving the efficiency of data processing through deep learning models.

[0042] (2) High spatial resolution: Through deep learning models such as U-net, high-precision image segmentation can be achieved, which is suitable for small-scale fine monitoring.

[0043] (3) Fast and accurate: Combining deep learning and image processing technology, it can quickly and accurately estimate FVC and AGB, which is suitable for rapid surveys of large areas.

[0044] (4) Classification and processing of dry and fresh grass: Multi-step image segmentation and classification technology solves the problems of spectral feature differences and model adaptability, reduces noise interference, and improves estimation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1It is a schematic diagram of the logical principle of the lossless estimation system based on RGB image of the present invention.

[0046] Figure 2 It is a principle diagram of the lossless estimation method based on RGB image of the present invention.

[0047] Figure 3 This is the result diagram of the image segmentation module of the present invention, 3a is the original image collected, 3b is the image predicted by the U-net network, 3c is the filled mask image, 3d is the preliminary mask image obtained by the convex hull mask algorithm, 3e is the final mask image synthesized, and 3f is the cropped sample image.

[0048] Figure 4 This is a result diagram of the hay and soil segmentation module of the present invention.

[0049] Figure 5 This is a schematic diagram of logging in step 1 implemented by a program in a computer-readable storage medium of the present invention.

[0050] Figure 6 This is a schematic diagram of the picture uploading interface in step 2 implemented by the program in the computer-readable storage medium of the present invention.

[0051] Figure 7 This is a schematic diagram of step 2 of uploading fresh grass pictures implemented by a program in a computer-readable storage medium of the present invention.

[0052] Figure 8 This is a schematic diagram of a fresh grass picture of step 2 implemented by a program in a computer-readable storage medium of the present invention.

[0053] Fig. 9 A schematic diagram of fresh grass image segmentation and calculation in step three implemented by a program in a computer-readable storage medium of the present invention.

[0054] Fig.10 This is a result viewing interface diagram of the program implemented in the computer-readable storage medium of the present invention.

[0055] Fig.11 This is a graph showing the results of preliminary estimation of FVC and AGB of fresh grass images in step 4 implemented by the program in the computer-readable storage medium of the present invention.

[0056] Fig.12 A schematic diagram of inputting auxiliary data for step five implemented by a program in a computer-readable storage medium of the present invention.

[0057] Fig.13 This is a graph showing the accurate FVC and AGB estimation results of fresh grass images implemented by the program in the computer-readable storage medium of the present invention.

[0058] Fig.14 This is a schematic diagram of a hay picture of step 2 implemented by a program in a computer-readable storage medium of the present invention.

[0059] Fig.15 The schematic diagram of the interface including the soil image is selected for step three implemented by the program in the computer-readable storage medium of the present invention.

[0060] Fig.16 This is a graph showing the results of preliminary estimation of FVC and AGB of hay images in step 4 implemented by the program in the computer-readable storage medium of the present invention.

[0061] Fig.17 A schematic diagram of an interface for selecting auxiliary data and adjusting a regression model implemented by a program in a computer-readable storage medium of the present invention.

[0062] Fig.18 This is a graph showing the accurate FVC and AGB estimation results of fresh grass images implemented by the program in the computer-readable storage medium of the present invention. DETAILED DESCRIPTION

[0063] The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0064] See the present invention Figure 1 As shown, the lossless estimation system based on RGB image of the present invention includes an image acquisition module, an image preprocessing module, an image segmentation module, a fresh grass segmentation calculation module, a hay and soil segmentation module, a regression model prediction module, and an auxiliary data processing module. The modules are connected and communicated through data streams.

[0065] The image acquisition module transmits the collected original image data to the image preprocessing module; the image preprocessing module preprocesses the image data and transmits it to the image segmentation module; the image segmentation module is used to segment and classify the preprocessed image data, and then transmit it to the fresh grass segmentation calculation module and the hay and soil segmentation module; the regression model prediction module constructs a regression model based on the vegetation pixel ratio data output by the fresh grass segmentation calculation module and the hay and soil segmentation module, and performs regression prediction on the vegetation pixel ratio of fresh grass or hay. The auxiliary data processing module processes and introduces auxiliary data and transmits it to the regression model prediction module.

[0066] In the present invention, the image acquisition module uses an RGB camera, such as a smartphone camera, to acquire grassland sample images, ensuring that the images are clear and cover the entire sample area. Preferably, the present invention stores the acquired images in a local or cloud storage space for subsequent processing.

[0067] Furthermore, the image preprocessing module of the present invention resizes the collected images to 512×512 pixels to meet the input requirements of the U-net model. Preferably, the present invention stores the adjusted images in a processing server for subsequent processing.

[0068] In the present invention, the image segmentation module uses the trained U-net model to segment the preprocessed image, and divides the image into two parts: a sample frame and a background. Preferably, the present invention uses a morphological dilation operation to fill in small holes or disconnected parts that appear during the sample frame prediction process, and uses a convex hull mask algorithm to generate a final mask to ensure the integrity and accuracy of the sample frame and sample quadrat parts.

[0069] Furthermore, in the present invention, when performing classification and segmentation processing, a fresh grass segmentation calculation module and a dry grass and soil segmentation module are set, wherein the fresh grass segmentation calculation module is used to calculate the vegetation index (CIVE) and generate a corresponding grayscale image. Specifically, the Otsu threshold method is used to segment the grayscale image, and the image is divided into vegetation area and non-vegetation area, and the vegetation pixel ratio is calculated, that is, the ratio of the number of pixels in the vegetation area to the total number of pixels.

[0070] In the present invention, the hay and soil segmentation module has the following specific methods: first, for the hay data set, the vegetation index threshold method is used to remove the fresh grass part, and only the hay and soil parts are retained; then, the TurboPixels superpixel segmentation algorithm is used to perform preliminary segmentation on the image, and the Sobel operator and Canny edge detection are combined for fine segmentation. The color features and texture features of the image area are extracted, and the image area is classified using the K-means clustering algorithm. Finally, the proportion of vegetation pixels after classification is calculated.

[0071] Furthermore, the regression model prediction module of the present invention uses a random forest regression model to construct regression models of FVC and AGB based on features such as the vegetation pixel ratio, and performs regression prediction on the input feature, i.e., the vegetation pixel ratio of fresh grass or hay, to obtain preliminarily calculated FVC and AGB.

[0072] Furthermore, the auxiliary data processing module of the present invention processes and introduces auxiliary data, including vegetation density and average plant height, into the regression model to construct the regression model. The present invention improves the prediction performance of the regression model by introducing auxiliary data to obtain more accurate FVC and AGB.

[0073] In the present invention, in the image segmentation module, for the precise segmentation of the sample plot and the background, a combination logic algorithm using the U-net deep learning model combined with the morphological dilation operation and the convex hull mask algorithm is introduced. When calculating the percentage of hay pixels in the hay and soil segmentation module, it is recommended to use the logic of first segmentation and then classification, and a multi-step image segmentation and classification algorithm is recommended. When the auxiliary data processing module and the regression model prediction module estimate FVC and AGB, a single pixel percentage is introduced to obtain a preliminary prediction result, and then auxiliary data is added at any time to obtain an accurate prediction result, so as to meet the data processing methods with different needs.

[0074] See the present invention Figure 2 As shown, a lossless estimation method based on RGB image is proposed. The specific steps are:

[0075] Step 1: Use an RGB camera to take grassland sample images on the ground, pre-process the collected images including resizing and data standardization, and have the user determine the grassland attributes for future use;

[0076] Step 2: When the processed RGB image is obtained, the U-net deep learning model is used to segment the sample image into two parts: the sample frame and the background image. The morphological dilation operation is performed on the segmented binary image, and the convex hull mask algorithm is used to generate the final mask.

[0077] Step 3: When the grassland image in the cropped sample plot is obtained, for the fresh grass image, the vegetation index is calculated, and the image is divided into the vegetation area and the non-vegetation area by the threshold method to obtain the vegetation pixel ratio. For the hay image, the vegetation index threshold method is used to remove the fresh grass part, and a multi-step image segmentation and classification method is used to achieve fine segmentation of soil and hay, and then the vegetation pixel ratio is calculated;

[0078] Step 4: When the vegetation pixel ratio of the image is obtained, a machine learning regression model is used, with the vegetation pixel ratio as the input feature, to preliminarily calculate the grassland vegetation coverage FVC and aboveground biomass AGB through the input features;

[0079] Step 5: After the preliminary FVC and AGB calculations, the user inputs the auxiliary data module, adds the auxiliary data as input features of the regression model, and calculates the precise FVC and AGB.

[0080] In the present invention, the pain point of accurate segmentation of sample frame and background in grassland sample image is addressed by combining U-net deep learning model and traditional image processing technology. Through multi-level feature extraction, morphological dilation operation and convex hull mask algorithm, multi-step and multi-technique combination logic, high-precision sample area segmentation is achieved.

[0081] In order to enable those skilled in the art to better understand the present invention, the technical solution of the present invention is further described below in conjunction with the accompanying drawings and implementation examples.

[0082] Example 1

[0083] The above non-destructive estimation method is introduced in detail in this embodiment, specifically:

[0084] Step 1: Data input and preprocessing

[0085] The data input is the grassland sample RGB image collected by the image acquisition module, such as Figure 3 (a) shows that the image contains an irregularly shaped white box, which is used to mark the sampling area on the actual ground, and the background is natural grassland.

[0086] Preprocessing includes resizing and data standardization. Specifically, resizing is to adjust the original image size from 2048×1536 pixels to 512×512 pixels to meet the input requirements of the U-net model. This adjustment ensures the consistency of the model input and improves the efficiency and effect of model processing.

[0087] Data normalization is to normalize the RGB values ​​of the image, mapping the value of each pixel from [0,255] to [0,1] to eliminate the lighting differences between images and ensure the consistency of the input data and the stability of the model training.

[0088] Step 2: Use the U-net model to segment the sample frame and background, perform morphological dilation operations, convex hull masking algorithm processing, and apply masking processing

[0089] In this embodiment, the U-net model includes two parts, an encoder and a decoder, which are connected by a jump connection. Specifically, the jump connection connects the feature maps of the same resolution of the encoder and the decoder to retain more detail information and improve the accuracy of segmentation.

[0090] Furthermore, the encoder part is composed of multiple convolutional layers and maximum pooling layers, which are used to gradually extract features of the image while reducing the spatial resolution. Preferably, the convolutional layer uses a 3×3 convolution kernel to scan the image and extract feature information, similar to applying a filter to detect specific patterns or features in the image. The maximum pooling layer is a downsampling method that reduces the size of the image by taking the maximum value within the pooling window, reduces the computational cost, and extracts the main features.

[0091] The decoder part restores the original resolution of the image through deconvolution (also called transposed convolution or upsampling).

[0092] In this embodiment, the morphological dilation operation is to expand the pixels in the white area of ​​the binary image to make it "dilate". This operation uses a 3×3 square kernel structure element to gradually scan the image. If there is a foreground pixel (white) in the area covered by the structure element, all pixels in the area are set as foreground.

[0093] See also Figure 3 As shown in (d), the morphological dilation operation expands the sample frame area and fills the small holes or breaks that may appear during the segmentation process of the U-net model, ensuring the integrity and coherence of the sample area and making the segmentation result more accurate.

[0094] The convex hull mask algorithm in this embodiment has the following specific steps:

[0095] First, binary image generation: The binary image generated by morphological dilation has a white foreground (sample frame) and a black background;

[0096] Secondly, contour detection: use cv2.findContours() to extract the contours of all connected areas.

[0097] Then, calculate the convex hull: use cv2.convexHull() to calculate the smallest convex polygon for each contour.

[0098] Finally, draw the mask: use cv2.drawContours() function to draw these convex hulls as binary masks.

[0099] In this embodiment, mask processing is applied by using the cv2.bitwise_and() function to apply the generated convex hull mask to the original RGB image, retaining only the sample area and excluding the background, such as Figure 3 (f) as shown.

[0100] Step 3 Combinatorial logic algorithm for accurate segmentation of hay and soil

[0101] In order to solve the problem of accurate segmentation of hay and soil in grassland sample images, the present invention combines a variety of image processing technologies and realizes high-precision segmentation and classification through the combination logic of multiple steps and multiple technologies. The specific steps are:

[0102] Step 3.1 Vegetation index threshold method to remove fresh grass

[0103] The vegetation index calculation method used in the present invention uses CIVE (Color Index of Vegetation Extraction) to distinguish fresh grass from other parts. Furthermore, CIVE is calculated by combining the red, green, and blue channel data of the image, and the formula is as follows:

[0104] CIVE=0.441*R-0.811*G+0.385*B+18.787

[0105] According to the CIVE value, a threshold is set using the Otsu threshold algorithm, and the area with a CIVE value lower than the threshold is identified as fresh grass and removed. This step removes the fresh grass part, providing clean data input for the subsequent segmentation of hay and soil, ensuring the purity of the initial data and the accuracy of subsequent processing.

[0106] Step 3.2 Preliminary division of hay and soil

[0107] The processed image is preliminarily segmented using the TurboPixels algorithm. The TurboPixels algorithm divides the image into multiple superpixel regions of uniform size by minimizing energy. The pixels in each superpixel region have similar color and texture features, ensuring the fine granularity and regional consistency of the segmentation results. This step gives each part of the image a unique region ID, providing good basic data for subsequent fine segmentation.

[0108] Step 3.3 Collect edge information

[0109] After the initial segmentation, the Sobel operator is used to calculate the horizontal and vertical gradient strength of the image to preliminarily extract the edge information of the image. The Sobel operator obtains the gradient strength through convolution operation to help identify the edge of the image. On this basis, the Canny edge detection algorithm is used to further accurately extract edge features. The Canny algorithm detects strong and weak edges by setting high and low double thresholds to ensure the accuracy and robustness of edge detection. This step outputs accurate edge data, providing reliable boundary information for subsequent fine segmentation.

[0110] Step 3.4 Fine division of hay and soil

[0111] The watershed algorithm is introduced in the fine segmentation stage. By processing the gradient information of the image, the image is regarded as a topographic map. The areas with large gradients are regarded as "ridge lines", and this information is used to simulate the "water flow" process and segment the image into different areas. Using the detected edges as markers, the watershed algorithm expands from the center outward until the boundaries meet, forming a watershed, and finely segmenting the image into different areas. This step significantly improves the accuracy of hay and soil segmentation and ensures the segmentation effect under complex boundary conditions.

[0112] Step 3.5 Classification

[0113] Finally, the color and texture features of each image region are extracted, and the K-means clustering algorithm is used to classify the image regions. The image is divided into K cluster centers based on the extracted features. K cluster centers are randomly selected in the initial stage, and then each pixel is assigned to the nearest cluster center through iterative optimization, and the position of the cluster center is updated until convergence. The final classification result effectively distinguishes the hay and soil areas, ensures the accurate identification of different areas, and further improves the accuracy of FVC (Fractional Vegetation Cover) and AGB (Above-Ground Biomass) estimation.

[0114] Step 4: Make a preliminary prediction based on the vegetation pixel ratio of the above image. This step does not rely on additional auxiliary data. It mainly uses the pixel information directly extracted from the image to quickly calculate the preliminary FVC and AGB values ​​through the regression model. This method can quickly provide a rough estimate of grassland vegetation coverage and biomass, and is suitable for scenarios that require rapid assessment.

[0115] Step 5, introduce auxiliary data processing. The auxiliary data includes the average vegetation height and the number of plants in the sample plot manually input from the field measurement. After the auxiliary data is analyzed and processed, it is used to adjust the parameters of the regression model.

[0116] At this time, the regression model not only relies on the proportion of vegetation pixels in the image, but also combines more detailed auxiliary data, thereby improving the accuracy of FVC and AGB prediction.

[0117] Example 2

[0118] The present invention introduces step 2 of the above non-destructive estimation method in detail, and the specific steps are:

[0119] Step 2.1: Separate the sample frame and background

[0120] The model architecture is as follows:

[0121] The U-net model is a deep learning model based on convolutional neural networks, using an encoder-decoder structure. The encoder part consists of multiple convolutional layers and pooling layers, which gradually extracts the features of the image while reducing the spatial resolution. The decoder part restores the original resolution of the image through deconvolution (also called transposed convolution or upsampling).

[0122] The encoder part gradually extracts the feature information of the grassland sample image and generates a low-resolution feature map. The decoder part gradually restores the size of the image and generates a high-resolution feature map. The feature maps of the same size in the encoder and decoder are fused through jump connections to ensure the retention of the sample frame and background details and overall information in the grassland sample image. The output segmented binary image distinguishes the sample frame and background. Among them, the sample frame part in the binary image is set to white, and the background part is set to black. So far, the specific size and position of the sample frame in the image are obtained.

[0123] The U-net model is used for image segmentation, and the segmentation result is a binary image as the final output image, in which the sample frame area is white (pixel value 255) and the background is black (pixel value 0).

[0124] Step 2.2: Fill in the frame

[0125] For the generated binary image, this system introduces a morphological dilation operation to expand the sample frame part, that is, the white pixel area, fill small holes and breaks, improve the integrity of the deep learning model segmentation results, and repair possible prediction errors. Using the morphological dilation image processing operation, the pixels in the white area of ​​the binary image are "expanded" by expanding them. This operation uses a structural element (3×3 square kernel) to scan the image step by step. If there is a foreground pixel (white) in the area covered by the structural element, all pixels in the area are set to foreground.

[0126] Step 2.3: Generate sample mask

[0127] So far, we have obtained a binary image after repair and filling, in which the target area frame is represented by white pixel value 255 and the background area is represented by black pixel value 0. However, the specific sample square inside the sample square cannot be obtained, and it is difficult to obtain a complete and accurate sample square area when the sample square is irregular in shape or the partially blocked sample square is not closed. Therefore, the present invention designs a convex hull mask algorithm.

[0128] Example 3

[0129] The present invention uses the convex hull mask algorithm in step 2.3 of the non-destructive estimation method in the above embodiment 2 as follows:

[0130] Step 2.3.1, use the binary image A obtained by the morphological dilation operation as the input of the convex hull mask algorithm, use the image processing algorithm to detect the contours in the binary image, use cv2.findContours() to extract the contours of all connected areas, extract the boundary points of all connected areas in the image, record each boundary point as a point set, and output the contour list.

[0131] Step 2.3.2, for each detected contour, use the convex hull algorithm to calculate its convex hull, which is the smallest convex polygon containing all the points of the contour. Use the cv2.convexHull() function to calculate the convex hull of each contour to ensure that the polygon surrounding the foreground object is convex, that is, the interior of the polygon does not contain any concave area.

[0132] Step 2.3.3, use the calculated convex hull point set as the outer contour to create a new binary image B. In this mask image, fill the area inside the convex hull with white pixel value 255, and keep the area outside the convex hull with black pixel value 0.

[0133] In step 2.3.4, the new binary image B is ANDed with the binary image A obtained after morphological dilation, that is, the white parts are replaced with black parts, and the new binary image B after replacement is used as the final mask and applied to the original image to retain the sample area and exclude the background to obtain the complete sample area.

[0134] Example 4

[0135] In the present invention, the specific method for dividing hay and soil in step 3 of the non-destructive estimation method in Example 1 is as follows:

[0136] 4.1 Vegetation index threshold method to remove fresh grass

[0137] Steps:

[0138] (1) Calculate the CIVE (Color Index of Vegetation Extraction) value of each pixel in the image. The formula is as follows: CIVE = 0.441*R-0.811*G+0.385*B+18.787

[0139] (2) According to the Otsu threshold method, the CIVE threshold is set to a, and pixels with CIVE values ​​lower than a are identified as fresh grass and removed.

[0140] Removal effect: The image after removing fresh grass is as follows Figure 3 (b) shows that the remaining area is mainly hay and soil.

[0141] 2. TurboPixels superpixel segmentation

[0142] Operation steps: Use TurboPixels algorithm to perform preliminary segmentation on the image and set to generate 500 superpixel regions. The pixels in each superpixel region have similar color and texture features.

[0143] 3.Sobel operator and Canny edge detection

[0144] Steps:

[0145] (1) Use the Sobel operator to calculate the horizontal and vertical gradients of the image and obtain the edge information of the image.

[0146] (2) Use the Canny edge detection algorithm to further extract the edge features of the image and set a low threshold and a high threshold, for example, setting the low threshold to 50 and the high threshold to 150.

[0147] 4. Watershed algorithm fine segmentation

[0148] Steps:

[0149] (1) Use the watershed algorithm to process the gradient information of the image. First, the edges detected by Sobel and Canny are used as markers for the watershed algorithm.

[0150] (2) Through the "water flow" process, the image is divided into different areas until all marked points meet to form a watershed.

[0151] 5. K-means clustering algorithm classification

[0152] Steps:

[0153] (1) Extract the color features and texture features of each image region.

[0154] (2) Use the K-means clustering algorithm to divide the image area into two categories (hay and soil). Set the number of initial cluster centers to 2 and the number of iterations to 100.

[0155] Classification effect: The final classification result is as follows Figure 3 As shown in (c), the hay and soil areas are effectively distinguished.

[0156] This embodiment aims to demonstrate the innovative image processing process and its application in grassland hay and soil segmentation through detailed step-by-step instructions. The present invention makes important improvements on the prior art and can achieve high-precision image segmentation and classification in complex natural scenes.

[0157] The present invention designs a modular prediction system that introduces auxiliary data. Based on the preliminary estimation based on the percentage of vegetation pixels, users can add auxiliary data (such as average vegetation height and number of plants) at any time for accurate prediction to meet the requirements for data accuracy in different scenarios.

[0158] In the image processing flow of the software, when preliminarily estimating FVC (Fractional Vegetation Cover) and AGB (Above-Ground Biomass), the system first makes a preliminary prediction based on the vegetation pixel ratio of the image. This step does not rely on additional auxiliary data, but mainly uses pixel information directly extracted from the image to quickly calculate the preliminary FVC and AGB values ​​through a regression model. This method can quickly provide a rough estimate of grassland vegetation coverage and biomass, and is suitable for scenarios that require rapid assessment.

[0159] The innovation of this design is that the system can choose different data processing methods according to different needs and scenarios. In time-sensitive situations, users can use only image data for quick evaluation; when high-precision analysis is required, users can add auxiliary data to obtain more accurate results. This design not only increases the flexibility of the system, but also meets the diverse needs of different users, thereby improving the applicability and user experience of the software.

[0160] Through this modular prediction method, the system can provide the dual capabilities of rapid response and accurate analysis, ensuring reliable data support in various application scenarios. This feature significantly improves the practicality and market competitiveness of the system.

[0161] Based on the significant differences in spectral characteristics and physical properties between hay and fresh grass. Fresh grass usually has a higher moisture content and a more obvious green feature, while hay often lacks moisture and exhibits different color and texture characteristics. In order to further improve the prediction accuracy, in the design of the present invention, the image processing process of hay and fresh grass is clearly separated. This design strategy significantly improves the accuracy of FVC (Fractional Vegetation Cover) and AGB (Above-Ground Biomass) estimation.

[0162] In the present invention, fresh grass image processing is first processed by vegetation index threshold method to remove soil background. This process mainly uses vegetation indexes such as CIVE (Color Index of Vegetation Extraction) to segment fresh grass areas through specific thresholds. The system only makes preliminary predictions based on the proportion of vegetation pixels, and users can subsequently choose to add auxiliary data (such as average vegetation height and number of plants) to obtain accurate prediction results.

[0163] The hay image processing is also independently processed by the system of the present invention. Due to the obvious differences in the characteristics of hay and soil, the system uses a multi-step segmentation algorithm for segmentation, and then further refines it by manually selecting images containing soil to ensure accurate separation of hay and soil. Subsequently, the FVC and AGB are initially estimated through regression model prediction, and users can also add auxiliary data for accurate prediction.

[0164] As another specific technical solution of the present invention, the present invention provides a computer-readable storage medium, which stores a computer program for accurately predicting FVC and AGB by a non-destructive estimation method based on RGB images, which is the non-destructive estimation method based on RGB images mentioned above. The program is preferably named Eco-treasure in the present invention, and when the program is executed by a processor, the following steps are implemented:

[0165] Step 1: User registration and login, see Figure 5 As shown, the user registers and logs in to enter the program, and can choose to proceed to the next step after completion;

[0166] Step 2: Upload image and preprocess, see Figure 6 As shown, users upload images of fresh grass and hay, which are processed by the image preprocessing module. Figure 7-8 As shown, the image is resized and the resolution and color channels are standardized; proceed to the next step;

[0167] Step 3: Image segmentation and calculation

[0168] The fresh grass image enters the fresh grass segmentation calculation module, see Fig. 9 As shown in Figure 1, the vegetation index threshold method is used to segment the image, and the soil background is removed to distinguish fresh grass from other parts; the Otsu threshold algorithm is used to set the threshold, and the area below a specific value is identified as the fresh grass part and removed, and the vegetation pixel ratio of the fresh grass image is calculated to obtain the segmented image and the calculation results, see Fig.12 As shown;

[0169] The hay image enters the hay segmentation calculation module. For uploading hay images, see Fig.14 As shown, after the image enters preprocessing, the program displays multiple images for users to choose from, see Fig.15 , select the image containing soil, identify and segment the hay and soil areas, and calculate the vegetation pixel ratio of the hay image;

[0170] Step 4: Proportion of vegetation pixels and preliminary FVC and AGB estimation

[0171] The vegetation pixel ratio obtained in step 3 is input into the regression model prediction module to obtain preliminary FVC and AGB values. Fig.11 , Fig.16 As shown;

[0172] Step 5: Manual input and accurate calculation of auxiliary data

[0173] For hay images and fresh grass images, the actual data measured on site, including the average vegetation height and the number of plants in the sample plot, are independently input as auxiliary data to the regression model prediction module. Fig.12 , Fig.17 As shown, the parameters and results of the model were adjusted to obtain accurate FVC and AGB values. Fig.13 , Fig.18 These results provide users with detailed grassland vegetation coverage and biomass information to support ecological monitoring and management decisions.

[0174] Through the above complete process, the software realizes a full set of functions from image acquisition, preprocessing, segmentation to the final vegetation coverage and biomass estimation, providing a scientific and accurate ecological monitoring tool.

[0175] In summary, the method for nondestructive estimation of grassland vegetation coverage and aboveground biomass based on RGB images proposed in this study not only reduces costs and improves efficiency, but also shows significant advantages in accuracy, adaptability and eco-friendliness. This method can be widely used in grassland ecological monitoring and management, providing reliable technical support for ecosystem protection and sustainable development.

[0176] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A lossless estimation method based on RGB images, characterized in that: The specific steps of the method are: Step 1: Use an RGB camera to take grassland sample images on the ground, pre-process the collected images including resizing and data standardization, and have the user determine the grassland attributes for future use; Step 2: When the processed RGB image is obtained, the U-net deep learning model is used to segment the sample image into two parts: the sample frame and the background image. The morphological dilation operation is performed on the segmented binary image, and the convex hull mask algorithm is used to generate the final mask. Step 3: When the grassland image in the cropped sample plot is obtained, for the fresh grass image, the vegetation index is calculated, and the image is divided into the vegetation area and the non-vegetation area by the threshold method to obtain the vegetation pixel ratio; for the hay image, the vegetation index threshold method is used to remove the fresh grass part, and a multi-step image segmentation and classification method is used to achieve fine segmentation of soil and hay, and then the vegetation pixel ratio is calculated; Step 4: When the vegetation pixel ratio of the image is obtained, a machine learning regression model is used, with the vegetation pixel ratio as the input feature, to preliminarily calculate the grassland vegetation coverage FVC and aboveground biomass AGB through the input features; Step 5: After the preliminary calculation of FVC and AGB, the user inputs the auxiliary data module, adds the auxiliary data as the input features of the regression model, and calculates the accurate FVC and AGB.

2. The lossless estimation method based on RGB image according to claim 1, characterized in that: In step 1, the RGB image size is first adjusted from 2048×1536 pixels to 512×512 pixels, and then normalized to map the value of each pixel from [0,255] to [0,1].

3. The lossless estimation method based on RGB image according to claim 1, characterized in that: In step 2, the specific method is to first use the U-net deep learning model to perform image segmentation and output a binary image; then perform a morphological dilation operation, a 3×3 square kernel structure element, and gradually scan the image. If there is a white foreground pixel in the area covered by the structure element, all pixels in the area are set as foreground; finally, use the convex hull masking algorithm to apply masking processing to the original RGB image, where the function used for applying masking processing is cv2.bitwise_and().

4. The lossless estimation method based on RGB image according to claim 3, characterized in that: The convex hull mask algorithm is specifically as follows: first, contour detection is to extract the contours of all connected areas using cv2.findContours() on the binary image generated by morphological dilation, with the foreground being white and the background being black; Then, calculate the convex hull and calculate the minimum convex polygon of each contour using cv2.convexHull(); finally, draw the mask: use cv2.drawContours() function to draw these convex hulls as binary masks.

5. The lossless estimation method based on RGB image according to claim 1, characterized in that: In step 3, the specific method for distinguishing fresh grass images from dry grass images is: Step 3.1 Vegetation index threshold method to remove fresh grass Calculate the CIVE value of each pixel in the image using the following formula: CIVE=0.441*R-0.811*G+0.385*B+18.787; According to the Otsu threshold method, the threshold of CIVE is set to a, and pixels with CIVE values ​​lower than a are identified as fresh grass and removed; Step 3.2 TurboPixels superpixel segmentation The image was preliminarily segmented using the TurboPixels algorithm, and 500 superpixel regions were generated; Step 3.3 Use Sobel operator and Canny edge detection Use the Sobel operator to calculate the horizontal and vertical gradients of the image to obtain the edge information of the image; then use the Canny edge detection algorithm to further extract the edge features of the image and set the low threshold and high threshold; Step 3.4 Watershed algorithm fine segmentation Use the watershed algorithm to process the gradient information of the image. First, the edges detected by Sobel and Canny are used as the marking points of the watershed algorithm. Then, through the "water flow" process, the image is divided into different areas until all the marking points meet to form a watershed. Step 3.5 K-means clustering algorithm classification The color features and texture features of each image area are extracted, and the image area is divided into two categories, hay and soil, using the K-means clustering algorithm. The number of initial cluster centers and the number of iterations are set to obtain the final hay and soil areas.

6. A lossless estimation system based on RGB images, characterized in that: It includes an image acquisition module, an image preprocessing module, an image segmentation module, a fresh grass segmentation calculation module, a hay and soil segmentation module, a regression model prediction module, and an auxiliary data processing module. The modules are connected and communicated through data streams.

7. The lossless estimation system based on RGB image according to claim 6, characterized in that: The image acquisition module transmits the collected original image data to the image preprocessing module; the image preprocessing module preprocesses the image data and transmits it to the image segmentation module; the image segmentation module is used to segment and classify the preprocessed image data, and then transmit it to the fresh grass segmentation calculation module and the hay and soil segmentation module; the regression model prediction module constructs a regression model based on the vegetation pixel ratio data output by the fresh grass segmentation calculation module and the hay and soil segmentation module, and performs regression prediction on the vegetation pixel ratio of fresh grass or hay. The auxiliary data processing module processes and introduces auxiliary data and transmits it to the regression model prediction module.

8. The lossless estimation system based on RGB image according to claim 6 or 7, characterized in that: Auxiliary data include average vegetation height and number of plants in the sample plot.

9. A computer-readable storage medium storing a computer program for accurately predicting FVC and AGB using a lossless estimation method based on RGB images, characterized in that: The lossless estimation method based on RGB image is any one of claims 1-5.

10. The computer-readable storage medium according to claim 6, wherein: When the program is executed by the processor, the following steps are implemented: Step 1: User registration and login, the user registers and logs in to enter the program; Step 2: Upload and preprocess images. Users upload images of fresh grass and hay. The image preprocessing module resizes the images and performs standardization of resolution and color channels. Then proceed to the next step. Step 3: Image segmentation and calculation The fresh grass image enters the fresh grass segmentation calculation module, and the vegetation index threshold method is used to segment the image, and the soil background is removed to distinguish the fresh grass from other parts. The Otsu threshold algorithm is used to set the threshold, and the area below a specific value is identified as the fresh grass part and removed. The vegetation pixel ratio of the fresh grass image is calculated to obtain the segmented image and the calculation result. The hay image enters the hay segmentation calculation module, selects the image containing soil to identify and segment the hay and soil areas, and calculates the vegetation pixel ratio of the hay image; Step 4: Proportion of vegetation pixels and preliminary calculation of FVC and AGB The vegetation pixel ratio obtained in step 3 is input into the regression model prediction module to obtain preliminary FVC and AGB values; Step 5: Manual input and accurate calculation of auxiliary data After the actual data measured on site, including the average vegetation height and the number of plants in the sample plot, were used as auxiliary data input to the independent regression model prediction modules for fresh grass image processing and hay image processing, the parameters and results of the models were adjusted to obtain accurate FVC and AGB values, respectively.

Images