Farming agricultural machine operation quality detection method based on unmanned aerial vehicle

Abstract

The invention discloses a farming agricultural machine operation quality detection method based on an unmanned aerial vehicle, and relates to the field of farming agricultural machine operation quality detection, in particular to a farming agricultural machine operation quality detection method. The invention aims to solve the problems of high labor cost consumption and low efficiency in the detection task of the flatness of the land ploughed by a rotary cultivator, and provides the method for detecting the operation quality of the farming agricultural machine based on the unmanned aerial vehicle. The farming agricultural machine operation quality detection method based on the unmanned aerial vehicle comprises the following steps: S1, obtaining RGB image data; s2, processing the image data obtained in the step S1 to obtain processed data; s3, based on the processed data obtained in the step S2, obtaining filtered three-dimensional point cloud data Plidar; and S4, calculating the soil crushing rate and the land flatness based on the step S3.

Description

Technical Field

[0001] The invention relates to the field of tillage agricultural machinery operation quality detection, and in particular to a tillage agricultural machinery operation quality detection method. Background Art

[0002] With the advancement of agricultural modernization, the level of agricultural mechanization has been continuously improved, and the widespread use of tillage machinery such as rotary tillers has greatly improved the efficiency of land cultivation. The quality of its operation can greatly affect the subsequent transplanting and seedling raising process, and thus affect the yield of farmland. Therefore, how to accurately evaluate the quality of tillage machinery operations has become a major proposition in modern agricultural production. Traditional tillage quality assessment methods mostly rely on manual inspection or simple mechanical measurement tools, which have problems such as inaccurate measurement and low work efficiency, and are difficult to meet the needs of modern agricultural production for efficient and accurate quality assessment.

[0003] In recent years, with the rapid development of UAV technology, LiDAR technology and computer vision technology, UAV-based automated detection methods have gradually become a research hotspot. UAVs have been widely used in ground condition monitoring and modeling due to their flexibility and efficiency. In farmland environments, UAVs can carry out precise measurements by carrying high-precision sensors and quickly obtain a wide range of post-cultivation surface data. As a high-precision ranging technology, LiDAR can measure ground height information by emitting laser beams and model the surface, with the characteristics of high resolution and high precision. In the existing technology, the combination of LiDAR and computer vision is usually used to evaluate scenes such as highway flatness, which means that it also has application prospects in field environments, providing technical potential for combining LiDAR with vision to evaluate post-cultivation land conditions.

[0004] In agricultural production, the evaluation of the quality of agricultural machinery operations on cultivated land relies on a large number of people going down to the fields to manually measure the properties of the land. While the efficiency is low, the need to manually select measurement points can easily lead to low coverage and difficult to ensure accuracy. As for the drone field patrol scheme, the difference in land quality makes it impossible to survey and map the key areas of concern based on the fixed flight path, thus reducing efficiency. At the same time, the traditional manned drone field patrol also causes a waste of human resources. The topography and soil conditions of each piece of land are different, so it is necessary for the drone to be able to adjust the flight path in real time according to environmental changes and ground features. This requires a system that can dynamically generate flight paths, especially in complex and irregular farmland environments, and can flexibly adjust the path according to real-time image data and sensor information to ensure that each piece of cultivated land area can be accurately measured. Therefore, a method is needed that can obtain potential measurement point information based on drone vision, dynamically adjust the flight path, and enable the drone to focus on locations with poor operation quality during the low-flying measurement phase.

[0005] In the prior art, the evaluation of the surface flatness after tillage mainly relies on manual measurement or traditional ground survey equipment. Although these methods have relatively high accuracy, they generally have problems such as high labor costs, cumbersome operations, and large time consumption, and it is difficult to meet the requirements of fine and large-scale agricultural production for efficient measurement. In large-area farmland, these disadvantages are particularly prominent. With the development of TOF (Time-of-Flight) laser ranging sensor technology, TOF laser ranging sensors can quickly obtain ground distance data due to their high precision and high efficiency characteristics, providing a basis for the evaluation of surface flatness. By analyzing the data collected by the TOF laser ranging sensor, the height information of the ground can be accurately extracted, and then the flatness of the surface can be evaluated.

[0006] Therefore, with the continuous expansion of the scale of agricultural production and the improvement of the requirements for refined management, there is an urgent need for a new method and equipment that can efficiently and accurately measure the surface flatness after rotary tillage, making up for the deficiencies in operation efficiency in the prior art. Summary of the Invention

[0007] The purpose of the present invention is to solve the problems of high labor cost consumption and low efficiency in the task of detecting the land flatness after rotary tillage, and to propose a method for detecting the operation quality of tillage agricultural machinery based on an unmanned aerial vehicle (UAV).

[0008] The method for detecting the operation quality of tillage agricultural machinery based on an unmanned aerial vehicle includes the following steps:

[0009] Step S1: Obtain RGB image data;

[0010] Step S2: Process the image data obtained in Step S1 to obtain processed data;

[0011] Step S3: Based on the processed data obtained in Step S2, obtain filtered three-dimensional point cloud data P lidar ;

[0012] Step S4: Calculate the soil fragmentation rate and land flatness based on Step S3.

[0013] Preferably, in Step S1, the process of obtaining RGB image data is as follows:

[0014] Install a GNSS module and a lidar on the unmanned aerial vehicle;

[0015] The unmanned aerial vehicle flies in the air at a height greater than or equal to 10 m and less than or equal to 20 m from the ground;

[0016] The GNSS module provides positioning for the measurement points;

[0017] The lidar obtains RGB image data.

[0018] Preferably, in step S2, the image data obtained in step S1 is processed to obtain processed data;

[0019] The specific process is as follows:

[0020] Step S21: Input the RGB image obtained in S1 into a pre-trained field object segmentation model, and the pre-trained field object segmentation model outputs a field mask image;

[0021] The pre-trained field object segmentation model is a pre-trained U-Net model;

[0022] Perform pixel-level dot multiplication on the field mask image and the RGB image obtained in S1 to obtain a farmland RGB image;

[0023] Step S22: Input the farmland RGB image obtained in step S21 into a pre-trained post-tillage surface semantic segmentation model, and the pre-trained post-tillage surface semantic segmentation model outputs a mask image of the tillage area;

[0024] The pre-trained post-tillage surface semantic segmentation model is a pre-trained U-Net model;

[0025] Based on the mask image of the tillage area output by the pre-trained post-tillage surface semantic segmentation model in step S22, obtain the proportion of the post-tillage area in the farmland RGB image obtained in step S21; The specific process is as follows:

[0026] Assume that the pixel set of the mask image of the tillage area output by the pre-trained post-tillage surface semantic segmentation model in step S22 is I vis ={p 1 , p 2 ,…, p k ,…, p m};

[0027] Where p k represents the kth pixel of the mask image of the tillage area output by the pre-trained post-tillage surface semantic segmentation model, k = 1, 2,…, m;

[0028] Based on the total number of pixels N crop of the mask image of the tillage area output by the pre-trained post-tillage surface semantic segmentation model and the total number of pixels N total of the farmland RGB image obtained in step S21, calculate the cultivated land area coverage rate C total ; It is expressed as:

[0029]

[0030] Step S23: Input the mask image of the tillage area output by the pre-trained post-tillage surface semantic segmentation model in Step S22 into the pre-trained soil roughness prediction model, and the pre-trained soil roughness prediction model outputs a heat map S;

[0031] Each pixel in the heat map S represents a soil roughness;

[0032] Set a threshold L, and the positions where the soil roughness is greater than the threshold L are the set P of potential measurement points.

[0033] Preferably, the acquisition process of the pre-trained field feature segmentation model in Step S21 is as follows:

[0034] Input the labeled farmland, background, and RGB image dataset into the field feature segmentation model, pre-train the field feature segmentation model, and obtain the pre-trained field feature segmentation model;

[0035] The field feature segmentation model is a U-Net model.

[0036] Preferably, the acquisition process of the pre-trained post-tillage surface semantic segmentation model in Step S22 is as follows:

[0037] Input the farmland RGB image dataset with labeled tillage areas into the post-tillage surface semantic segmentation model, pre-train the post-tillage surface semantic segmentation model, and obtain the pre-trained post-tillage surface semantic segmentation model;

[0038] The post-tillage surface semantic segmentation model is a U-Net model.

[0039] Preferably, the acquisition process of the pre-trained soil roughness prediction model in Step S23 is as follows:

[0040] Input the set of mask images of the tillage area with labeled soil roughness into the soil roughness prediction model, pre-train the soil roughness prediction model, and obtain the pre-trained soil roughness prediction model;

[0041] The soil roughness prediction model is a ResNet-50 model.

[0042] Preferably, in Step S3, based on the processed data obtained in Step S2, filtered three-dimensional point cloud data P lidar ;

[0043] The specific process is as follows:

[0044] S31: Display the potential measurement points in the set P of potential measurement points obtained in S23 on the RGB image data obtained in Step S1, and set the flight path of the drone according to the positions of the potential measurement points in the RGB image obtained in Step S1;

[0045] After the UAV reaches above the potential measurement point, the lidar carried on the UAV obtains the three-dimensional point cloud data P of the potential measurement point lidar and RGB image data;

[0046] S32. Let j = 1, and extract the time-depth sequence x of the j-th measurement point in the three-dimensional point cloud data P lidar in the;

[0047] S33. Let i = 1, and process the i-th time-depth value of the j-th measurement point using the weighted sliding window filtering algorithm to obtain the filtered three-dimensional point cloud data corresponding to the i-th time-depth value of the j-th measurement point;

[0048] The formula of the weighted sliding window filtering algorithm is as follows:

[0049]

[0050] Among them,

[0051] n is the window size;

[0052] x i is the i-th time-depth value of the j-th measurement point within the window;

[0053] y i is the filtered three-dimensional point cloud data corresponding to the i-th time-depth value of the j-th measurement point within the window;

[0054] i is the i-th;

[0055] S34. Let i = i + 1, and repeat S33 until the filtered three-dimensional point cloud data corresponding to all time-depth values of the j-th measurement point is obtained;

[0056] S35. Let j = j + 1, and repeat S33 to S34 until the filtered three-dimensional point cloud data P corresponding to the time-depth sequence x of all measurement points in the three-dimensional point cloud data P lidar is obtained; lidar .

[0057] Preferably, in step S4, the soil fragmentation rate and land flatness are calculated based on step S3;

[0058] The specific process is as follows:

[0059] Step S41. Calculate the soil fragmentation rate based on step S3; the specific process is as follows:

[0060] Step S411. Input the RGB image data obtained by the lidar carried on the UAV in step S31 into the pre-trained panoramic segmentation model MaskFormer. The pre-trained panoramic segmentation model MaskFormer outputs a set of soil block masks, and the set of soil block masks contains B soil block masks;

[0061] Step S412: Calculate the length height and width width of the minimum bounding rectangle of the b-th soil block mask using the OpenCV library;

[0062] Calculate the diagonal length d of the minimum bounding rectangle of the b-th soil block mask based on the length height and width width of the minimum bounding rectangle of the b-th soil block mask;

[0063]

[0064] where width represents the width of the minimum bounding rectangle of the b-th soil block mask, height represents the length of the minimum bounding rectangle of the b-th soil block mask; b = 1, 2, …, B;

[0065] Step S413: Multiply the diagonal length d of the minimum bounding rectangle of the b-th soil block mask by the single-pixel size coefficient r to obtain the size of the b-th soil block in the real-world coordinate system;

[0066] Until the sizes of all soil blocks in the real-world coordinate system are obtained;

[0067] Step S414: Calculate the soil fragmentation rate;

[0068] The specific process is as follows:

[0069] Count the total number N of all soil blocks in the soil block mask set B total ;

[0070] Count the number N of soil blocks in the soil block mask set B whose sizes in the real-world coordinate system are less than 7 cm small ;

[0071] Calculate the soil fragmentation rate according to the following formula:

[0072]

[0073] Step S42: Calculate the soil flatness based on Step S3; The specific process is as follows:

[0074] Step S421: Let j = 1 and i = 1;

[0075] Read the i-th time-depth value z of the j-th measurement point from the filtered three-dimensional point cloud data P lidar where i = 1, 2, …, n; Calculate the average time-depth of the n time-depth values of the j-th measurement point i ;

[0076] Step S422: Based on the i-th time-depth value z of the j-th measurement point i calculate the i-th time-depth value z of the j-th measurement point iRelative height difference

[0077] Step S423: Based on the i-th time-depth value z of the j-th measurement point i Relative height difference Δz i , calculate the root mean square error RMSE of the flatness of the j-th measurement point; the expression is:

[0078]

[0079] where RMSE represents the root mean square error of the flatness of the j-th measurement point;

[0080] Step S424: Convert the root mean square error RMSE of the flatness of the j-th measurement point into the international roughness index IRI; the expression is:

[0081]

[0082] Step S425: Let j = j + 1, and repeat Steps S421 to S424 until the international roughness index IRI of all measurement points in the three-dimensional point cloud data P lidar is obtained.

[0083] Preferably, the process of obtaining the single-pixel size coefficient r in Step S413 is as follows:

[0084] Place a reference marker at the ground to be measured. Before the measurement starts, the drone hovers to take an image of the reference marker, count the pixel length of the reference marker on the captured image, and obtain the single-pixel size coefficient r through the pixel length of the reference marker and the actual length of the reference marker.

[0085] Preferably, the process of obtaining the pre-trained panoramic segmentation model MaskFormer is as follows:

[0086] Input the RGB image dataset of the marked soil blocks into the panoramic segmentation model MaskFormer, pre-train the panoramic segmentation model MaskFormer, and obtain the pre-trained panoramic segmentation model MaskFormer.

[0087] The beneficial effects of the present invention are as follows:

[0088] The purpose of the present invention is to solve the problems of high labor cost consumption and low efficiency in the task of detecting the flatness of the land after tillage by a rotary tiller, and thus propose a method for detecting the quality of tillage agricultural machinery based on a drone. This method automatically plans the flight path through a vision-based autonomous measurement point positioning algorithm, measures the proportion of the tilled area and the geometric shape of the ground in the tilled area through multi-view and multi-modal perception, and evaluates the operation quality of the tillage agricultural machinery.

[0089] The present invention proposes a method for detecting the quality of tillage agricultural machinery based on drones, innovatively combining machine vision and lidar technologies. And three tillage quality evaluation indicators are proposed, namely the proportion of post-tillage soil, the flatness of the post-tillage land, and the soil fragmentation rate of the post-tillage surface. At the same time, through the dynamic field patrol planning method, the goal of completing the measurement of all evaluation indicators in one flight is achieved. Existing alternative solutions generally rely on manual visual inspection or manual fieldwork for soil fragmentation rate measurement and flatness measurement. Compared with the existing solutions, this method is not only highly efficient but also can cover a larger farmland area, saving manpower and material resources.

[0090] The present invention aims to utilize advanced drone technology to break through the limitations of existing manual solutions and construct a set of unmanned, automated, and intelligent high-efficiency tillage agricultural machinery operation quality evaluation method, so as to improve the planting efficiency of large-scale farmland and achieve the goal of cost reduction and efficiency increase.

[0091] First of all, the primary problem to be solved by the present invention is how to effectively perceive the field situation, that is, indicators such as the ratio of the post-tillage area to the total field area and the approximate roughness of the operation area. In the post-tillage operation field of the tillage agricultural machinery, there are significant differences in RGB features such as texture and brightness between the areas of missed tillage and less tillage and the effective tillage area. Therefore, a trained semantic segmentation model can be used to process the field images. The core lies in using a two-stage field situation prediction model. First, through the ground object segmentation model, the field shape and the area of the area to be tilled are obtained; then, through the post-tillage area segmentation model, the effective tillage area is segmented and counted pixel by pixel to obtain the area ratio. The image of the obtained effective tillage area is sent to the roughness regression model in the next stage to perceive the approximate roughness of the field surface.

[0092] Secondly, the present invention focuses on solving the problem of the flight path planning of drones in the field. Due to the huge area and numerous fields of large-scale farmland, drones need to streamline their operation time as much as possible and focus on detecting areas where the post-tillage surface may be uneven and the soil fragmentation rate is low. If the drone takes full-field low-altitude photography, not only will it take a long time, but the data volume will also be extremely large, reducing the processing efficiency. The present invention endeavors to solve how to extract potential measurement points from the post-tillage operation area image obtained in the previous stage and detect these positions. The core lies in performing regression prediction through the RGB image features of the farmland to obtain a heat map of the roughness of the surface texture, setting a threshold, and extracting the positions with the highest roughness. At the same time, using the internal and external parameters of the drone camera and the world-image coordinate system transformation pair, the affine transformation of the measurement point positions is carried out to obtain the accurate longitude and latitude coordinates in reality, so as to plan the flight path of the drone in the close-range shooting stage.

[0093] Third, the present invention focuses on solving the evaluation method of soil fragmentation rate. The soil fragmentation rate affects soil physical and chemical properties such as soil water conductivity, and has a great impact on subsequent planting operations. The traditional soil fragmentation rate evaluation scheme is to manually excavate the surface soil layer at a given measurement point and count the distribution of soil block sizes, that is, the proportion of soil blocks smaller than a given threshold in the total soil blocks. Since the particle size distribution of soil blocks is basically uniform, there is also a newer unmanned scheme based on optics, that is, using the front camera of the tillage machine to take images of the surface after tillage, and estimating the surface soil fragmentation rate by means of artificial vision and scale comparison. Although such methods improve efficiency and automation, the degree of intelligence is low, and AI technology has not been further used to replace human labor. The present invention aims to further improve the degree of automation, use a panoramic segmentation model based on deep learning, use AI to extract information such as soil block distribution in RGB images, and use morphology to calculate its size parameters, and then automatically calculate the soil fragmentation rate index.

[0094] In addition, the present invention is committed to scientifically and reasonably measuring the land flatness efficiently. The surface situation after tillage is complex. The traditional method of calculating the standard deviation by measuring the soil layer height often has poor accuracy and is time-consuming and laborious. LiDAR technology can efficiently and accurately model complex surface conditions. Therefore, the present invention uses LiDAR to capture the surface point cloud data at the measurement point, suppresses high-frequency noise based on linear filtering, and proposes a new land flatness evaluation index based on the laser point cloud. After reconstructing the surface structure from the laser point cloud, the root mean square error (RMSE) is used to evaluate the surface flatness, and the international roughness index (IRI) is calculated.

[0095] Compared with the existing detection schemes for the quality of cultivated land after agricultural machinery operation, the present invention has improvements in terms of efficiency, automation, and intelligence, and the accuracy loss is very small. From the perspective of efficiency, this scheme is carried out by the autonomous patrol flight of the drone to complete data analysis and result summary in a short time. From the perspective of the degree of automation, the present invention can achieve unmanned autonomous operation in multiple links. From the perspective of the degree of intelligence, compared with the traditional scheme, the present invention uses deep learning technology in multiple links, improving the degree of intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Figure 1 It is a flowchart of the method of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0097] Detailed Embodiment 1: The method for detecting the operation quality of tillage agricultural machinery based on a drone according to this embodiment includes the following steps:

[0098] Step S1: Obtain RGB image data;

[0099] Step S2: Process the image data obtained in Step S1 to obtain processed data;

[0100] Step S3: Based on the processed data obtained in Step S2, obtain the filtered three-dimensional point cloud data P lidar ;

[0101] Step S4: Calculate the soil fragmentation rate and land flatness based on Step S3.

[0102] Specific Embodiment 2: The difference between this embodiment and Specific Embodiment 1 is that: RGB image data is obtained in Step S1; the specific process is as follows:

[0103] Install a GNSS module and a lidar on the unmanned aerial vehicle;

[0104] The unmanned aerial vehicle flies in the air at a height greater than or equal to 10m and less than or equal to 20m from the ground;

[0105] The GNSS module provides positioning for the measurement points;

[0106] The lidar obtains RGB image data.

[0107] 1. Selection of the unmanned aerial vehicle platform, selection of the lidar, and construction of the geodetic platform; the specific process is as follows:

[0108] (1) Select a multi-rotor unmanned aerial vehicle with stable flight performance (vertical hover accuracy reaches ±0.1 m, horizontal hover accuracy reaches ±0.3 m) to ensure the clarity of the collected images;

[0109] Select a multi-rotor unmanned aerial vehicle with good low-altitude performance (able to fly stably at a height lower than 3m and higher than 0.5m) to ensure the subsequent close-range shooting link; low altitude means flying stably at a height lower than 3m and higher than 0.5m;

[0110] At the same time, the unmanned aerial vehicle should have the function of autonomous flight with set waypoint anchors;

[0111] In addition, the unmanned aerial vehicle must be equipped with a high-precision positioning system (GNSS, RTK or PPK) to ensure that accurate position metadata can be obtained;

[0112] (2) Select a lidar with a depth range greater than 0.3m and less than 10m, a refresh rate of 30fps, a depth resolution of 1280x720 or higher, and a color resolution of 1280×800 or higher;

[0113] (3) Equipment installation, the equipment includes a GNSS module, a lidar, and an RGB camera with high resolution;

[0114] The GNSS provides accurate positioning for the measurement points;

[0115] The lidar is used to measure depth information, obtain the radar point cloud of the land, and thus model the ground surface;

[0116] A high-definition RGB camera captures optical images of the ground surface for subsequent analysis and processing;

[0117] 2. High-altitude flight and image acquisition; The specific process is as follows:

[0118] (1) According to the FOV parameters and resolution of the camera, set an appropriate flight altitude and fly at a height of 10 - 20 m above the ground;

[0119] It must be able to capture centimeter-level details of the ground surface texture;

[0120] The flight path of the unmanned aerial vehicle must be precisely planned to ensure that there is sufficient overlap area for each captured image. Generally, the image overlap rate needs to reach 60% - 80%, that is, in every two adjacent images, approximately two-thirds of the area is overlapping;

[0121] The present invention can adopt grid flight, that is, fly along a regular grid path, capture images at certain intervals, and ensure that there is an overlap area between every two images;

[0122] The photography range needs to ensure that it can cover the entire operation area;

[0123] (2) Set the camera shooting parameters;

[0124] The camera needs to select appropriate shooting parameters, such as exposure time, aperture size, etc., to ensure that the obtained image has an appropriate exposure level and can clearly show the overall situation of the operation area;

[0125] (3) After shooting, use an image stitching tool to stitch the obtained unmanned aerial vehicle images to obtain an overall image of the farmland.

[0126] Other steps and parameters are the same as those in the first specific implementation manner.

[0127] The third specific implementation manner: The difference between this implementation manner and the first or second specific implementation manner is that in step S2, the image data obtained in step S1 is processed to obtain processed data;

[0128] The specific process is as follows:

[0129] Step S21: Input the RGB image obtained in S1 into a pre-trained field object segmentation model, and the pre-trained field object segmentation model outputs a field mask image;

[0130] The pre-trained field object segmentation model is a pre-trained U-Net model;

[0131] Perform pixel-level dot multiplication on the field mask image and the RGB image obtained in S1 to obtain a farmland RGB image (only the farmland) with boundary debris removed;

[0132] After the ground object processing, the farmland part in the large farmland image can be concerned, without processing the RGB information of other pixels;

[0133] The ground object segmentation model can divide the field block area and obtain the range and shape information of the farmland to be measured;

[0134] Step S22: Input the farmland RGB image with boundary debris removed obtained in step S21 into the pre-trained semantic segmentation model of the post-tillage ground surface, and the pre-trained semantic segmentation model of the post-tillage ground surface outputs a mask image of the tillage area;

[0135] There are significant texture differences and clear boundaries between the tilled area and the untilled area. Therefore, this semantic segmentation model should be able to infer the mask of the tillage area;

[0136] The pre-trained semantic segmentation model of the post-tillage ground surface is a pre-trained U-Net model;

[0137] Based on the mask image of the tillage area output by the pre-trained semantic segmentation model of the post-tillage ground surface in step S22, obtain the proportion of the post-tillage area in the farmland RGB image with boundary debris removed obtained in step S21; The specific process is as follows:

[0138] Assume that the pixel set of the mask image of the tillage area output by the pre-trained semantic segmentation model of the post-tillage ground surface in step S22 is I vis ={p 1 , p 2 ,…, p k ,…, p m};

[0139] Where p k represents the k-th pixel of the mask image of the tillage area output by the pre-trained semantic segmentation model of the post-tillage ground surface, k = 1, 2,…, m;

[0140] Based on the total number of pixels N crop of the mask image of the tillage area output by the pre-trained semantic segmentation model of the post-tillage ground surface and the total number of pixels N total of the farmland RGB image with boundary debris removed obtained in step S21, calculate the cultivated land area coverage rate C total ; Expressed as:

[0141]

[0142] Step S23: Input the mask image of the tillage area output by the pre-trained semantic segmentation model of the post-tillage ground surface in step S22 into the pre-trained soil roughness prediction model, and the pre-trained soil roughness prediction model outputs a heat map S (the heat map is a matrix);

[0143] Each pixel in the heat map S represents a soil roughness;

[0144] Set a threshold L, and the positions where the soil roughness is greater than the threshold L are the set P of potential measurement points;

[0145] Other steps and parameters are the same as those in the first or second specific implementation manner.

[0146] Specific implementation manner four: The difference between this implementation manner and one of the first to third specific implementation manners is that: the acquisition process of the pre-trained field object segmentation model in step S21 is as follows:

[0147] Input the labeled farmland, background, and RGB image dataset into the field object segmentation model, and pre-train the field object segmentation model to obtain a pre-trained field object segmentation model;

[0148] The field object segmentation model is a U-Net model.

[0149] Other steps and parameters are the same as those in one of the first to third specific implementation manners.

[0150] Specific implementation manner five: The difference between this implementation manner and one of the first to fourth specific implementation manners is that: the acquisition process of the pre-trained post-tillage surface semantic segmentation model in step S22 is as follows:

[0151] Input the labeled farmland RGB image dataset of the tillage area into the post-tillage surface semantic segmentation model, and pre-train the post-tillage surface semantic segmentation model to obtain a pre-trained post-tillage surface semantic segmentation model;

[0152] The post-tillage surface semantic segmentation model is a U-Net model.

[0153] Other steps and parameters are the same as those in one of the first to fourth specific implementation manners.

[0154] Specific implementation manner six: The difference between this implementation manner and one of the first to fifth specific implementation manners is that: the acquisition process of the pre-trained soil roughness prediction model in step S23 is as follows:

[0155] Input the masked image set (each pixel marks the soil roughness) of the tillage area with labeled soil roughness into the soil roughness prediction model, and pre-train the soil roughness prediction model to obtain a pre-trained soil roughness prediction model;

[0156] The soil roughness prediction model is a ResNet-50 model.

[0157] Other steps and parameters are the same as those in one of the first to fifth specific implementation manners.

[0158] Specific Embodiment VII: The difference between this embodiment and any one of Embodiments I to VI is that: based on the processed data obtained in step S2 in step S3, the filtered three-dimensional point cloud data P is obtained. lidar The specific process is as follows:

[0159] S31. Display the potential measurement points in the potential measurement point set P obtained in S23 on the RGB image data obtained in step S1, and set the flight path of the unmanned aerial vehicle according to the positions of the potential measurement points in the RGB image obtained in step S1 (the unmanned aerial vehicle must be equipped with a high-precision positioning system (GNSS, RTK or PPK)).

[0160] After the unmanned aerial vehicle reaches above the potential measurement point (shooting at a height greater than or equal to 0.5 m and less than or equal to 1 m), it needs to hover stably for a period of time. The lidar carried on the unmanned aerial vehicle obtains the three-dimensional point cloud data P lidar (measuring the depth information of each point) and the RGB image data;

[0161] The lidar takes pictures many times per second when shooting;

[0162] S32. Let j = 1, and extract the time-depth sequence x of the j-th measurement point in the three-dimensional point cloud data P lidar The depth of each measurement point changes with time; the depth of each measurement point is known;

[0163] S33. Let i = 1, and use the weighted sliding window filtering algorithm to process the i-th time-depth value of the j-th measurement point to obtain the filtered three-dimensional point cloud data corresponding to the i-th time-depth value of the j-th measurement point.

[0164] The formula of the weighted sliding window filtering algorithm is as follows:

[0165] Among them,

[0166]

[0167] where

[0168] n is the window size; within one window are the values from the i-th to the i - n + 1-th of a measurement point in the time series;

[0169] x i is the i-th time-depth value of the j-th measurement point within the window;

[0170] y i is the filtered three-dimensional point cloud data corresponding to the i-th time-depth value of the j-th measurement point within the window;

[0171] i is the i-th;

[0172] S34. Let \(i = i + 1\), and repeat S33 until the filtered three-dimensional point cloud data corresponding to all the time-depth values of the \(j\)th measurement point are obtained;

[0173] S35. Let \(j = j + 1\), and repeat S33 to S34 until the three-dimensional point cloud data \(P\) lidar in all the filtered three-dimensional point cloud data \(P\) corresponding to the time-depth sequences \(x\) of all the measurement points in lidar ;

[0174] (1). Due to the change in height, the parameter settings of the camera need to be adjusted accordingly, including parameters such as focal length, exposure time, ISO sensitivity, and aperture size. Ensure that the brightness of the near-surface soil image is moderate and the details are clear.

[0175] (2). The unmanned aerial vehicle flies according to the path points. When it reaches above the measurement point, it hovers for a period of time until the shooting and point cloud sampling are completed.

[0176] (3). The images are stored in lossless formats such as RAW or TIFF, and metadata such as measurement point information and position information are attached to assist subsequent processing.

[0177] (4). After obtaining the point cloud data and the visual data of the near-surface land shooting, data preprocessing is first carried out; the specific process is as follows:

[0178] Since the lidar measurement is affected by high-frequency noise and the results are somewhat unstable, the collected data has certain fluctuations and needs to be filtered to minimize its impact on the calculation as much as possible. The present invention uses a weighted sliding window filtering algorithm to preprocess the depth data. This method recalculates all the depth values within a sliding window and uses the depth values at different positions within the window to calculate a new depth value through weighted calculation to replace the depth value at the center point of the window, which is a linear smoothing method. The principle of the sliding window weighted filtering algorithm is simple and the calculation time is short. It can effectively reduce the interference of the depth value fluctuation during data processing on the results while better retaining the undulation of the post-tillage ground surface.

[0179] Other steps and parameters are the same as those in Embodiments 1 to 6.

[0180] Embodiment 8: The difference between this embodiment and any one of Embodiments 1 to 7 is that in step S4, the soil fragmentation rate and land flatness are calculated based on step S3;

[0181] The specific process is as follows:

[0182] Step S41. Calculate the soil fragmentation rate based on step S3;

[0183] The specific process is as follows:

[0184] Step S411: Input the RGB image data obtained by the lidar carried on the drone in step S31 into the pre-trained panoramic segmentation model MaskFormer. The pre-trained panoramic segmentation model MaskFormer outputs a set of soil block masks (outputting multiple mask images B), and the set of soil block masks contains B soil block masks; each soil block mask represents one soil block.

[0185] Assume that the distribution of the soil fragmentation rate inside the field is a uniform distribution, then the soil fragmentation rate on the ground surface can characterize the overall soil fragmentation rate. Input the image S taken at the measurement point into the soil block segmentation model; the soil block segmentation model is a type of panoramic segmentation model, which combines traditional semantic segmentation and instance segmentation. It can not only identify the category to which each pixel in the image belongs, but also distinguish different instances within the same category. The soil block segmentation model can label the shape and boundary of the soil blocks in the image and obtain a set of soil block masks B.

[0186] Step S412: Use the OpenCV library to calculate the length height and width width of the minimum bounding rectangle of the b-th soil block mask.

[0187] Calculate the diagonal length d of the minimum bounding rectangle of the b-th soil block mask according to the length height and width width of the minimum bounding rectangle of the b-th soil block mask.

[0188]

[0189] Among them, width represents the width of the minimum bounding rectangle of the b-th soil block mask, and height represents the length of the minimum bounding rectangle of the b-th soil block mask; b = 1, 2,..., B.

[0190] In the present invention, the diagonal length of the minimum bounding rectangle is used as the soil block size measurement value.

[0191] The computer vision processing library OpenCV can be used for calculation to obtain the minimum bounding matrix.

[0192] Step S413: Multiply the diagonal length d of the minimum bounding rectangle of the b-th soil block mask by the single-pixel size coefficient r to obtain the size of the b-th soil block in the real-world coordinate system.

[0193] Until the sizes of all soil blocks in the real-world coordinate system are obtained.

[0194] Step S414: Calculate the soil fragmentation rate; the specific process is as follows:

[0195] Count the total number N of all soil blocks in the set of soil block masks B total ;

[0196] Count the number N of soil clods in the soil clod mask set B whose sizes in the real-world coordinate system are less than 7 cm. small ;

[0197] Calculate the soil fragmentation rate according to the following formula:

[0198]

[0199] The soil fragmentation rate is defined as the ratio of the number of soil clods with sizes less than 7 cm at the measurement point to the total number of soil clods;

[0200] Step S42: Calculate the soil flatness based on Step S3;

[0201] The specific process is as follows:

[0202] The present invention uses the international roughness index IRI as the soil flatness value. IRI is calculated considering the root mean square error (RMSE) of flatness as a parameter. Calculating with RMSE can offset the inclination problems caused by the camera suspension position not being perpendicular to the ground or the attitude of the drone not being horizontal during flight.

[0203] Step S421: Let j = 1 and i = 1;

[0204] Read the filtered three-dimensional point cloud data P lidar The i-th time-depth value z of the j-th measurement point in i , i = 1, 2,..., n; Calculate the average time-depth of the n time-depth values of the j-th measurement point

[0205] Step S422: Based on the i-th time-depth value z of the j-th measurement point i , calculate the relative height difference of the i-th time-depth value z of the j-th measurement point i

[0206] Step S423: Based on the relative height difference Δz of the i-th time-depth value z of the j-th measurement point i i , calculate the root mean square error RMSE of the flatness of the j-th measurement point; The expression is:

[0207]

[0208] Among them, RMSE represents the root mean square error of the flatness of the j-th measurement point;

[0209] Step S424: Convert the root mean square error RMSE of the flatness of the j-th measurement point into the international roughness index IRI; The expression is:

[0210]

[0211] Step S425: Let j = j + 1, and repeat steps S421 to S424 until the three-dimensional point cloud data P lidar for all the measurement points in the international roughness index IRI;

[0212] Given the roughness standard deviation RMSE, the international roughness index IRI is calculated through an empirical formula.

[0213] Other steps and parameters are the same as those in the first to seventh specific embodiments.

[0214] Specific embodiment nine: The difference between this embodiment and one of the first to eighth specific embodiments is that the process of obtaining the single-pixel size coefficient r in step S413 is as follows:

[0215] Place a reference marker (such as a ruler with a length of 1 m) at the ground to be measured. Before the measurement starts, the UAV hovers to take an image of the reference marker, and count the pixel length of the reference marker on the captured image. The single-pixel size coefficient r is obtained through the pixel length of the reference marker and the actual length of the reference marker.

[0216] Other steps and parameters are the same as those in the first to eighth specific embodiments.

[0217] Specific embodiment ten: The difference between this embodiment and one of the first to ninth specific embodiments is that the process of obtaining the pre-trained panoramic segmentation model MaskFormer is as follows:

[0218] Input the RGB image dataset of the labeled soil blocks into the panoramic segmentation model MaskFormer, and pre-train the panoramic segmentation model MaskFormer to obtain the pre-trained panoramic segmentation model MaskFormer.

[0219] Other steps and parameters are the same as those in the first to ninth specific embodiments.

[0220] Key points of the present invention:

[0221] 1. A two-stage method for post-tillage area positioning and area ratio evaluation. The present invention adopts a two-stage process. First, use a pre-trained ground object segmentation model to process large-scale farmland RGB images to obtain a field area mask, and obtain a field area image through pixel-by-pixel AND operation. Then, input this image into the post-tillage area segmentation model to obtain a post-tillage area mask. Through pixel-level statistics, obtain the ratio of the post-tillage area to the overall farmland area as an important indicator for evaluating the post-tillage field situation.

[0222] 2. Dynamic Planning Scheme for Operation Trajectory Based on UAV Vision. The present invention utilizes deep learning to perform graph regression prediction using the RGB features of the post-tillage field images, obtaining a roughness heat map, and thereby obtaining potential measurement points. At the same time, the present invention utilizes the internal and external camera parameters to achieve precise annotation of the measurement point positions, enabling the UAV to focus on potential surface uneven positions and improving the operation efficiency.

[0223] 3. Method for Evaluating Soil Crushing Rate Based on Machine Vision. The present invention utilizes a panoramic segmentation model and morphology-based statistics to automatically obtain the pixel-level mask of soil clods, and obtains the size of soil clods by calculating the diagonal length of the minimum circumscribed matrix and the pixel coefficient, thereby automatically completing the calculation of the soil crushing rate.

[0224] 4. Disturbance Filtering Method Based on Linear Filtering. Affected by air flow and control signals, the flight of the UAV will inevitably be subject to certain dynamic disturbances. Poor flight stability may cause deviations in measurement accuracy. The present invention suppresses high-frequency noise through a weighted average sliding window filtering method in the data post-processing stage. This method can effectively improve the robustness of depth measurement, enabling the system to still provide stable and accurate surface flatness data even under complex flight conditions. Through this innovative disturbance elimination technology, the present system greatly improves the application effect in the actual agricultural environment and ensures the high reliability of measurement data.

[0225] 5. Surface Flatness Monitoring Based on LiDAR. The present invention relates to a method and system for measuring land flatness based on a UAV carrying a LiDAR flying close to the ground. By flying a UAV carrying a LiDAR sensor at low altitude, the laser point cloud data of the surface is obtained in real time, and the surface morphology information of the farmland is obtained using the high-precision measurement ability of the LiDAR. Then, indicators such as RMSE (Root Mean Square Error) and international IRI (International Roughness Index) are used to quantitatively evaluate the flatness of the land in combination with the point cloud data. This method can not only automatically calculate and generate a flatness evaluation report, reducing manual intervention, but also provide high-precision land flatness data according to the evaluation results to help farmers or agricultural managers optimize tillage and land leveling operations. In addition, the present invention also relates to an automated data processing and analysis system for processing and analyzing laser point cloud data, calculating RMSE and IRI indicators, thereby realizing real-time monitoring and precise evaluation of land flatness, and having the characteristics of high efficiency, high precision, and high degree of automation.

[0226] The present invention may also have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and deformations according to the present invention, but these corresponding changes and deformations should all fall within the protection scope of the appended claims of the present invention.

Claims

1. A method for detecting the quality of tillage and agricultural machinery operations based on drones, characterized in that: The method comprises the following steps: Step S1, obtaining RGB image data; Step S2, processing the image data acquired in step S1 to obtain processed data; Step S3: Based on the processed data obtained in step S2, the filtered three-dimensional point cloud data P is obtained. lidar ; Step S4: Calculate the soil crushing rate and land flatness based on step S3.

2. The method for detecting the quality of tillage and agricultural machinery operation based on an unmanned aerial vehicle according to claim 1, characterized in that: In step S1, RGB image data is obtained; the specific process is: The drone is equipped with GNSS modules and lidar; The drone is flying in the air at a distance of 10m or more and 20m or less from the ground; The GNSS module provides positioning for the measuring points; The laser radar obtains RGB image data.

3. The method for detecting the quality of tillage and agricultural machinery operation based on an unmanned aerial vehicle according to claim 2, characterized in that: In the step S2, the image data acquired in the step S1 is processed to obtain processed data; The specific process is: Step S21, inputting the RGB image obtained in S1 into a pre-trained field object segmentation model, and the pre-trained field object segmentation model outputs a field mask image; The pre-trained field object segmentation model is a pre-trained U-Net model; Perform pixel-level dot multiplication between the field mask image and the RGB image obtained by S1 to obtain the farmland RGB image; Step S22, inputting the farmland RGB image obtained in step S21 into a pre-trained cultivated land surface semantic segmentation model, and the pre-trained cultivated land surface semantic segmentation model outputs a mask image of the cultivated area; The pre-trained cultivated land surface semantic segmentation model is a pre-trained U-Net model; Based on the mask image of the cultivated area output by the pre-trained cultivated land surface semantic segmentation model in step S22, the proportion of the cultivated area in the farmland RGB image obtained in step S21 is obtained; the specific process is as follows: Assume that the pixel set of the mask image of the cultivated area output by the pre-trained cultivated land surface semantic segmentation model in step S22 is I vis ={p1,p2,…,p k ,…,p m }; where p k The kth pixel of the mask image of the cultivated area output by the pre-trained cultivated land semantic segmentation model, k = 1, 2, ..., m; The total number of pixels N of the mask image of the cultivated area output by the pre-trained cultivated land semantic segmentation model crop and the total number of pixels N of the farmland RGB image obtained in step S21 total , calculate the cultivated land coverage rate C total ; expressed as: Step S23, inputting the mask image of the cultivated area output by the pre-trained post-cultivation land surface semantic segmentation model in step S22 into the pre-trained soil roughness prediction model, and the pre-trained soil roughness prediction model outputs a heat map S; Each pixel in the heat map S represents a soil roughness; A threshold L is set, and the location where the soil roughness is greater than the threshold L is the potential measurement point set P.

4. The method for detecting the quality of tillage machinery operation based on an unmanned aerial vehicle according to claim 3 is characterized in that: The process of obtaining the pre-trained field object segmentation model in step S21 is as follows: Input the marked farmland, background and RGB image data sets into the field object segmentation model, pre-train the field object segmentation model, and obtain a pre-trained field object segmentation model; The field object segmentation model is a U-Net model.

5. The method for detecting the quality of tillage and agricultural machinery operation based on an unmanned aerial vehicle according to claim 4, characterized in that: The acquisition process of the pre-trained cultivated land surface semantic segmentation model in step S22 is as follows: The farmland RGB image dataset with marked cultivated areas is input into the cultivated land surface semantic segmentation model, and the cultivated land surface semantic segmentation model is pre-trained to obtain a pre-trained cultivated land surface semantic segmentation model; The cultivated land surface semantic segmentation model is a U-Net model.

6. The method for detecting the quality of tillage and agricultural machinery operation based on an unmanned aerial vehicle according to claim 5, characterized in that: The process of obtaining the pre-trained soil roughness prediction model in step S23 is as follows: Inputting a mask image set of a cultivated area with soil roughness marked thereon into a soil roughness prediction model, pre-training the soil roughness prediction model, and obtaining a pre-trained soil roughness prediction model; The soil roughness prediction model is a ResNet-50 model.

7. The method for detecting the quality of tillage and agricultural machinery operation based on an unmanned aerial vehicle according to claim 6, characterized in that: In step S3, based on the processed data obtained in step S2, the filtered three-dimensional point cloud data P is obtained. lidar ; The specific process is: S31, displaying the potential measuring points in the potential measuring point set P obtained in S23 on the RGB image data obtained in step S1, and setting a track for the UAV according to the position of the potential measuring points in the RGB image obtained in step S1; After the UAV reaches the potential measuring point, the laser radar on the UAV obtains the 3D point cloud data P of the potential measuring point. lidar and RGB image data; S32, let j = 1, extract the three-dimensional point cloud data P lidar The time depth series x of the jth measuring point in ; S33, let i=1, use the weighted sliding window filtering algorithm to process the i-th time depth value of the j-th measuring point, and obtain the filtered three-dimensional point cloud data corresponding to the i-th time depth value of the j-th measuring point; The weighted sliding window filtering algorithm formula is as follows: in, n is the window size; x i is the i-th time-depth value of the j-th measuring point in the window; y i is the filtered 3D point cloud data corresponding to the i-th time-depth value of the j-th measuring point in the window; i is the ith one; S34, let i=i+1, repeat S33, until the filtered three-dimensional point cloud data corresponding to all time depth values ​​of the j-th measuring point are obtained; S35, let j = j + 1, repeat S33 to S34 until the three-dimensional point cloud data P is obtained lidar The filtered 3D point cloud data P corresponding to the time depth series x of all measurement points in lidar .

8. The method for detecting the quality of tillage and agricultural machinery operation based on an unmanned aerial vehicle according to claim 7, characterized in that: In step S4, the soil crushing rate and the land flatness are calculated based on step S3; The specific process is: Step S41: Calculate the soil crushing rate based on step S3; the specific process is as follows: Step S411: input the RGB image data obtained by the laser radar carried by the UAV in step S31 into the pre-trained panoramic segmentation model MaskFormer, and the pre-trained panoramic segmentation model MaskFormer outputs a soil block mask set, and the soil block mask set includes B soil block masks; Step S412, using the OpenCV library to calculate the length height and width width of the minimum circumscribed rectangle of the b-th soil block mask; Calculate the diagonal length d of the minimum circumscribed rectangle of the b-th soil block mask according to the height and width of the minimum circumscribed rectangle of the b-th soil block mask; Wherein, width represents the width of the minimum bounding rectangle of the b-th soil block mask, and height represents the length of the minimum bounding rectangle of the b-th soil block mask; b=1,2,…,B; Step S413, multiplying the diagonal length d of the minimum circumscribed rectangle of the b-th soil block mask by the single pixel size coefficient r to obtain the size of the b-th soil block in the real world coordinate system; Until the sizes of all soil blocks in the real world coordinate system are obtained; Step S414, calculating the soil crushing rate; The specific process is: Count the number of all soil blocks N in the soil block mask set B total ; Count the number of soil blocks N in the soil block mask set B whose size is less than 7 cm in the real world coordinate system small ; The soil crushing rate is calculated according to the following formula: Step S42: Calculate soil flatness based on step S3; the specific process is: Step S421, let j=1, i=1; Read the filtered 3D point cloud data P lidar The i-th time depth value z of the j-th measuring point in i , i=1,2,…,n; Calculate the average time depth of the n time depth values ​​at the jth measurement point Step S422: Based on the i-th time depth value z of the j-th measuring point i , calculate the i-th time depth value z of the j-th measuring point i The relative height difference Step S423: Based on the i-th time depth value z of the j-th measuring point i The relative height difference Δz i , calculate the RMSE of the flatness standard deviation of the jth measuring point; the expression is: Among them, RMSE represents the standard deviation of the flatness of the jth measuring point; Step S424, convert the RMSE of the roughness standard deviation of the j-th measuring point into the international roughness index IRI; the expression is: Step S425: Let j = j + 1, and repeat steps S421 to S424 until the three-dimensional point cloud data P is obtained. lidar The International Roughness Index (IRI) of all measuring points.

9. The method for detecting the quality of tillage and agricultural machinery operation based on an unmanned aerial vehicle according to claim 8, characterized in that: The process of obtaining the single pixel size coefficient r in step S413 is as follows: A reference marker is placed on the ground to be measured. Before the measurement begins, the drone takes an image of the reference marker by hovering. The pixel length of the reference marker is counted on the captured image. The single pixel size coefficient r is obtained through the pixel length of the reference marker and the actual length of the reference marker.

10. The method for detecting the quality of tillage and agricultural machinery operation based on an unmanned aerial vehicle according to claim 9, characterized in that: The acquisition process of the pre-trained panoramic segmentation model MaskFormer is as follows: The RGB image dataset of the marked soil blocks is input into the panoramic segmentation model MaskFormer, and the panoramic segmentation model MaskFormer is pre-trained to obtain the pre-trained panoramic segmentation model MaskFormer.

Images