Ploughing point cloud model image height analysis and distribution decision-making system

Abstract

The invention relates to the technical field of image analysis, and further relates to a cultivated land point cloud model image height analysis and distribution decision-making system. The system comprises an image acquisition part used for acquiring a depth map corresponding to an RGB image of a target area, and constructing a homogeneous three-dimensional point cloud under a camera coordinate system according to the depth map and a camera internal parameter matrix; the height analysis part is used for performing initial ground area extraction on the three-dimensional point cloud under the world coordinate system to complete height analysis; and the operation decision distribution part is used for carrying out cultivated land area distribution. The method can automatically, accurately and reliably identify and allocate the cultivated land area, and has good adaptability and robustness to topographic relief and low sundries in a real farmland environment.

Description

Technical Field

[0001] The present invention belongs to the technical field of image analysis, and in particular relates to a farmland cloud model image height analysis and allocation decision system. Background Art

[0002] With the global population growing and the increasing demand for food, as well as the increasing cost of agricultural labor and the growing scarcity of resources, precision agriculture has become an inevitable trend in the development of modern agriculture. The core of precision agriculture lies in the use of information technology, sensor technology, automated control technology, and other means to implement refined field management based on the temporal and spatial variability of farmland environments and crop growth, in order to achieve the sustainable agricultural development goals of high yield, high quality, high efficiency, low consumption, and environmental protection. Among them, real-time and accurate perception of the farmland environment is the foundation for achieving precise management and automated operations. In particular, the effective identification and positioning of ground conditions, especially arable areas (arable land), is crucial for key applications such as variable-rate seeding, variable-rate fertilization, variable-rate spraying, automated tillage, and path planning.

[0003] In traditional agricultural production, the identification of cultivated land primarily relies on the farmer's experience and visual observation. This approach is not only inefficient and labor-intensive, but also highly subjective. It makes it difficult to achieve large-scale, standardized, and precise operations, and it falls short of meeting the demands of automated and intelligent agriculture. Therefore, researching and developing technologies and systems that can automatically, quickly, and accurately identify cultivated land areas has significant practical significance and application value.

[0004] In recent years, with the development of sensor technology and computer vision, a variety of technical solutions for farmland environmental perception and ground detection have emerged. Some early methods utilized simple sensors, such as ultrasonic sensors or laser rangefinders, to obtain ground height information directly below or in front of agricultural machinery through single-point ranging. However, the spatial information provided by these sensors is very limited. They can only reflect the height of the detection point and cannot capture the regional surface morphology. They have difficulty effectively distinguishing low vegetation, crop residue, and the actual ground, and are easily affected by the complex texture and tilt angle of the surface. Although mechanical probes can directly contact the ground, they are contact measurements, which are slow to operate, may damage the surface, and are difficult to adapt to complex terrain. Summary of the Invention

[0005] The main purpose of this invention is to provide a system for analyzing and allocating cultivated land area cloud model images. This system uses an RGB-D camera to acquire color and depth information of the target area, constructs a three-dimensional point cloud, and converts it to a world coordinate system. The system then adaptively segments the point cloud height using a Gaussian mixture model (GMM). This system robustly identifies the initial ground area by combining spatial region of interest (ROI) constraints and local inclination geometry analysis. Furthermore, iterative weighted distance filtering is used to obtain a refined ground point cloud. Finally, the weighted average height of the refined ground point cloud is compared with a preset cultivated land height threshold to achieve intelligent allocation of cultivated land areas. The system also provides fault tolerance when no ground is detected. This technology has the beneficial effect of automatically, accurately, and reliably identifying and allocating cultivated land areas. It exhibits good adaptability and robustness to the undulating terrain and low debris found in real farmland environments, reduces reliance on manual experience, and reduces interference from environmental factors. It effectively supports variable-variable operations in precision agriculture and intelligent perception and decision-making by automated agricultural machinery, helping to improve agricultural production efficiency and resource utilization.

[0006] In order to solve the above problems, the technical solution of the present invention is achieved as follows:

[0007] A farmland cloud model image height analysis and allocation decision system, the system comprising:

[0008] The image acquisition part is used to obtain the depth map corresponding to the RGB image of the target area, and construct a homogeneous 3D point cloud in the camera coordinate system based on the depth map and the camera internal parameter matrix. According to the camera's rotation matrix and translation vector, the transformation from the world coordinate system to the camera coordinate system is represented, and the homogeneous 3D point cloud in the camera coordinate system is converted to the world coordinate system to obtain a 3D point cloud in the world coordinate system.

[0009] The height analysis part is used to extract the initial ground area of the 3D point cloud in the world coordinate system and complete the height analysis to determine whether the ground is detected;

[0010] The operation decision allocation part is used to obtain a fine ground area point cloud when it is determined that the ground is detected, and calculate the weighted average ground height based on the fine ground area point cloud. When the weighted average ground height is within the set cultivated land height threshold range, the target area part corresponding to the fine ground area point cloud is allocated as the cultivated land area part; when it is determined that the ground is not detected, the RGB image of the target area of the next frame is height analyzed, and the cultivated land area is allocated according to the height analysis result of the RGB image of the target area of the next frame.

[0011] Furthermore, the height analysis part extracts the initial ground area of the three-dimensional point cloud in the world coordinate system and completes the height analysis to determine whether the ground is detected. The process specifically includes: performing adaptive threshold segmentation based on the Gaussian mixture model on the height direction of the three-dimensional point cloud in the world coordinate system to obtain candidate ground point clouds; performing fixed threshold segmentation based on elliptical area constraints on the distance of the three-dimensional point cloud in the horizontal plane in the world coordinate system to obtain point clouds of the area of interest; determining the initial ground area point cloud based on the candidate ground point clouds and the point clouds of the area of interest; calculating the local inclination distribution of the initial ground area point cloud, and determining the ground detection result based on the local inclination distribution and the preset inclination distribution threshold to determine whether the ground is detected.

[0012] Furthermore, when the job decision allocation part determines that the ground is detected, iterative screening is performed based on the weighted distance from all points in the three-dimensional point cloud in the world coordinate system to the initial ground area point cloud to obtain a refined ground area point cloud.

[0013] Furthermore, let the camera internal parameter matrix be Homogeneous 3D point cloud in camera coordinate system:

[0014]

[0015] Where N is the number of point clouds, Represents the three-dimensional coordinates of the i-th point in the camera coordinate system; is the X-axis coordinate of the i-th point in the camera coordinate system; is the Y-axis coordinate of the i-th point in the camera coordinate system; is the Z-axis coordinate of the i-th point in the camera coordinate system; represents a set of 3-row and 3-column matrices whose elements are all real numbers; let the rotation matrix be and the translation vector is The homogeneous three-dimensional point cloud in the camera coordinate system is converted to the world coordinate system according to the following formula to obtain the three-dimensional point cloud in the world coordinate system:

[0016]

[0017] T is the transpose operation; SO(3) represents the group of all 3-by-3 real rotation matrices;

[0018]

[0019] is the X-axis coordinate of the i-th point in the world coordinate system; is the Y-axis coordinate of the i-th point in the world coordinate system; is the Z-axis coordinate of the i-th point in the world coordinate system.

[0020] Furthermore, the height direction of the 3D point cloud in the world coordinate system is segmented by an adaptive threshold based on a Gaussian mixture model. The process of obtaining the candidate ground point cloud specifically includes: extracting the Z-axis coordinates of all points from the 3D point cloud to form a data set Let the number of components of the Gaussian mixture model be K, then the one-dimensional Gaussian mixture model is expressed as:

[0021]

[0022] Among them, π k is the mixing coefficient of the kth Gaussian component, and satisfies μ k is the mean of the kth Gaussian component; σ k is the standard deviation of the kth Gaussian component; Represents a one-dimensional normal distribution:

[0023]

[0024] Use the maximum expectation algorithm to iteratively estimate So that the model parameters {π k , μ k ,σ k}converges to the local optimal solution, at this time the Gaussian mixture model of the Z height distribution is obtained; according to the results of the Gaussian mixture modeling, the three-dimensional point cloud in the world coordinate system is adaptively threshold segmented in the height direction to obtain the candidate ground point cloud.

[0025] Furthermore, according to the results of Gaussian mixture modeling, the three-dimensional point cloud in the world coordinate system is segmented in the height direction by adaptive thresholding. The process of obtaining the candidate ground point cloud includes: in the one-dimensional height direction, the lowest cluster of points corresponding to the ground is set, and the mean μ is selected. k The smallest Gaussian component is used as the ground component:

[0026]

[0027] Based on the selected ground composition is the mean of the selected ground components; is the standard deviation of the selected ground component; based on the posterior probability To determine whether the probability of a point belonging to the ground component is greater than the first judgment threshold δ:

[0028]

[0029] like Then confirm The corresponding points are candidate ground points; the points that meet the above posterior probability judgment are selected from the three-dimensional point cloud P in the world coordinate systemw Take it out to form a candidate ground point cloud:

[0030]

[0031] Furthermore, the distance of the 3D point cloud in the horizontal plane in the world coordinate system is segmented with a fixed threshold based on the ellipse region constraint. The process of obtaining the point cloud of the region of interest specifically includes: taking the center of the camera as the center of the ellipse (x0, y0), setting the semi-major axis a, the semi-minor axis b and the rotation angle θ, and using the following segmentation process to extract the point cloud of the region of interest from the point cloud:

[0032] Set inequality constraints:

[0033]

[0034] For each point in the point cloud The points that satisfy the above inequality constraints are taken from the three-dimensional point cloud P in the world coordinate system W Take it out and form the point cloud P of the region of interest ROI .

[0035] Furthermore, the process of determining the initial ground region point cloud according to the candidate ground point cloud and the region of interest point cloud includes: using the intersection of the candidate ground point cloud and the region of interest point cloud as the initial ground region point cloud.

[0036] Furthermore, the process of determining the initial ground region point cloud according to the candidate ground point cloud and the region of interest point cloud includes: taking the union of the candidate ground point cloud and the region of interest point cloud as the initial ground region point cloud.

[0037] The farmland cloud model image height analysis and allocation decision system of the present invention has the following beneficial effects:

[0038] This system and method for analyzing and assigning height distribution decisions for farmland cloud model images effectively combines color and depth information provided by RGB-D cameras and converts it into a stable world coordinate system for analysis, achieving precise perception of the three-dimensional geometry of farmland environments. Its core advantage lies in its use of a highly adaptive threshold segmentation technique based on the Gaussian mixture model (GMM). This technique dynamically learns and identifies potential ground layers based on the actual height distribution of the scene, overcoming the limitations of traditional fixed threshold methods when dealing with varying terrain undulations and slopes, significantly improving the robustness of ground recognition.

[0039] Furthermore, the system not only relies on height statistics, but also introduces local inclination distribution analysis as a means of geometric verification, ensuring that the identified ground area has the flat or gentle slope characteristics that cultivated land should have, and effectively eliminating some non-ground objects that are similar in height but inconsistent in shape (such as low, messy vegetation piles). At the same time, through the spatial constraints of the region of interest (ROI), the analysis is focused on the area related to the current operation, which improves processing efficiency and reduces interference from irrelevant information in the distance. After the ground is preliminarily determined, an iterative screening method based on weighted distance is used to refine the ground point cloud, which can further eliminate noise and outliers, obtain a more accurate and complete ground representation, and lay the foundation for the subsequent accurate calculation of the average ground height.

[0040] Ultimately, the system compares the weighted average height of the detailed ground area with the set threshold range for cultivated land height to make decisions about the allocation of cultivated land, ensuring that the allocation results are directly linked to actual agronomic requirements. Furthermore, the system has designed processing logic for failure to detect the ground, namely analyzing the next frame of data. This fault-tolerant mechanism ensures operational stability and decision-making reliability in complex environments or those with poor sensor data. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of the system structure of the cultivated land cloud model image height analysis and allocation decision system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0043] refer to Figure 1: A cultivated land cloud model image height analysis and allocation decision system, the system includes: an image acquisition part, used to obtain a depth map corresponding to the RGB image of the target area, and construct a homogeneous three-dimensional point cloud in the camera coordinate system according to the depth map and the camera internal parameter matrix; according to the camera's rotation matrix and translation vector, the transformation from the world coordinate system to the camera coordinate system is represented, and the homogeneous three-dimensional point cloud in the camera coordinate system is converted to the world coordinate system to obtain a three-dimensional point cloud in the world coordinate system; a height analysis part, used to perform initial ground area extraction on the three-dimensional point cloud in the world coordinate system, complete height analysis to determine whether the ground is detected; an operation decision allocation part, used to obtain a fine ground area point cloud when it is determined that the ground is detected, and calculate the weighted average ground height based on the fine ground area point cloud. When the weighted average ground height is within the set cultivated land height threshold range, the target area part corresponding to the fine ground area point cloud is allocated as the cultivated land area part; when it is determined that the ground is not detected, the RGB image of the target area of the next frame is height analyzed, and the cultivated land area is allocated according to the height analysis result of the RGB image of the target area of the next frame.

[0044] The workflow of the system starts with the image acquisition part. This part is the data source of the entire system, and its main task is to capture the visual information and depth information of the target area. Usually, this is achieved using an RGB-D camera or a combination of an ordinary RGB camera and a sensor such as LiDAR. RGB images provide rich color and texture information, which helps to identify surface cover, such as distinguishing between bare soil, green vegetation, withered crop residues, etc. However, RGB images alone cannot directly provide accurate distance or height information. Therefore, the acquisition of depth maps is crucial. Each pixel value on the depth map represents the distance from the camera to the corresponding point in the scene. By combining RGB images and depth maps, the system can construct a colorful three-dimensional point cloud model for the target area. Specifically, for each pixel (u, v) in the depth map, its corresponding depth value is d, combined with the camera's internal parameter matrix K∈R 3×3 , the three-dimensional coordinates of the point in the camera coordinate system can be calculated The camera intrinsic parameter matrix K usually contains the focal length of the camera (f x , f y ) and principal point coordinates (c x , c y ), which describe the geometric characteristics of camera imaging. The conversion relationship is roughly:

[0045]

[0046] In order to facilitate subsequent coordinate transformation, these three-dimensional points are usually expressed as homogeneous coordinates Where i ranges from 1 to N, and N is the total number of points in the point cloud. In this way, we get the original 3D point cloud in the camera's own coordinate system:

[0047]

[0048] This point cloud is the basis for all subsequent analysis and contains the geometric structure information of the scene.

[0049] However, the camera coordinate system is relative to the camera itself. As the platform carrying the camera (such as a tractor or drone) moves or changes its posture, the camera coordinate system will also change. In order to be able to analyze and compare in a unified and stable reference system, the point cloud in the camera coordinate system must be converted to the world coordinate system. The world coordinate system is a fixed reference system, for example, it can be set on the geoid or with a certain starting point as the origin. This conversion process requires knowing the posture of the camera in the world coordinate system, that is, its position and orientation. This is usually described by the external parameters of the camera, including a rotation matrix and a translation vector Here SO(3) represents the special orthogonal group of all 3×3 real rotation matrices. This rotation matrix and the translation vector together define the transformation from the world coordinate system to the camera coordinate system. To transform the point in the camera coordinate system Convert the point to the world coordinate system The inverse of this transformation needs to be applied. Specifically, the point in the world coordinate system The formula can be Calculated. Here Is the inverse of the rotation matrix. For a rotation matrix, its inverse matrix is equal to its transposed matrix. After this transformation, all three-dimensional points are placed in the same global reference frame. In this world coordinate system, the height coordinate of the point Usually it represents the actual height of the point relative to a reference plane (such as the horizontal ground), which is crucial for subsequent height analysis. and It can usually be estimated in real time through sensor fusion (such as combining GPS and IMU inertial measurement units) or visual odometry, SLAM (simultaneous localization and mapping) and other technologies.

[0050] Next, we will enter the height analysis part. The core task of this part is to transform the three-dimensional point cloud P into the world coordinate system. w The purpose of processing is to preliminarily identify which points may belong to the ground area and determine whether there is a recognizable ground in the scene. This is the basis for subsequent decision-making. The ground of cultivated land usually has relatively low height and relatively flat or gently sloping features. Therefore, the height information (i.e. the height in the world coordinate system) The value) is the key feature to distinguish the ground from non-ground (such as vegetation, rocks, shadows of agricultural machinery, etc.). The system will first The coordinates are analyzed to try to roughly divide the point cloud into candidate ground areas and non-ground areas. For example, the distribution histogram of height values can be analyzed. The height values of ground points are usually concentrated in a relatively low range. One possible method is to use a statistical model, such as a Gaussian mixture model (GMM), to fit the distribution of height values. Assuming that the height data is a mixture of multiple Gaussian distributions, representing different categories such as the ground, low vegetation, and higher obstacles, then the Gaussian component with the lowest mean is likely to correspond to the ground. By setting a certain probability threshold or selecting points belonging to this component, a candidate ground point cloud set can be obtained. At the same time, in order to exclude the influence of irrelevant areas in the distance or sensor edge noise, it is usually necessary to combine the spatial position information in the horizontal plane. Filtering is performed. For example, a region of interest (ROI) can be defined to process only points within a certain range in front of the camera. This region can be rectangular, circular, or more complex, such as an ellipse, defined by its center, major and minor semi-axes, and rotation angle. Intersecting or unioning the candidate ground points obtained from height segmentation with the points within the spatial ROI can generate a more reliable initial ground area point cloud. Height information alone is not enough; the ground typically has a certain degree of flatness. Therefore, the system also analyzes the local geometric features of this initial ground area point cloud, such as calculating the normal vectors of points within the local neighborhood of the point or fitting local facets and then analyzing the inclination angles of these normal vectors or facets. If the inclination angles in most areas are relatively small, below a set threshold (e.g., 15 or 20 degrees), the system determines that the ground has been detected. Conversely, if the initially extracted area is highly undulating or has generally large inclination angles, the system may determine that the ground has not been detected. This judgment directly influences the subsequent decision-making process.

[0051] Finally, there is the job decision allocation part. This part decides how to allocate the target area based on the results of the height analysis. Its processing logic is divided into two cases. The first case is that the height analysis part determines that "the ground is detected". In this case, the system believes that a reliable ground reference has been found. However, this initial ground area may not be accurate enough, may contain a small number of non-ground points or fail to cover all ground points. Therefore, a refined processing is required. Based on the initial ground area point cloud, the point cloud P in the entire world coordinate system can be screened in an iterative manner. w. For example, calculate the distance from all points to the currently estimated ground model (which may be a weighted distance, with the weights taking into account factors such as the reliability or density of the points), add or retain points with a distance less than a certain threshold in the fine ground area point cloud, and iterate repeatedly until the point cloud converges. This process can better eliminate noise and non-ground interference to obtain a more accurate and complete fine ground area point cloud. With a fine ground point cloud, the next step is to calculate the representative height of this area. Taking into account that the ground may not be absolutely flat, but has slight undulations or gentle slopes, it may not be appropriate to directly take the lowest or highest point. Therefore, the system calculates the weighted average ground height of all points in this fine ground area point cloud. The weights can be set based on a variety of factors, such as distance from the camera, measurement accuracy, point cloud density, etc. The goal is to get a robust value that represents the overall height level of the area. Calculate the weighted average ground height After that, the system will compare it with a pre-set cultivated land height threshold interval [h min , h max ] for comparison. This threshold interval is defined according to specific agronomic requirements and operation types. For example, for the land to be sown, the surface may be required to be very flat, with height fluctuations within a few centimeters. Then the threshold interval may be set to [-0.05m, +0.05m] (relative to the local average height). If the calculated weighted average ground height If the point cloud falls within this range, the system determines that the portion of the original target area corresponding to this fine ground point cloud is "cultivated land." This determination can be used to guide agricultural machinery operations, for example, controlling a planter to lower its actuators in the area for seeding, or recording the area as qualified cultivated land.

[0052] The second case is when the height analysis determines "ground not detected." This may occur when the camera's field of view is completely covered by tall, dense crops, there are large obstacles, or when sensor data quality is extremely poor (for example, due to strong light interference that invalidates the depth map). In this case, without reliable ground information as a reference, the system cannot effectively calculate ground height and determine the suitability of cultivated land. In this case, the system does not force an allocation decision based on the data of the current frame, as this could lead to incorrect operation (for example, sowing in an unsuitable area). Instead, the system skips the decision-making process for the current frame and directly processes the RGB image and depth map of the target area in the next frame. This means that the system has a certain degree of fault tolerance and continuous processing capabilities. It waits for data from subsequent frames, hoping to redetect the ground under the new perspective or scene. Then, based on the height analysis results of the next frame, it makes the appropriate determination of cultivated land and area allocation. This processing method ensures the reliability of the decision and avoids hasty decisions made when insufficient information is available.

[0053] Consider an autonomous tractor equipped with a camera operating in a field. The goal is to identify flat, bare soil areas suitable for precision seeding. The camera is mounted 2.5 meters in front of the tractor, looking slightly downward. At a certain moment, the camera captures a frame of data. The image acquisition component obtains a 640x480 pixel RGB image and the corresponding depth map. Assume that the camera intrinsic parameter matrix is Through these data, the homogeneous three-dimensional point cloud in the camera coordinate system is calculated Contains approximately N = 300,000 points. At the same time, the GPS and IMU fusion system on the tractor is used to obtain the current camera's attitude relative to the world coordinate system (the origin is set at the field entrance, and the Z axis is vertically upward): rotation matrix (For example, it means a rotation of -15 degrees around the X axis) and the translation vector The system uses these external parameters to Convert the point cloud P to the world coordinate system w At this point, each point of The value represents its approximate height from the ground (assuming that the XY plane of the world coordinate system is roughly parallel to the ground).

[0054] Next is the height analysis. The system extracts all the points Coordinates constitute the data set Z. Z is fitted using a Gaussian mixture model (GMM) with the number of components K = 3. Parameters are iteratively estimated using the expectation maximization (EM) algorithm. Assume that the mean values of the three components obtained from the fitting results are μ1 = 0.05 m, μ2 = 0.45 m, and μ3 = 1.5 m. The system selects the component with the smallest mean value as the ground component, i.e., k * =1, its mean m, standard deviation m. Set the posterior probability threshold δ = 0.7. The system calculates the posterior probability that each point i belongs to the ground component All satisfied The points are selected as candidate ground points, forming P cand At the same time, the system defines an elliptical region of interest (ROI), with its center at the ground projection point (x0, y0) 3 meters in front of the camera, semi-major axis a = 2.5 meters, semi-minor axis b = 1.8 meters, and rotation angle θ = 0 degrees. w The elliptic inequality is satisfied The points are extracted to form P ROI . The candidate ground point cloud P cand and the point cloud P of the region of interest ROI Take the intersection and get the initial ground area point cloud Pinitial_ground =P cand ∩P ROI Then, calculate P initial_ground The local inclination distribution of the midpoint. It is found that the inclination of most points is less than 10 degrees, which is lower than the preset inclination distribution threshold (for example, 20 degrees). Therefore, the height analysis part determines that "ground is detected".

[0055] Enter the task decision allocation part. Since the ground is detected, the system starts to perform fine processing. initial_ground Based on iterative screening P w For example, use RANSAC or similar methods to fit an initial plane, then calculate the weighted distance of all points to the plane (points with closer distance and higher depth measurement confidence have greater weights), add points with a distance less than 0.1 meters to the fine ground point set, and refit the plane, repeat several times, and finally obtain the fine ground area point cloud P fine_ground Assume that this point cloud contains about 50,000 points. Next, calculate the weighted average ground height of these points For example, using the inverse of the confidence level of the depth measurement at each point as the weight, we can calculate The system preset threshold range of cultivated land height is [-0.06m, +0.06m], and the condition is met. Therefore, the system finally decides: fine_ground The corresponding target area part (which can be determined by back-projecting to the original image or building an occupancy grid map) is assigned as the "arable land area part". This information is then sent to the tractor's control system, instructing the sowing mechanism to operate on this area in the next travel. If in another frame of the image, there is a dense weed in front of the camera's field of view, with a height of about 0.5 meters. Then in the height analysis, the lowest mean component fitted by the GMM It may still be close to 0 meters (representing the real ground under the weeds), but because the height values of a large number of points are concentrated around 0.5 meters, the posterior probability of belonging to the ground component is There are very few points greater than the threshold δ = 0.7, or the P formed by these points initial_ground Very sparse and discontinuous. Further local tilt analysis may result in the calculated average tilt or tilt variance exceeding the threshold due to irregularities in the weed surface. In this case, the height analysis component will determine that "ground not detected." Consequently, the work decision allocation component does not perform detailed processing, height calculation, or allocate cultivated land areas. The system directly processes the next frame of data and waits for clearer ground information before making a decision.

[0056] Furthermore, the height analysis part extracts the initial ground area of the three-dimensional point cloud in the world coordinate system and completes the height analysis to determine whether the ground is detected. The process specifically includes: performing adaptive threshold segmentation based on the Gaussian mixture model on the height direction of the three-dimensional point cloud in the world coordinate system to obtain candidate ground point clouds; performing fixed threshold segmentation based on elliptical area constraints on the distance of the three-dimensional point cloud in the horizontal plane in the world coordinate system to obtain point clouds of the area of interest; determining the initial ground area point cloud based on the candidate ground point clouds and the point clouds of the area of interest; calculating the local inclination distribution of the initial ground area point cloud, and determining the ground detection result based on the local inclination distribution and the preset inclination distribution threshold to determine whether the ground is detected.

[0057] The process first looks at the height information of the point cloud, that is, the Z-axis coordinate in the world coordinate system Based on a common assumption that the ground in a farmland environment is usually a relatively low and continuous surface in the scene, the system uses an adaptive threshold segmentation method based on a Gaussian mixture model (GMM) to process height data. Specifically, it first starts with the 3D point cloud P w Extract the Z-axis coordinates of all points to form a one-dimensional data set The system assumes that this height distribution can be composed of a mixture of K different Gaussian distributions (also called Gaussian components), each of which may represent a different type of ground feature, such as bare ground, low weeds, crop canopy, etc. The probability density function of GMM is:

[0058]

[0059] where π k is the mixing coefficient of the kth component (weight, satisfying ∑π k =1,π k ≥0), μ k and σ k are the mean and standard deviation of the kth Gaussian component, is the probability density function of a one-dimensional Gaussian distribution. The system usually uses the Expectation-Maximization (EM) algorithm to iteratively estimate the model parameters. This allows the model to best fit the observed height data Z. When the EM algorithm converges, the system obtains a statistical model of the scene height distribution. The next step is the adaptive threshold segmentation step: the system will select the mean μ k The smallest Gaussian component, whose index is k * =argmin k∈{1，...，K} μ k, because this component is most likely to represent the ground. Then, for each point i in the point cloud, the system calculates whether it belongs to this selected ground component k * The posterior probability This probability value reflects the height of point i The system sets a first judgment threshold δ (for example, δ = 0.7 or 0.8). Then point i is considered to be a ground point. All points that meet this condition are gathered together to form a candidate ground point cloud P cand The "adaptive" nature of this method is that the threshold segmentation is not based on a fixed height value, but on the statistical characteristics of the ground component learned from the data itself. and the posterior probability.

[0060] Relying solely on height information may not be enough to accurately define the ground area of interest. For example, there may be ground that meets the height conditions in the distance or to the side, but it is not the area that needs to be paid attention to in the current operation; or there may be noise points at the edge of the sensor. Therefore, the system then constrains the distance of the three-dimensional point cloud in the world coordinate system in the horizontal plane. This step adopts a fixed threshold segmentation method based on elliptical area constraints. It defines an elliptical area on the XY plane (horizontal plane) of the world coordinate system. This ellipse is usually centered on the projection point of the camera (or supporting platform) on the ground or the expected front focus point (x0, y0), and sets its semi-major axis a, semi-minor axis b and possible rotation angle θ. These parameters (x0, y0, a, b, θ) define a region of interest (ROI). For the point cloud P w For each point i in the system, the system checks its horizontal coordinate Whether it falls inside or on the boundary of this ellipse. This is determined by an inequality constraint. For example, for an ellipse with a rotation angle of θ, the condition is All points that meet this spatial constraint are extracted to form the point cloud P of the region of interest. ROI The threshold segmentation here is “fixed” because the parameters of the ellipse are usually pre-set or slowly updated based on the vehicle status information, rather than dynamically learned from each frame of data like GMM.

[0061] The candidate ground point cloud P based on height screening is obtained cand and the point cloud P of the region of interest based on spatial position screening ROI Afterwards, the system needs to combine these two pieces of information to determine the final initial ground area point cloud P initial_ground The commonly used method is to take the intersection of these two point clouds, that is, P initial_ground =P cand ∩PROI This means that a point must satisfy two conditions simultaneously: its height value is statistically likely to be considered as ground height, and it is spatially located within the current operating area of interest. The intersection operation can effectively filter out points that have a height similar to the ground but are outside the area (such as the road surface in the distance), and points that are within the area but have a height that is obviously inconsistent (such as rocks or tall grass in the area), thereby obtaining a more accurate and relevant initial ground estimate.

[0062] The last step is to use geometric features to analyze the initial ground area point cloud P initial_ground Verify and finally make a judgment on whether the ground is detected. Even if it passes the height and space screening, P initial_ground There may still be some non-ground points in the map, or the area itself may not be flat and suitable for farming. The ground, especially the cultivated land, usually has a relatively flat surface. Therefore, the system will calculate P initial_ground The local inclination distribution of the midpoint. This is usually achieved by: initial_ground For each point in the point cloud or on the point cloud, consider the points in a small neighborhood around it, use these neighborhood points to fit a local small plane, and then calculate the normal vector of this plane. The angle between the normal vector and the vertical direction of the world coordinate system (usually the positive direction of the Z axis) is the local inclination of the point. The system counts all the calculated local inclination values to form a histogram of the inclination distribution or calculates its statistical characteristics, such as the average inclination, the standard deviation of the inclination, or the proportion of points exceeding a certain angle threshold. Then, this inclination distribution feature is compared with a preset inclination distribution threshold. For example, a maximum allowable average inclination θ may be set. avg_max and a maximum allowable inclination standard deviation σ θ_max , or require it to be below a certain inclination threshold (such as θ point_max = 15 degrees) must be higher than a certain percentage (such as p min =90%). If the calculated tilt distribution meets the preset threshold, indicating that the area is generally flat and consistent with ground characteristics, the system will ultimately determine that "ground has been detected." Conversely, if the tilt angles are generally large or the distribution is very discrete, exceeding the threshold, it is considered that no reliable ground has been detected. This may be due to vegetation interference, rugged terrain, or data noise, and the system will be judged as "ground not detected."

[0063] Let’s take a practical example to illustrate: suppose the system on the autonomous tractor processes a frame of data, and the converted world coordinate system point cloud P w Already generated. The height analysis module extracts the Z coordinate Use GMM fitting with K=3 to get the lowest mean component Set the posterior probability threshold δ = 0.75. By calculating About 60,000 points were selected to form P cand At the same time, set the ellipse ROI parameters: the center (x0, y0) is the ground point 2 meters in front of the tractor, the semi-major axis a = 2.0m, the semi-minor axis b = 1.5m, and the rotation angle θ = 0. w Applying the ellipse constraint, we get P containing about 100,000 points. ROI . Take the intersection P initial_ground =P cand ∩P ROI , we get an initial ground area point cloud of about 50,000 points. initial_ground The local inclination is calculated from the points in the grid. The inclination values of all points are obtained by fitting the neighborhood plane. Statistics show that the average inclination is 5.2 degrees, the standard deviation is 3.1 degrees, and 96% of the points have an inclination of less than 15 degrees. Assume that the preset inclination distribution threshold is: the average inclination is ≤8 degrees, and the proportion of points with an inclination of less than 15 degrees is ≥90%. Since the calculation results are 5.2≤8 and 96%≥90%, both conditions are met. Therefore, the system determines that "the ground is detected" and the P grid composed of these 50,000 points is used as the ground. initial_ground If the analyzed area is a weed-covered plot, P initial_ground If there are too few points, or the calculated average inclination reaches 25 degrees, the system will determine that "ground is not detected." This detailed, multi-step analysis process ensures the accuracy and reliability of the initial ground area extraction, laying a solid foundation for subsequent farmland allocation decisions.

[0064] Furthermore, when the job decision allocation part determines that the ground is detected, iterative screening is performed based on the weighted distance from all points in the three-dimensional point cloud in the world coordinate system to the initial ground area point cloud to obtain a refined ground area point cloud.

[0065] Specifically, this refinement process employs an iterative screening strategy based on weighted distance. It not only considers the Euclidean distance between points, but also introduces the concept of "weighted" distance, allowing the screening process to more intelligently handle various complex situations. The calculation of this weighted distance can take into account multiple factors. For example, the confidence of the depth measurement of a point is an important weighting factor. Typically, depth cameras attach a confidence score to their measurements. Points with high confidence can have their corresponding distance weight increased (or, in other words, be given a higher tolerance when determining the distance threshold). Conversely, the influence of measurement points with low confidence (perhaps located at the edge of an object or far away) should be reduced. Another possible weighting factor is the consistency of the point normal vector. If the local surface normal vector of a candidate point is close to the average normal vector direction of the surrounding confirmed ground points, then it is more likely to belong to the ground, and its weighted distance can be considered smaller. Furthermore, point cloud density may also be considered; denser areas may represent more reliable surface information. By introducing these weights, the system can more robustly handle sensor noise, subtle ground cover, and measurement differences from different viewpoints, avoiding the false inclusion of non-ground points or exclusion of true ground points simply based on a fixed distance threshold.

[0066] The iterative screening process is as follows: First, the P obtained in the height analysis stage is initial_ground As the ground point set of the zeroth iteration. Then, enter the iteration loop. In each iteration, the system will be based on the current ground point set (for example, denoted as P ground_iter_k ) builds a temporary ground surface model. This model can be implicit, such as by calculating the distance from the query point to P ground_iter_k The distance to the nearest neighbor in the world coordinate system can also be explicit, such as using the current ground point to fit a local or global surface (such as a plane, a quadratic surface, or a more complex non-parametric surface model such as a moving least squares surface). Then, for the complete point cloud P in the world coordinate system, w Every point p in j (Or for efficiency, we can only consider P initial_ground Points in the neighboring area), calculate its weighted distance d to the current ground surface model weighted (p j, P ground_iter_k ). The system sets a distance threshold d thresh , this threshold defines the maximum allowable range that a point can deviate from the current estimated ground. All points that satisfy d weighted (p j, P ground_iter_k )≤d thresh Point p j are considered to be qualified ground points in the current iteration, and they together constitute the ground point set P for the next iteration ground_iter_k+1This process will be repeated, that is, ground_iter_k+1 Replace P ground_iter_k , rebuild the ground model, calculate the weighted distance of all points again and filter them. Iterations usually continue until a certain stopping condition is met, for example, the ground point set generated by two consecutive iterations has very little change (such as the number of points added or removed is less than a certain ratio), or the preset maximum number of iterations I is reached. max When the iteration stops, the final stable ground point set is determined as the fine ground area point cloud P fine_ground .

[0067] Assume that in the previous stage, the system has determined P containing about 50,000 points initial_ground , and it is determined that the ground is detected. The point cloud P of the entire scene w Contains 300,000 points. Now begins the refinement process. Set the maximum number of iterations I max =5, weighted distance threshold d thresh = 0.07 m. Weighted distance considering Euclidean distance d E and depth confidence c∈[0, 1], defined as d weighted =d E / (c+0.1) (add 0.1 to avoid division by zero). Iteration 1: Start. w Each point p j , calculate its The weighted distance to the nearest point in . All d weighted Point composition ≤0.07 Assume that after this screening, the point set size becomes 53,000, some noise points are removed, and some real ground points close to the initial ground but not previously included are absorbed. Iteration 2: Based on, repeat the above process. Calculate P w All points The weighted distance of the implicit surface represented by the point. Assume that the point set size is increased to 54,500, further including more points close to the ground. Iteration 3: Based on Filter and get Contains 55,100 points. Iteration 4: Based on Filter and get Contains 55,200 points. Compared to the above, the point set changes very little (only 100 points are added). Iteration 5: The preset maximum number of iterations I is reached max =5, or it stops early because the change is small enough. System output (or The final P contains 55,200 points. fine_ground , compared to the initial P initial_ground , which is expected to more accurately reflect the actual ground shape, remove more non-ground interference, and possibly fill some small holes caused by imperfect initial segmentation.

[0068] Furthermore, let the camera internal parameter matrix be Homogeneous 3D point cloud in camera coordinate system:

[0069]

[0070] Where N is the number of point clouds, Represents the three-dimensional coordinates of the i-th point in the camera coordinate system; is the X-axis coordinate of the i-th point in the camera coordinate system; is the Y-axis coordinate of the i-th point in the camera coordinate system; is the Z-axis coordinate of the i-th point in the camera coordinate system; represents a set of 3-row and 3-column matrices whose elements are all real numbers; let the rotation matrix be and the translation vector is The homogeneous three-dimensional point cloud in the camera coordinate system is converted to the world coordinate system according to the following formula to obtain the three-dimensional point cloud in the world coordinate system:

[0071]

[0072] T is the transpose operation; SO(3) represents the group of all 3-by-3 real rotation matrices;

[0073]

[0074] is the X-axis coordinate of the i-th point in the world coordinate system; is the Y-axis coordinate of the i-th point in the world coordinate system; is the Z-axis coordinate of the i-th point in the world coordinate system.

[0075] In precision agriculture applications, such as when a tractor or drone equipped with a camera is operating in a field, the camera's position and attitude are constantly changing. The origin of the camera coordinate system is usually at the camera's optical center, and the direction of its coordinate axes is closely related to the camera's orientation (for example, the Z axis is along the optical axis, the X axis is horizontal to the right, and the Y axis is vertically downward). If height analysis and farmland allocation decisions are performed directly in this local, moving coordinate system, effective comparison and integration across time and space will not be possible. For example, it is impossible to determine whether the absolute height of an area meets farming standards, nor can a globally consistent farmland map be generated. Therefore, all point cloud data must be converted to a unified, fixed reference coordinate system, namely the world coordinate system. The world coordinate system usually selects a reference that is fixed in the environment, such as a corner or entrance of a farmland as the origin. The X and Y axes are defined on the horizontal plane (possibly aligned with the geographic direction), and the Z axis is perpendicular to the horizontal plane and upward, usually opposite to the direction of gravity. The converted world coordinate system point cloud The coordinates of each point It has a global consistent meaning, especially It can directly reflect the height of the point relative to the world reference plane.

[0076] The key to achieving this coordinate transformation is to accurately know the real-time pose (position and attitude) of the camera in the world coordinate system. This is described by a set of quantities called camera external parameters, specifically a rotation matrix and a translation vector Rotation Matrix Is a 3×3 real matrix (belongs to R 3×3 set), and it belongs to the special orthogonal group SO(3), which means it is a pure rotation transformation that preserves distances and angles, and its determinant is +1. Defines the rotation operation required to rotate from the coordinate axis direction of the world coordinate system to the coordinate axis direction of the camera coordinate system. is a three-dimensional real vector (belonging to R 3 ), which describes the relative position relationship between the origins of the two coordinate systems. According to the provided transformation relationship description, here Refers to the coordinate vector of the origin of the world coordinate system in the camera coordinate system. These external parameters and They are usually not fixed. For cameras on mobile platforms, they need to be continuously estimated through real-time positioning and orientation systems (such as the fusion of GPS and IMU inertial measurement units) or visual SLAM (simultaneous localization and mapping) technology.

[0077] With the point in the camera coordinate system And the description of the transformation from world coordinate system to camera coordinate system and The goal is to calculate the coordinates of the point in the world coordinate system:

[0078]

[0079] This requires performing an inverse transformation from the camera coordinate system to the world coordinate system. The calculation process described is to perform this inverse transformation. First, the coordinates of the point in the camera coordinate system are Subtract the translation vector This step The result is a vector pointing from the world coordinate system origin to point i, but the coordinates of this vector are still expressed based on the axis direction of the camera coordinate system. Next, the coordinate representation of this vector needs to be rotated from the camera coordinate system back to the world coordinate system. This is done by multiplying the rotation matrix on the left. The inverse matrix of to achieve this. Represents the rotation from world to camera, its inverse represents the rotation from the camera to the world. For the rotation matrix belonging to SO(3), its inverse matrix is equal to its transposed matrix, that is, Here T stands for transpose operation. Therefore, multiply the vector obtained in the first step by The conversion of the coordinate axis direction is completed, and the coordinates of the final point i in the world coordinate system are obtained. right The same operation is performed on all N points in the world to obtain the complete three-dimensional point cloud P in the world coordinate system. w .

[0080] A tractor is moving along a farmland. The RealSense camera above it (about 2 meters from the ground) captures a marked point on the ground 5 meters ahead. In the camera coordinate system, the coordinates of this point may be Meters (assuming the Y axis is downward and the Z axis is forward). At this point, the camera's posture relative to the world coordinate system (the origin is at the field entrance and the Z axis is upward) is estimated by GPS / IMU as follows: (For example, the camera coordinate system has a pitch angle of 10 degrees and a yaw angle of 5 degrees relative to the world coordinate system, and the specific value is a 3×3 matrix), and the translation vector Meters (indicates the coordinates of the world origin in the current camera coordinate system). To calculate the position of this marker in the world coordinate system First calculate the difference vector Then, calculate The inverse matrix of (i.e., its transpose). Finally, multiply this 3×3 inverse rotation matrix by the left side of the difference vector: After matrix-vector multiplication, the result is This is the coordinate of the marker in the world coordinate system. For example, the calculated result may be (28.5, 55.2, 0.05) T meters, of which Meters directly reflects the height of the point relative to the origin of the world coordinate system (the ground at the entrance of the field). Repeating this process for all N points in the camera's field of view (for example, N = 300,000) places the geometry of the entire scene in a unified, physically meaningful world coordinate system.

[0081] Furthermore, the height direction of the 3D point cloud in the world coordinate system is segmented by an adaptive threshold based on a Gaussian mixture model. The process of obtaining the candidate ground point cloud specifically includes: extracting the Z-axis coordinates of all points from the 3D point cloud to form a data set Let the number of components of the Gaussian mixture model be K, then the one-dimensional Gaussian mixture model is expressed as:

[0082]

[0083] Among them, π k is the mixing coefficient of the kth Gaussian component, and satisfies π k ≥0;μ k is the mean of the kth Gaussian component; σ k is the standard deviation of the kth Gaussian component; Represents a one-dimensional normal distribution:

[0084]

[0085] Use the maximum expectation algorithm to iteratively estimate So that the model parameters {π k , μ k ,σ k}converges to the local optimal solution, at this time the Gaussian mixture model of the Z height distribution is obtained; according to the results of the Gaussian mixture modeling, the three-dimensional point cloud in the world coordinate system is adaptively threshold segmented in the height direction to obtain the candidate ground point cloud.

[0086] Okay, let's elaborate on how to use the Gaussian mixture model (GMM) to perform adaptive threshold segmentation of the height direction of the 3D point cloud in the world coordinate system in the image height analysis and allocation decision system of the cultivated land cloud model to obtain the candidate ground point cloud P. cand This step is one of the core links in the initial ground area extraction. It aims to intelligently identify the points that are most likely to belong to the ground in terms of height from a complex scene containing various ground objects (ground, vegetation, obstacles, etc.).

[0087] First, the input of the process is a 3D point cloud that has been transformed into the world coordinate system:

[0088]

[0089] Since the ground is mainly distinguished by its height relative to other objects, the focus of the analysis is on the vertical coordinate of the point cloud, that is, the Z-axis coordinate The system first starts from P w Extract the Z coordinate values of all N points to form a one-dimensional height data set This dataset, Z, contains the height distribution of all observed points in the scene. In a typical farmland environment, this height distribution is typically multimodal rather than monotonous, as the scene may contain a ground plane, vegetation of varying heights, crop residue, or other objects. Simply setting a fixed height threshold to segment the ground is often ineffective because it fails to adapt to the natural undulations and slope variations of the ground, as well as the presence of various non-ground objects.

[0090] In order to overcome the limitations of fixed thresholds, the system adopts a method based on Gaussian mixture model (GMM) to model the height distribution Z. The core idea of GMM is to assume that the observed height data Z is generated by K potential different categories (or components), and the height distribution of each category can be described by a one-dimensional Gaussian (normal) distribution. Therefore, the probability density function of the entire data set Z is It can be expressed as the weighted sum of these K Gaussian distributions. The specific form is:

[0091]

[0092] Here, K is a pre-set number of components (for example, K=3 based on experience, representing ground, low vegetation / debris, and tall vegetation / obstacles, respectively). For each component k (from 1 to K), π k is its mixing coefficient or weight, which indicates the proportion of the component in the entire mixture model. All π k are all non-negative and their sum is 1 ( π k ≥0). μ k is the mean of the kth Gaussian component, representing the average height of the category corresponding to this component (such as the ground). k is the standard deviation of the kth Gaussian component (its square is the variance), which describes the degree of dispersion or distribution width of the height value of this category. It is a standard one-dimensional Gaussian probability density function, and its specific form is:

[0093]

[0094] It gives the observed height value under the kth component The probability density of .

[0095] However, the parameters in the model For a specific scenario, the data Z is unknown and needs to be estimated through a learning algorithm. The system uses the widely used expectation-maximization (EM) algorithm to complete this task. The EM algorithm is an iterative optimization process that aims to find the parameter value that maximizes the probability of data Z appearing under the GMM model (i.e., the likelihood function). It mainly consists of two alternating steps: E-step (Expectation) and M-step (Maximization). In the E-step, based on the currently estimated parameters, the algorithm calculates the probability of each data point. The posterior probability (also called "responsibility" or "soft assignment") generated by each component k. In the M step, the algorithm uses the probability calculated in the E step as a weight to re-estimate the parameters π of each component k. k , μ k ,σ k , so that the updated parameters can better explain the weighted data. These two steps are iterated repeatedly until the model parameters converge to a set of local optimal solutions, that is, the parameters no longer change significantly. At this point, the EM algorithm successfully fits a Gaussian mixture model for the observed height data Z, and obtains a statistical description of the scene height structure.

[0096] With this fitted GMM model, the system can perform adaptive threshold segmentation. This segmentation process uses model parameters learned from the data instead of relying on fixed height values that may not be applicable to the current scene. Although the specific segmentation criteria (such as the posterior probability threshold) are defined in subsequent steps, the results of GMM modeling are the basis for this adaptive segmentation. For example, the system will usually identify the segment with the minimum mean Then, based on the complete GMM model (including all K components and their parameters), any height value can be calculated The probability of belonging to this inferred "ground component" is calculated. Points with sufficiently high probabilities are initially identified as candidate ground points. This process is "adaptive" because the segmentation criteria (which has the lowest mean, the shape and weight of each component) are completely dependent on the actual distribution of the Z data in the current frame, and can automatically adapt to different average ground heights and scene complexities.

[0097] For example, in an actual case, the tractor's front field of view includes both flat bare ground (actual height of about 0.0 meters), scattered weeds about 0.15 meters high, and a small amount of crop residues about 0.4 meters high. w Contains N = 250,000 points. Extract the Z coordinates to form the data set Z. Set the number of GMM components K = 3. Run the EM algorithm for iterative estimation. Assume that the model parameters obtained after the algorithm converges are: component Element Element The system identifies the component with the smallest mean 1 ( m) is most likely to represent the ground. Then, based on the learned GMM model results, the system will w Each point i in the GMM is judged (for example, calculating whether its posterior probability of belonging to component 1 is greater than a certain threshold δ). All points that pass this GMM-based adaptive judgment standard are classified as candidate ground points, forming a candidate ground point cloud P cand This GMM-based adaptive segmentation method can effectively distinguish potential ground areas from complex backgrounds, providing a basis for subsequent spatial constraints and geometric verification, and significantly improving the accuracy and robustness of cultivated land identification.

[0098] Furthermore, according to the results of Gaussian mixture modeling, the three-dimensional point cloud in the world coordinate system is segmented in the height direction by adaptive thresholding. The process of obtaining the candidate ground point cloud includes: in the one-dimensional height direction, the lowest cluster of points corresponding to the ground is set, and the mean μ is selected. k The smallest Gaussian component is used as the ground component Based on the selected ground composition is the mean of the selected ground components; is the standard deviation of the selected ground component; based on the posterior probability To determine whether the probability of a point belonging to the ground component is greater than the first judgment threshold δ:

[0099]

[0100] like Then confirm The corresponding points are candidate ground points; the points that meet the above posterior probability judgment are selected from the three-dimensional point cloud P in the world coordinate system w Take it out to form a candidate ground point cloud:

[0101]

[0102] Based on a common physical assumption that the ground is the lowest main continuous surface constituting the scene base in common farmland or natural environments, the system needs to identify the one that best represents the ground from the fitted K Gaussian components. This is done by comparing the mean of all components. The system selects the one with the smallest mean The component of * , that is, satisfy Once the ground component k is determined * , the corresponding learned mean and standard deviation It is used to represent the typical height of the ground in the scene and its variation range. For example, if three components are fitted, their means are 0.05 meters, 0.3 meters, and 1.0 meters respectively, the system will select the component with a mean of 0.05 meters as the ground component k * .

[0103] Next, for the world coordinate system point cloud P w Each point i (with height ), the system needs to quantitatively evaluate the ground component k at this point * How likely is it to be generated relative to the other non-ground components (j≠k * ) is generated. This is done by calculating whether the point belongs to the ground component k * The posterior probability This posterior probability is calculated based on the idea of Bayes' theorem, and its specific calculation method takes into account two factors: one is the height of point i With the ground component k * Gaussian distribution The degree of coincidence (i.e. likelihood), and the second is the ground component k * The proportion or prior probability of itself in the entire mixed model This joint contribution Normalized, that is, divided by the total probability density of the point i under the entire GMM model Therefore, the posterior probability The value range is between 0 and 1, which accurately quantifies the height at a known point i. Under the condition that the point is composed of the ground component k * The probability of being generated. A value close to 1 means that the altitude of the point is very consistent with the characteristics of the ground component and is much more likely to be generated by the ground component than by other components.

[0104] With the posterior probability of each point The system can then perform adaptive threshold segmentation. The system sets a predefined "first judgment threshold" δ, which is a value between 0 and 1, usually set based on experience or the requirements for classification confidence (for example, δ may be 0.7, 0.8 or 0.9). For each point i in the point cloud, the system calculates the posterior probability Compare with the threshold δ. If That is, the point belongs to the specified ground component k * If the probability of is greater than the set confidence threshold, then the system confirms that point i is a candidate ground point. On the contrary, if Then the point is not considered as a candidate ground point because it is more likely to belong to other non-ground components or the uncertainty of its attribution is high. This judgment process is "adaptive" because the judgment is based on It is calculated directly from the GMM model learned from the scene data, rather than relying on a fixed height value. For example, in a scene where the ground is generally high, will be higher accordingly, then a point with a higher absolute height may also be higher because of its If the height exceeds δ, it is classified as the ground. In the scene where the ground is generally low, the point with the same absolute height may be classified as the ground due to its If δ is not reached, it is not considered as ground.

[0105] The last step is to collect all the points that pass the posterior probability threshold to form a candidate ground point cloud P cand The system traverses the original world coordinate point cloud P w All points in The points i are retained and their complete world coordinates are maintained These selected points together constitute the candidate ground point cloud This P cand The point set is the preliminary ground estimation result obtained by adaptive segmentation based only on height information, which will serve as the input for subsequent steps (such as spatial ROI constraints and geometric property analysis).

[0106] For example, suppose that after GMM fitting, the ground component k is identified * =1 parameter is rice, meters, and its mixing coefficient The other components are k = 2 (mean 0.4 m) and k = (mean 0.9 m), with mixing coefficients of 0.3 and 0.2 respectively. Set the judgment threshold δ = 0.7. Now consider point A, whose height m. Calculate its posterior probability γ A，1, and found that its value (numerator is large, denominator is not much larger than numerator) is 0.95. Since 0.95>0.7, point A is confirmed as a candidate ground point. Consider point B again, its height m. Calculate its posterior probability γ B，1 , and finds that its value (the numerator is small because 0.25 is far from 0.1 and 0.25 is closer to the mean of component 2, 0.4) may be only 0.3. Since 0.3≤0.7, point B is not selected as a candidate ground point. The system performs such calculations and judgments on all N points, and finally gathers all points that meet the conditions like point A to form P cand This method of adaptive threshold segmentation using GMM and posterior probability enables the system to more intelligently and accurately identify ground areas in various complex scenarios, providing a solid foundation for farmland allocation decisions.

[0107] Furthermore, the distance of the 3D point cloud in the horizontal plane in the world coordinate system is segmented with a fixed threshold based on the ellipse region constraint. The process of obtaining the point cloud of the region of interest specifically includes: taking the center of the camera as the center of the ellipse (x0, y0), setting the semi-major axis a, the semi-minor axis b and the rotation angle θ, and using the following segmentation process to extract the point cloud of the region of interest from the point cloud:

[0108] Set inequality constraints:

[0109]

[0110] For each point in the point cloud The points that satisfy the above inequality constraints are taken from the three-dimensional point cloud P in the world coordinate system w Take it out and form the point cloud P of the region of interest ROI .

[0111] Furthermore, the process of determining the initial ground region point cloud according to the candidate ground point cloud and the region of interest point cloud includes: using the intersection of the candidate ground point cloud and the region of interest point cloud as the initial ground region point cloud.

[0112] Furthermore, the process of determining the initial ground region point cloud according to the candidate ground point cloud and the region of interest point cloud includes: taking the union of the candidate ground point cloud and the region of interest point cloud as the initial ground region point cloud.

[0113] The first method is to transform the candidate ground point cloud P cand and the point cloud P of the region of interest ROI The intersection of the two points is taken as the initial ground area point cloud P initial_ground This means that a 3D point will be included in P only if it satisfies both conditions. initial_groundMiddle: First, according to the previous height analysis based on Gaussian mixture model (GMM), the height of the point It is considered to have a high probability of belonging to the ground (that is, the point is at P cand Second, the horizontal position of the point must fall within a predefined spatial region of interest (e.g., an elliptical region) (i.e., the point is within P ROI Using the intersection P initial_ground =P cand ∩P ROI The core idea of this strategy is to strictly screen and only retain those points that have passed both height and spatial position verification. The advantage of this is that it can effectively filter out two types of interference points: one is points whose height features resemble the ground but are not located in the current area of interest (for example, distant roads or the ground of adjacent fields); the other is points whose heights are located in the area of interest but obviously do not conform to the ground features (for example, stones, tall weeds or equipment parts in the area). Therefore, the P obtained by the intersection initial_ground It usually has a higher confidence level, contains fewer non-ground points (false positives), and is a more "pure" ground point set. This provides a more reliable input for subsequent analysis that relies on the geometry of the point set (such as inclination calculation), which helps to reduce misjudgments. In the cultivated land cloud model image height analysis and allocation decision system, the intersection method helps to accurately define the scope of the land that is most likely to be suitable for farming, and avoid misjudging non-cultivated land areas as cultivated land areas. This is especially important for application scenarios that require precise operations (such as variable seeding or fertilization). However, the disadvantage of the intersection method is that it may be too strict. If the GMM segmentation is not perfect or the ROI area setting is slightly deviated, some real ground points located in the boundary area may be mistakenly excluded, resulting in P initial_ground Become sparse or patchy.

[0114] The second method is to transform the candidate ground point cloud P cand and the point cloud P of the region of interest ROI The union of the initial ground area point cloud P initial_ground This means that a 3D point will be included in P as long as it meets at least one of the two conditions. initial_ground Middle: Either the height of the point is judged by the GMM model as possibly belonging to the ground (the point is at P cand In the middle), or the position of the point falls within the region of interest (the point is in P RoI (in), or both are satisfied. Use the union P initial_ground =P cand ∪P ROIThe core idea of the strategy is to perform inclusive merging, and include as many points that may be related to the ground as possible in the initial consideration range. The advantage of this is that it retains information to the maximum extent and is less likely to miss real ground points. In particular, when the GMM classification probability of some ground points is not high or the ROI range does not completely cover all nearby ground points, the union operation may still include these points. Therefore, the P obtained by the union initial_ground It is usually larger and denser than the point set obtained by the intersection. However, a significant disadvantage of this method is that the risk of introducing noise and non-ground points is greatly increased. All points that are within the region of interest, regardless of their height (which may be tall crops or obstacles), will be included. At the same time, all points that are outside the region of interest but are mistakenly judged by the GMM to be ground height (which may be low objects in the distance or sensor noise) will also be included. This leads to P initial_ground The inclusion of a large number of non-ground points in the dataset makes subsequent geometric verification steps, such as inclination analysis, more difficult and unreliable, potentially requiring more robust outlier removal algorithms. In farmland identification applications, directly using the union results can lead to the initial identification of numerous areas unsuitable for cultivation as ground, increasing the risk of subsequent decision-making errors. While this may theoretically improve the recall of true ground points, the resulting loss in accuracy is generally unacceptable unless the system implements robust filtering mechanisms.

[0115] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. The cultivated land cloud model image height analysis and allocation decision system is characterized by: The system comprises: The image acquisition part is used to obtain the depth map corresponding to the RGB image of the target area, and construct a homogeneous 3D point cloud in the camera coordinate system based on the depth map and the camera internal parameter matrix. According to the camera's rotation matrix and translation vector, the transformation from the world coordinate system to the camera coordinate system is represented, and the homogeneous 3D point cloud in the camera coordinate system is converted to the world coordinate system to obtain a 3D point cloud in the world coordinate system. The height analysis part is used to extract the initial ground area of the 3D point cloud in the world coordinate system and complete the height analysis to determine whether the ground is detected; The operation decision allocation part is used to obtain a fine ground area point cloud when it is determined that the ground is detected, and calculate the weighted average ground height based on the fine ground area point cloud. When the weighted average ground height is within the set cultivated land height threshold range, the target area part corresponding to the fine ground area point cloud is allocated as the cultivated land area part; when it is determined that the ground is not detected, the RGB image of the target area of the next frame is height analyzed, and the cultivated land area is allocated according to the height analysis result of the RGB image of the target area of the next frame.

2. The cultivated land cloud model image height analysis and allocation decision system according to claim 1, characterized in that: The height analysis part extracts the initial ground area of the three-dimensional point cloud in the world coordinate system and completes the height analysis to determine whether the ground is detected. The process specifically includes: performing adaptive threshold segmentation based on the Gaussian mixture model on the height direction of the three-dimensional point cloud in the world coordinate system to obtain candidate ground point clouds; performing fixed threshold segmentation based on elliptical area constraints on the distance of the three-dimensional point cloud in the horizontal plane in the world coordinate system to obtain the point cloud of the area of interest; determining the initial ground area point cloud based on the candidate ground point cloud and the point cloud of the area of interest; calculating the local inclination distribution of the initial ground area point cloud, and determining the ground detection result based on the local inclination distribution and the preset inclination distribution threshold to determine whether the ground is detected.

3. The cultivated land cloud model image height analysis and allocation decision system according to claim 2, characterized in that: When the job decision allocation part determines that the ground is detected, it iterative screening is performed based on the weighted distance from all points in the three-dimensional point cloud in the world coordinate system to the initial ground area point cloud to obtain a refined ground area point cloud.

4. The cultivated land cloud model image height analysis and allocation decision system according to claim 3, characterized in that: Assume that the camera internal parameter matrix is Homogeneous 3D point cloud in camera coordinate system: Where N is the number of point clouds, Represents the three-dimensional coordinates of the i-th point in the camera coordinate system; is the X-axis coordinate of the i-th point in the camera coordinate system; is the Y-axis coordinate of the i-th point in the camera coordinate system; is the Z-axis coordinate of the i-th point in the camera coordinate system; represents a set of matrices with 3 rows and 3 columns whose elements are all real numbers; let the rotation matrix be and the translation vector is The homogeneous three-dimensional point cloud in the camera coordinate system is converted to the world coordinate system according to the following formula to obtain the three-dimensional point cloud in the world coordinate system: T is the transpose operation; SO(3) represents the group of all 3-by-3 real rotation matrices; is the X-axis coordinate of the i-th point in the world coordinate system; is the Y-axis coordinate of the i-th point in the world coordinate system; is the Z-axis coordinate of the i-th point in the world coordinate system.

5. The cultivated land cloud model image height analysis and allocation decision system according to claim 4, characterized in that: The height direction of the 3D point cloud in the world coordinate system is segmented by an adaptive threshold based on a Gaussian mixture model. The process of obtaining the candidate ground point cloud specifically includes: extracting the Z-axis coordinates of all points from the 3D point cloud to form a data set Let the number of components of the Gaussian mixture model be K, then the one-dimensional Gaussian mixture model is expressed as: Among them, π k is the mixing coefficient of the kth Gaussian component, and satisfies π k ≥0;μ k is the mean of the kth Gaussian component; σ k is the standard deviation of the kth Gaussian component; Represents a one-dimensional normal distribution: Use the maximum expectation algorithm to iteratively estimate So that the model parameters {π k ,μ k ,σ k }converges to the local optimal solution, at this time the Gaussian mixture model of the Z height distribution is obtained; according to the results of the Gaussian mixture modeling, the three-dimensional point cloud in the world coordinate system is adaptively threshold segmented in the height direction to obtain the candidate ground point cloud.

6. The cultivated land cloud model image height analysis and allocation decision system according to claim 5, characterized in that: According to the results of Gaussian mixture modeling, the three-dimensional point cloud in the world coordinate system is segmented by adaptive threshold in the height direction. The process of obtaining the candidate ground point cloud includes: setting the lowest cluster of points corresponding to the ground in the one-dimensional height direction, selecting the mean μ k The smallest Gaussian component is used as the ground component: Based on the selected ground composition is the mean of the selected ground components; is the standard deviation of the selected ground component; based on the posterior probability To determine whether the probability of a point belonging to the ground component is greater than the first judgment threshold δ: like Then confirm The corresponding points are candidate ground points; the points that meet the above posterior probability judgment are selected from the three-dimensional point cloud P in the world coordinate system w Take it out to form a candidate ground point cloud:

7. The cultivated land cloud model image height analysis and allocation decision system according to claim 6, characterized in that: The distance of the 3D point cloud in the horizontal plane in the world coordinate system is segmented using a fixed threshold based on the ellipse region constraint. The process of obtaining the point cloud of the region of interest includes: setting the center of the camera as the center of the ellipse (x0, y0), setting the semi-major axis a, semi-minor axis b and rotation angle θ, and extracting the point cloud of the region of interest from the point cloud using the following segmentation process: Set inequality constraints: For each point in the point cloud The points that satisfy the above inequality constraints are taken from the three-dimensional point cloud P in the world coordinate system w Take it out and form the point cloud P of the region of interest ROI .

8. The cultivated land cloud model image height analysis and allocation decision system according to claim 7, characterized in that: The process of determining the initial ground area point cloud according to the candidate ground point cloud and the area of interest point cloud includes: taking the intersection of the candidate ground point cloud and the area of interest point cloud as the initial ground area point cloud.

9. The cultivated land cloud model image height analysis and allocation decision system according to claim 7, characterized in that: The process of determining the initial ground area point cloud according to the candidate ground point cloud and the area of interest point cloud includes: taking the union of the candidate ground point cloud and the area of interest point cloud as the initial ground area point cloud.

Images