Vision and laser radar fused agricultural machinery ground trafficability analysis method
By fusing vision and lidar, a multimodal access cost assessment system for farmland was constructed, which solved the problem of accessibility analysis for farmland, realized refined risk assessment and access cost quantification for farmland environment, and improved the safety and operation efficiency of autonomous tractors.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-14
- Publication Date
- 2026-03-10
AI Technical Summary
Existing technologies for farmland accessibility analysis suffer from challenges such as difficulty in environmental perception, weak scene generalization, limited ability to perceive surface physical properties, and difficulty in deploying high-precision perception algorithms on vehicle platforms. This results in autonomous tractors easily getting stuck and interrupting operations in low-accessibility areas such as mud and waterlogged areas.
A method combining visual and lidar analytics is employed to construct point cloud and semantic access cost mesh maps, fusing point cloud and image data to achieve multimodal access cost assessment of farmland. Specific steps include: acquiring point cloud and image data around agricultural machinery; constructing point cloud and visual access cost mesh maps; and fusing the two using a spatial transformation matrix to generate the final access cost ground mesh map.
It significantly improves the ability to identify low-accessibility areas such as mud and waterlogged areas, reduces the risk of vehicles getting stuck and skidding, improves the safety and continuity of route planning, reduces equipment wear and fuel consumption, and provides robust environmental perception solutions for various types of field scenarios.
Smart Images

Figure CN121640291A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for analyzing the ground accessibility of agricultural machinery, and more particularly to a method for analyzing the ground accessibility of agricultural machinery that combines vision and lidar. Background Technology
[0002] This section provides only background information relevant to this disclosure and is not necessarily prior art.
[0003] Currently, traditional agricultural production methods are facing an urgent need for transformation and upgrading. On the one hand, rural labor continues to flow out of the country, and labor costs are constantly rising; on the other hand, modern agriculture places higher demands on operational efficiency, precision, and resource utilization, prompting an accelerated increase in the level of agricultural mechanization and intelligence. Against this backdrop, autonomous driving tractors, as a new generation of intelligent agricultural equipment, are gradually becoming a key technological path to improve agricultural machinery operation efficiency, reduce reliance on manual labor, and achieve precision operations.
[0004] The core of autonomous tractor systems lies in environmental perception and autonomous control decision-making. Accurate perception of the operating environment is fundamental to ensuring system safety and operational reliability. Current research mainly focuses on positioning and navigation, obstacle detection, terrain mapping, farmland boundary extraction, and crop row recognition. However, research on farmland traversability analysis—a crucial issue affecting agricultural machinery operation safety and path planning efficiency—is relatively weak, becoming a significant bottleneck restricting further improvements in the intelligence level of autonomous agricultural machinery.
[0005] In actual farmland operations, tractors often face low-accessibility areas such as mud, waterlogged areas, and soft soil. Without timely obstacle identification and avoidance capabilities, tractors can easily get stuck, work interrupted, or even damage machinery, impacting work efficiency, fuel consumption, and personnel safety. Therefore, developing a accessibility algorithm suitable for autonomous tractor operation in complex field environments, and providing a communication cost map for autonomous tractor planning algorithms, has significant engineering value and theoretical significance for ensuring the safety and efficiency of agricultural machinery operations.
[0006] However, farmland accessibility analysis faces multiple technical challenges, including:
[0007] 1. Environmental perception is difficult and scene generalization is weak: Farmland is a highly unstructured environment with varied road shapes. The ground may be covered with various obstacles such as mud, puddles, crop residues, and stones. Moreover, the dusty and humid working environment seriously interferes with the imaging stability of sensors such as vision and lidar.
[0008] 2. Limited ability to perceive surface physical properties: Current perception technologies have weak capabilities in identifying physical properties such as surface humidity, texture, and bearing capacity. Traditional humidity sensors or geological analysis equipment are mostly contact-based or involve localized sampling, which is insufficient to meet the requirements of autonomous driving systems for large-scale, non-contact, and real-time prediction. 3. Difficulty in deploying high-precision perception algorithms on vehicle platforms: Although deep learning technology has made significant progress in semantic segmentation and surface recognition, its model parameters are complex and resource-intensive, making it difficult to achieve efficient deployment that balances real-time performance and accuracy on agricultural machinery platforms with limited computing power.
[0009] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention
[0010] Purpose of the invention: The technical problem to be solved by the present invention is to provide a method for analyzing the ground passability of agricultural machinery by fusing vision and lidar, which addresses the shortcomings of the existing technology.
[0011] To address the aforementioned technical problems, this invention discloses a method for analyzing the ground passability of agricultural machinery by fusing vision and lidar, comprising the following steps:
[0012] Step 1: Use the lidar installed on the agricultural machinery to acquire the raw point cloud data around the agricultural machinery and construct a point cloud passage cost grid map of the point cloud branches;
[0013] Step 2: Use the vision device installed on the agricultural machinery to acquire image data around the agricultural machinery and construct a semantic passage cost mesh diagram of the vision branch;
[0014] Step 3: Fuse the point cloud access cost mesh map and the semantic access cost mesh map to obtain the final access cost ground mesh map, which is used to evaluate ground accessibility.
[0015] Furthermore, the point cloud access cost mesh diagram for constructing point cloud branches described in step 1 includes:
[0016] Step 1-1: Preprocess the raw point cloud data to obtain preprocessed point cloud data;
[0017] Steps 1-2: A ground grid is constructed using a BEV grid size adaptive setting method based on point cloud density;
[0018] Steps 1-3 involve point cloud rasterization, which divides the preprocessed point cloud data from Step 1-1 into the ground grid constructed in Step 1-2, mapping the coordinates of each point in the point cloud to the ground grid index, as shown below:
[0019]
[0020]
[0021] in, These are the two-dimensional coordinates of the point cloud in the grid. It is a two-dimensional grid index in the ground grid after point cloud rasterization; and It is the entire ground grid. shaft and Minimum value in the axial direction; and It is the step size of the ground grid, set as a dimension. ;
[0022] Steps 1-4: Calculate grid height, extract height features from the ground grid, and perform convolution smoothing on the entire ground grid plane to fill in missing areas, as shown below:
[0023]
[0024] in, This is the completed ground elevation grid; This is the original ground elevation grid; This is a two-dimensional convolution operation; For smooth kernel;
[0025] Steps 1-5: Ground slope calculation, which calculates the ground slope by measuring the height changes in the grid.
[0026] Steps 1-6, quantification of passage cost, i.e., for each ground grid cell Construct the cost function Used to calculate the passage cost of point clouds;
[0027] Steps 1-7: Point cloud access cost map smoothing, that is, for each grid area within a preset range, calculate the average value of the point cloud access cost as the point cloud access cost for that area, and obtain the point cloud access cost map.
[0028] Furthermore, the construction of the semantic passage cost mesh graph for the visual branch described in step 2 includes:
[0029] Step 2-1: Preprocess the image data, including brightness normalization, color enhancement, and size adjustment;
[0030] Step 2-2: Extract multi-channel semantic features from the preprocessed image data, use an image recognition model to predict the image data, and output the predicted category of each pixel in the image data.
[0031] Steps 2-3: Based on the predicted category of each pixel, perform access cost quantization and construct a semantic access cost mesh graph, as follows:
[0032]
[0033] in, For pixels in image data The cost of passage For pixels The prediction category; The mapping function is represented as follows:
[0034]
[0035] in, The first step in ground semantic recognition category, For the first The cost after category mapping.
[0036] Furthermore, the fusion of the point cloud access cost mesh map and the semantic access cost mesh map mentioned in step 3 includes:
[0037] Step 3-1 involves representing each grid in the point cloud access cost grid map as a grid point, as follows:
[0038]
[0039] Step 3-2: Use the spatial transformation matrix to transform the grid points onto the image, finding the pixel grid in the image corresponding to the point cloud, as shown below:
[0040]
[0041] The spatial transformation matrix is the point cloud elevation grid. With image pixel grid The correspondence is represented as follows:
[0042]
[0043] in, This is the transformation matrix from the lidar coordinate system to the image coordinate system. The lidar coordinate system is a local three-dimensional coordinate system with the lidar itself as the origin. The image coordinate system has the upper left corner of the image as the origin, the horizontal axis is the u-axis, and the vertical axis is the v-axis. The position of the point is represented in pixels. This is the transformation matrix from the lidar coordinate system to the image coordinate system; This is the intrinsic parameter matrix of the vision device;
[0044] Step 3-3: The point cloud accessibility and semantic accessibility cost are fused, as shown below:
[0045]
[0046] in, The planar coordinate region of the ground The ultimate cost of passage, and The weight is the cost of passage.
[0047] Furthermore, the preprocessing of the raw point cloud data described in step 1-1 includes:
[0048] The range of point cloud data used to construct the point cloud branch access cost mesh is defined as follows:
[0049] Let the forward and backward direction of the agricultural machinery be... The axis, the left and right directions are Shaft, selected with dimensions of For a regular grid, the grid coordinate range is as follows:
[0050]
[0051] in, For grid length, Define the grid width; select point cloud data based on the grid length and width;
[0052] Based on the performance of the agricultural machinery, the passability restrictions are set as follows:
[0053]
[0054] in, For height, and These are the minimum and maximum heights that can be reached.
[0055] Furthermore, the construction of the ground grid described in steps 1-2 is represented as follows:
[0056]
[0057] in, For a single grid size, the grid is set to be square; It is the expected number of points per grid cell; It is the planar area of the entire regular grid, that is ; It refers to the number of point clouds within the grid area; and These are the lower and upper limits of the grid size.
[0058] Furthermore, the calculation of ground slope through height changes in the grid, as described in steps 1-5, includes:
[0059] The elevation changes in the horizontal and vertical directions are calculated and represented as follows:
[0060]
[0061]
[0062] in, It is the change in elevation in the horizontal direction. It is the change in elevation in the vertical direction. Represents the completed grid points Height;
[0063] The slope angle is calculated as follows:
[0064]
[0065] in, For grid points The slope angle.
[0066] Furthermore, the construction of the cost function described in steps 1-6 The cost of passing through a point cloud is used to calculate the following:
[0067]
[0068] in, For the sake of slope, For the price of uncertainty, The cost is the material and humidity of the floor. , and For weights; where slope cost , means as follows:
[0069]
[0070] in, The maximum slope threshold;
[0071] Uncertainty Cost , means as follows:
[0072]
[0073] in, This represents the number of point clouds within the grid.
[0074] Floor material and humidity cost , means as follows:
[0075]
[0076] in, For empirical functions, This represents the average reflection intensity of the point cloud within the grid.
[0077] Furthermore, the extraction of height features from the ground grid described in steps 1-4 includes: using average pooling, max pooling, or min pooling operations.
[0078] Furthermore, the image recognition model described in step 2-2 is the YOLOv8-seg model.
[0079] Beneficial effects:
[0080] 1. This invention integrates forward-looking visual images and lidar point cloud data to construct a surface accessibility analysis system suitable for farmland operation environments. It can identify high-passage-cost areas such as slippery, muddy, and waterlogged areas, and achieve refined risk assessment and quantification of passage costs.
[0081] 2. Compared with existing analysis methods that rely on a single sensor or are based solely on geometric features, this invention has significant advantages in terms of information redundancy, environmental adaptability, and accessibility.
[0082] 3. This invention can significantly reduce the risks of autonomous tractors getting stuck or slipping during field operations, improve the rationality of path planning and the continuity of operations, and at the same time reduce equipment wear and fuel consumption.
[0083] 4. This invention achieves robust traffic status perception in various types of field scenarios, providing a reliable and scalable environmental perception solution for agricultural autonomous driving. Attached Figure Description
[0084] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.
[0085] Figure 1 This is a schematic diagram of the overall structure of the present invention.
[0086] Figure 2 This is a schematic diagram of coordinate transformation from point cloud to semantic access cost map. Detailed Implementation
[0087] The overall concept of this invention is as follows:
[0088] 1. A multimodal access cost map constructed by integrating visual semantic segmentation and LiDAR point cloud data enables comprehensive evaluation of accessibility attributes such as field surface moisture, texture, and slope. This overcomes the limitations of traditional methods that rely solely on geometric features (such as height difference and point cloud sparsity) for accessibility analysis, significantly improving the ability to identify low-accessibility areas such as mud, waterlogged, and slippery areas. It allows for early detection and avoidance of risk areas, effectively reducing the probability of vehicles getting stuck and skidding, and improving the safety and continuity of path planning.
[0089] 2. A lightweight visual semantic segmentation network, YOLO-seg, is introduced to detect slippery areas, waterlogged areas, and muddy areas in farmland images in real time, and is fused with LiDAR point cloud information. This effectively reduces the inference burden on edge computing devices, improving the algorithm's deployment efficiency and real-time response capability. Compared to traditional semantic segmentation networks, the YOLO series models maintain high recognition accuracy while significantly reducing computational resource consumption, ensuring stable and rapid visual recognition and traffic risk perception on agricultural machinery vehicle platforms, meeting the dual requirements of real-time performance and energy efficiency in agricultural scenarios.
[0090] The overall technical solution of this invention is as follows: Figure 1 As shown, a vision-based and lidar-based ground accessibility analysis method for agricultural machinery is proposed, including:
[0091] I. Quantification of ground geometrical access cost of lidar point cloud branches.
[0092] 1. Input radar point cloud: Obtain raw point cloud data about the vehicle's surrounding environment using a lidar installed at an appropriate location on the tractor, and input it into the algorithm function;
[0093] 2. Point Cloud Preprocessing: Since the point cloud of a long-distance lidar is too sparse and may exceed the planned range, and the research focus of this invention is the passability of the ground, point clouds within a certain distance and height range from the original point cloud are selected, and the point clouds used are restricted to close range and the ground surface as much as possible.
[0094] Define the XY plane on the ground in front of the tractor, that is, Y (longitudinal direction of the tractor) parallel to the front of the tractor and X (lateral direction of the tractor) parallel to the left and right directions of the tractor. Use the center of the tractor's front axle as the origin of the coordinate system, and select the dimension (unit: meters / cm) as... A regular grid, the coordinate range corresponding to the entire grid: The height needs to be considered in conjunction with environmental information and the tractor's passability indicators to determine limitations. Then, a point cloud filtering algorithm is used to capture the point cloud within a predetermined range.
[0095] 3. Constructing the ground mesh: Since excessively large meshes will lose ground topographic features, and excessively small meshes will result in a large number of missing meshes, a BEV mesh size adaptive setting method based on point cloud density is adopted. The calculation formula is as follows:
[0096]
[0097] In the formula To adaptively set the individual grid size, the method sets the grid to a square; It is the expected number of points per grid cell; It is the planar area of the entire regular grid, that is ; It refers to the number of point clouds within the grid area; These are the lower and upper limits of the grid size.
[0098] 4. Point Cloud Rasterization: This involves dividing a 3D point cloud into corresponding planar grids. The core objective is to map the coordinates of each point to a 2D grid index. The basic principle is to discretize the coordinates using the grid step size (cell size) on the ground plane. The calculation formula is as follows:
[0099]
[0100] In the formula It is a two-dimensional grid index for point clouds; It is the entire grid interval Minimum value in the axial direction; The grid size and step size are set here to a size of [size value missing]. A square grid.
[0101] 5. Mesh Height Calculation: The height features within each mesh cell are extracted using average pooling. Alternatively, maximum and minimum pooling methods can be used. However, due to the sparsity of the point cloud, hollow meshes may appear within the entire ground mesh, reducing the representation of the ground geometry and hindering ground passability analysis. Therefore, convolution smoothing is performed on the entire mesh plane to fill in missing areas. The calculation formula is as follows:
[0102]
[0103] In the formula This is the completed ground elevation grid; This is for ground elevation grids that may have previously had gaps; This is a two-dimensional convolution operation; Smoothing kernels, such as mean kernels and Gaussian kernels, are used in this method. Standard-sized convolution kernel.
[0104] 6. Ground slope calculation: Ground slope can be calculated through the height changes in the grid. For each grid point... Estimate the elevation changes in the horizontal and vertical directions (approximate first-order partial derivatives):
[0105]
[0106]
[0107] in This is the size of the grid, which is set here to a size of [size missing]. A square grid.
[0108] slope angle Indicates the current angle of the grid's tilt towards the ground (unit: radians or degrees):
[0109]
[0110] Finally, the ground slope map was constructed.
[0111] 7. Traversability Cost Mapping: Traversability cost mapping is a crucial step in path planning and autonomous navigation. It aims to quantify the traversal difficulty of each grid cell based on ground geometry and physical properties, forming a two-dimensional traversability cost map to provide decision-making support for autonomous driving systems. For each ground grid cell… Construct peer cost function Taking into account: slope information Point cloud density and point cloud reflection intensity This is used to quantify ground undulations, uncertainties, ground material, and humidity, employing a weighted linear fusion calculation:
[0112]
[0113] In the formula For the cost of slope; This represents the maximum slope threshold that the system can tolerate.
[0114]
[0115] In the formula The cost of uncertainty; The minimum value after smoothing is 1, representing the number of point clouds within the grid.
[0116] =
[0117] In the formula The cost function is the cost of the ground material and humidity. It is an empirical function. For example, the reflectance of dry farmland (bare soil) is usually 0.2-0.3, while after rain when the humidity reaches 80%, the intensity can drop to below 0.1 because the water absorbs the laser.
[0118] The relationship between the reflection intensity of the mapped point cloud and the ground material and humidity is discrete and correlated.
[0119] The final total passage cost function model is as follows:
[0120]
[0121] In the formula The ground geometry access cost for point cloud branches; Hyperparameters are weighted by the costs of slope, reliability, material, and humidity.
[0122] 8. Point Cloud Access Cost Map Smoothing: Considering the existence of missing points and local noise within the mesh, the following spatial structure processing is performed to further improve the stability and usability of the point cloud access cost map: For each (Hyperparameters, if too large, will lose local geometric features, if too small, will result in poor smoothing effect) Calculate the average passage cost within the range, avoid extreme points affecting the path, and obtain the ground passage cost quantization map of the final point cloud branch.
[0123] II. Quantification of ground geometrical access costs in the visual branch.
[0124] 1. Image Input and Preprocessing: Acquire images from the forward-looking monocular / dual-view camera and preprocess them, including brightness normalization, color enhancement, and size adjustment. The size adjustment requires retaining the adjustment matrix for the calculation of the LiDAR-vision camera coordinate transformation matrix.
[0125] 2. Multi-channel semantic feature extraction: Since a pixel-level category semantic map is ultimately required, it is then converted into a pass cost quantization map for the visual branch. Therefore, the visual branch adopts the YOLOv8-seg model to output the category prediction for each pixel: (1) non-target ground areas such as dry fields and cement ground; (2) areas with moderate wet viscosity such as large paddy fields and small puddles; (3) areas with high wet viscosity such as sandy land, wet clay, and swamp; (4) solid obstacles such as stones, farm tools, hard equipment, and utility poles; (5) dynamic obstacles such as people, cattle, sheep, and dogs.
[0126] 3. Visual Branch Cost Quantization: To achieve the fusion of visual perception results with the point cloud branch cost quantization map and effective coupling with the agricultural machinery path planning module, a path cost quantization algorithm based on pixel-level semantic segmentation results is proposed. This algorithm assigns different path cost values to different categories of semantic information, outputting a visual branch cost map suitable for path planning.
[0127] Table 1. Category Label Definition Table
[0128] Category Number Category Name describe Dry fields, non-target areas Passable areas, concrete surfaces, and fields Large paddy fields and small puddles moderately slippery areas Sandy land, wet clay, marshland Difficult passage, vehicles may slip and get stuck solid obstacles Stones, utility poles, tool bags, etc. Dynamic obstacles People, animals and other moving entities
[0129] To quantify semantic classification into a cost of passage, this invention defines the following cost function:
[0130]
[0131] In the formula: For the predicted semantic category graph; The function for mapping categories to access costs is defined as follows:
[0132]
[0133] Ultimately, the ground access cost quantification of the visual branch and the drawing of the cost map are achieved.
[0134] III. Establishment of the geometric transformation relationship between radar and camera images.
[0135] Establishing a transformation matrix between the access cost mesh diagram formed by the point cloud branches and the access cost semantic mesh diagram of the visual branches is the transformation basis for the unified fusion of access costs. The coordinate transformation relationship is as follows: Figure 2 As shown.
[0136] The establishment of a complete transformation relationship includes the process of defining the coordinate system, calibrating parameters, and establishing transformation formulas. The complete transformation matrix is as follows:
[0137]
[0138] In the formula These are the transformation matrices from the radar grid coordinate system to the pixel coordinate system and from the radar coordinate system to the camera coordinate system, respectively. This is the camera intrinsic parameter matrix, pixel coordinate system → camera coordinate system.
[0139] By establishing the transformation relationship from the point cloud access cost mesh map to the semantic access cost pixel map, a bridge is provided for the fusion of the access cost results of the two branches. The final access cost fusion formula is as follows:
[0140]
[0141] In the formula The final, point cloud branch, and visual branch are indexed in the ground grid, respectively. The cost of ground access at that location; These are the cost weight hyperparameters for point cloud and vision, respectively, such as... .
[0142] In its specific implementation, this application provides a computer storage medium and a corresponding data processing unit. The computer storage medium is capable of storing a computer program, which, when executed by the data processing unit, can run the invention's content regarding a vision-and-laser fusion method for analyzing the ground accessibility of agricultural machinery, as well as some or all of the steps in various embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0143] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented using computer programs and their corresponding general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of computer programs, i.e., software products. These computer program software products can be stored in a storage medium and include several instructions to cause a device containing a data processing unit (which may be a personal computer, server, microcontroller, MCU, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.
[0144] This invention provides a conceptual framework and method for analyzing the ground accessibility of agricultural machinery by fusing vision and lidar. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.
Claims
1. A visual and laser radar fusion agricultural machine ground passability analysis method, characterized in that, The method comprises the following steps: Step 1: acquiring original point cloud data around the agricultural machine by using a laser radar arranged on the agricultural machine, and constructing a point cloud passable cost grid map of the point cloud branch; Step 2: acquiring image data around the agricultural machine by using a visual device arranged on the agricultural machine, and constructing a semantic passable cost grid map of the visual branch; Step 3: fusing the point cloud passable cost grid map and the semantic passable cost grid map to obtain a final passable cost ground grid map for evaluating the ground passability.
2. The method according to claim 1, wherein the method is characterized by, The step 1 of constructing the point cloud passable cost grid map of the point cloud branch comprises: Step 1-1: preprocessing the original point cloud data to obtain preprocessed point cloud data; Step 1-2: constructing a ground grid by using a BEV grid size self-adaptive setting method based on point cloud density; Step 1-3: point cloud gridding, that is, dividing the preprocessed point cloud data in step 1-1 into the ground grid constructed in step 1-2, and mapping the coordinates of each point in the point cloud to the ground grid index; Step 1-4: grid height calculation, extracting height features in the ground grid, and performing convolution smoothing on the entire ground grid plane to complete the missing area; Step 1-5: ground slope calculation, calculating the ground slope through the height change in the grid; Step 1-6, the passage cost quantification, i.e. for each ground grid cell , a cost function is constructed for computing the point cloud passage cost; Step 1-7: point cloud passable cost map smoothing, that is, calculating the average value of the point cloud passable cost of each preset range of grid area as the point cloud passable cost of the area, and obtaining the point cloud passable cost map.
3. The method according to claim 2, wherein, The step 2 of constructing the semantic passable cost grid map of the visual branch comprises: Step 2-1: preprocessing the image data, including brightness normalization, color enhancement and size adjustment; Step 2-2: performing multi-channel semantic feature extraction on the preprocessed image data, and using an image recognition model to predict the image data to output the predicted category of each pixel in the image data; Step 2-3: quantifying the passable cost according to the predicted category of each pixel to construct a semantic passable cost grid map.
4. The method according to claim 3, wherein, The step 3 of fusing the point cloud passable cost grid map and the semantic passable cost grid map comprises: Step 3-1: representing each grid in the point cloud passable cost grid map as a grid point; Step 3-2: converting the grid point to the image by using a space conversion matrix to find the pixel grid corresponding to the point cloud in the image; Step 3-3: fusing the point cloud passable cost and the semantic passable cost.
5. The method according to claim 4, wherein, The step 1-1 of preprocessing the original point cloud data comprises: Setting the range of the point cloud data used for constructing the point cloud passable cost grid map of the point cloud branch; According to the performance of the agricultural machine, setting the passability limit.
6. The method according to claim 5, wherein the method is characterized by, The step 1-2 of constructing the ground grid, that is, constructing the ground grid according to the expected number of points of each grid, the plane area of the entire regular grid, the lower limit and the upper limit of the grid size.
7. The method according to claim 6, wherein the method is characterized by, The step 1-5 of calculating the ground slope through the height change in the grid comprises: Calculating the elevation change in the horizontal direction and the vertical direction and calculating the slope angle.
8. The method according to claim 7, wherein the method is characterized by, The construction cost function described in steps 1-6 is used to calculate the point cloud traversal cost, that is, the point cloud traversal cost is calculated by using the slope cost, uncertainty cost, ground material and humidity cost and weight.
9. The method according to claim 8, wherein, The height feature extraction in the ground grid described in step 1-4 includes: using average pooling operation, maximum value pooling or minimum value pooling operation.
10. The method according to claim 9, wherein the method is characterized by, The image recognition model described in step 2-2 is a YOLOv8-seg model.