A method and system for evaluating terrain passability for tracked vehicles

Abstract

The present invention relates to the technical field of terrain accessibility analysis, and in particular to a terrain accessibility assessment method and system for tracked vehicles, comprising: obtaining point cloud information and image information around the tracked vehicle; fusing the point cloud information and image information after processing to obtain fused information; constructing an elevation map based on the fused information; and assessing the terrain accessibility of the tracked vehicle according to the elevation map combined with a cost function, wherein the cost function is used to comprehensively assess the terrain geometric attributes, image semantic information, the coupling relationship between the terrain geometric attributes and the image semantic information, and the vehicle's passability. The present invention enables a tracked vehicle to generate a terrain accessibility assessment result that matches the passability by sensing the environment and terrain information when traveling in an unstructured environment.

Description

Technical Field

[0001] The present invention relates to the technical field of terrain passability analysis, and in particular to a terrain passability assessment method and system for tracked vehicles. Background Art

[0002] Terrain accessibility analysis is one of the main tasks of autonomous navigation and is now widely used in fields such as extraterrestrial planet exploration, agricultural production, mining transportation, post-disaster search and rescue, and military. The scenes in the above fields often lack structured roads, and the terrain is rugged and varied. It is a very challenging task to identify areas where tracked vehicles can drive safely. The core issue of terrain accessibility analysis is how to obtain environmental data through on-board sensors, analyze and extract important information from these data, and evaluate the difficulty of vehicles passing through terrain.

[0003] There are some studies on terrain passability assessment. Based on sensor data, terrain is usually modeled in the form of digital elevation maps (DEM), and geometric features are extracted from the elevation maps to describe the terrain passability, such as roughness and slope. However, it is difficult to accurately analyze the terrain passability based on the geometric features of the terrain alone. For example, a tracked vehicle can easily pass through a bush, but it has a high roughness in the elevation map, which may result in an erroneous assessment result. Some researchers have combined semantic features with geometric features for analysis, but in actual environments, semantic information cannot be accurately extracted from images, and there is a lack of research on the impact of the connection between terrain geometric features and semantic information on passability. In addition, the results of terrain passability analysis should conform to the vehicle dynamics design parameters. Currently, relevant research is not sufficient.

[0004] Therefore, there is an urgent need for a terrain passability assessment method and system for tracked vehicles. Summary of the invention

[0005] The purpose of the present invention is to provide a terrain passability assessment method and system for tracked vehicles, which can enable the tracked vehicle to generate a terrain passability assessment result that matches the passability by sensing the environment and terrain information when traveling in an unstructured environment.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] A terrain passability assessment method for tracked vehicles, comprising:

[0008] Obtain point cloud information and image information around the tracked vehicle;

[0009] Processing and fusing the point cloud information and the image information to obtain fused information;

[0010] constructing an elevation map based on the fused information;

[0011] The terrain passability of the tracked vehicle is evaluated based on the elevation map in combination with a cost function, wherein the cost function is used to evaluate the passability by comprehensively considering terrain geometric attributes, image semantic information, the coupling relationship between the terrain geometric attributes and the image semantic information, and the vehicle's passability.

[0012] Optionally, fusing the point cloud information and the image information after processing includes:

[0013] Performing splicing processing on the point cloud information to obtain spliced ​​point cloud information;

[0014] Extracting and processing the image information to obtain semantic information of the image;

[0015] The spliced ​​point cloud information and image semantic information are fused to obtain the fused information, which is a point cloud containing semantic attributes and spatial position information.

[0016] Optionally, constructing an elevation map based on the fused information includes:

[0017] The elevation map is constructed according to a point cloud containing semantic attributes and spatial position information. The elevation map includes an elevation layer and a semantic information layer. The elevation layer is used to store elevation information of the terrain, and the semantic information layer is used to store terrain types and obstacle types.

[0018] Optionally, evaluating the terrain passability of the tracked vehicle according to the elevation map in combination with a cost function includes:

[0019] Generate a semantic score layer and a sensitivity layer based on the semantic information layer, wherein the semantic score layer is used to represent whether the terrain type is suitable for tracked vehicles to pass through with a weight value, and the sensitivity layer is used to represent the sensitivity of the terrain passability to the influence of the terrain geometric attributes with a weight value;

[0020] Generate a roughness layer and a slope layer based on the elevation layer, wherein the roughness layer is used to represent the undulation of the terrain, and the slope layer is used to represent the slope of the terrain surface;

[0021] Acquire the maximum climbing grade and the maximum roll stability angle of the tracked vehicle, and generate a vehicle trafficability layer in combination with the elevation layer, wherein the vehicle trafficability layer is used to indicate the difficulty of passing through the terrain;

[0022] A cost function is constructed based on the semantic score layer, the sensitivity layer, the roughness layer, the slope layer and the vehicle trafficability layer, and the terrain passability of the tracked vehicle is evaluated based on the cost function.

[0023] Optionally, obtaining the maximum climbing grade and the maximum roll stability angle of the tracked vehicle and generating a vehicle traffic capacity layer in combination with the elevation layer comprises:

[0024] Solving the average elevation within the range of action of the track and terrain of the tracked vehicle by convolution operation;

[0025] Predicting based on the average elevation and the positional relationship between the tracks, obtaining a pitch angle and a roll angle of the tracked vehicle;

[0026] The vehicle traffic capacity layer is obtained by calculating using the pitch angle, roll angle, maximum climbing grade, and maximum roll stability angle.

[0027] Optionally, the prediction method of the pitch angle and roll angle of the tracked vehicle is:

[0028]

[0029]

[0030]

[0031]

[0032] Among them, pitch x is the pitch angle of the tracked vehicle at the i-th grid when it travels along the x-axis of the geodetic coordinate system, y is the pitch angle of the tracked vehicle at the ith grid when traveling along the y-axis of the geodetic coordinate system, h i (δ)*k is the convolution operation at the i-th grid, B is the center distance of the vehicle track, b is the width of the vehicle track, roll x is the rolling angle of the tracked vehicle at the i-th grid when it travels along the x-axis of the geodetic coordinate system, roll y is the rolling angle of the tracked vehicle at the ith grid when traveling along the y-axis of the geodetic coordinate system, and L is the length of the ground contact section of the vehicle track.

[0033] Optionally, the method for calculating the pitch angle, roll angle, maximum climbing grade, and maximum roll stability angle in the vehicle traffic layer is:

[0034]

[0035] Among them, c i is the vehicle capacity value of the i-th grid, α max is the maximum climbing grade of the tracked vehicle, β max is the maximum roll stability angle of the tracked vehicle.

[0036] Optionally, the cost function is:

[0037]

[0038] Among them, t is the passability layer, r is the roughness layer, s is the slope layer, l is the semantic score layer, m is the sensitivity layer, c is the vehicle capacity layer, λ, μ and γ are the weights of the roughness layer, slope layer and vehicle capacity layer respectively, r th and th are the thresholds of the roughness layer and slope layer scores respectively, and ⊙ is the dot product operation of the matrix.

[0039] To further achieve the above-mentioned purpose, the present invention also provides a terrain accessibility assessment system for tracked vehicles, comprising: a data acquisition module, a data processing module, an elevation map establishment module, and a accessibility assessment module;

[0040] The data acquisition module is used to acquire point cloud information and image information around the tracked vehicle;

[0041] The data processing module is used to process and fuse the point cloud information and the image information to obtain fused information;

[0042] The elevation map building module is used to build an elevation map based on the fusion information;

[0043] The passability assessment module is used to assess the terrain passability of the tracked vehicle based on the elevation map in combination with a cost function, wherein the cost function is used to comprehensively assess the passability by taking into account terrain geometric attributes, image semantic information, the coupling relationship between the terrain geometric attributes and image semantic information, and the vehicle's passability.

[0044] The beneficial effects of the present invention are:

[0045] The present invention uses multiple sensors and data fusion methods for perception, so the environment and terrain information obtained are more detailed and the perception range is wider; semantic information is obtained through an open vocabulary semantic segmentation network to achieve the category of any object or terrain in any environment; by predicting the vehicle body angle when driving, the terrain difficulty can be analyzed based on the vehicle's passability, so that the passability evaluation result is more in line with the vehicle's performance parameters; a coupling relationship between terrain geometric attributes and semantic information is established and used for passability evaluation, thereby improving the passability analysis results in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0047] Figure 1 A flow chart of a method for evaluating terrain passability of a tracked vehicle according to an embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram of the installation of a laser radar and a camera according to an embodiment of the present invention;

[0049] Figure 3 Schematic diagram of the original point cloud and the spliced ​​point cloud according to an embodiment of the present invention;

[0050] Figure 4 A camera image and a semantic mask image according to an embodiment of the present invention;

[0051] Figure 5 A flow chart is generated for the accessibility layer of an embodiment of the present invention;

[0052] Figure 6 A schematic diagram of the position relationship between the non-zero weight part and the vehicle in an embodiment of the present invention;

[0053] Figure 7 The present invention is a schematic diagram of the structure of a terrain passability assessment system for tracked vehicles according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0056] This embodiment provides a method for evaluating the passability of terrain for tracked vehicles. Figure 1 As shown, including:

[0057] Step 1. Obtain the point cloud information around the vehicle and the image information in front;

[0058] Use multiple lidars to obtain point cloud information around the vehicle, and use cameras to obtain image information in front of the vehicle.

[0059] The multiple laser radars should include at least one rotating laser radar and at least one solid-state laser radar. At least one rotating laser radar should be placed on the top of the vehicle to cover the surroundings of the vehicle; at least one solid-state laser radar should face the front of the vehicle.

[0060] The installation scheme used is as follows Figure 2 As shown, it includes a mechanical laser radar located on the roof; two solid-state laser radars located at the front end of the car; and a camera located on the roof pointing to the front of the car.

[0061] Step 2. Process the point cloud information and image information;

[0062] The point clouds output by multiple LiDARs are stitched together to form a point cloud with wider coverage and higher density. The original point cloud and the stitched point cloud are as follows: Figure 3 The merging of point clouds is based on the relative spatial position of the sensors and is achieved by rotating and translating the point clouds.

[0063] Extract image semantic information based on image information, use open vocabulary semantic segmentation algorithm to extract image semantic information, and generate results and semantic mask images, such as Figure 4 The open vocabulary semantic segmentation algorithm can obtain detailed semantic information in any environment. The labels used by the open vocabulary semantic segmentation algorithm are shown in Table 1.

[0064] Step 3. Fuse the processed point cloud information and image information, and construct an elevation map based on the fused information;

[0065] The spliced ​​point cloud is fused with the image semantic information to obtain a point cloud containing semantic attributes. In addition to the spatial position information, the points in the point cloud also contain a semantic attribute.

[0066] Based on the point cloud containing semantic attributes, an elevation map is established. The elevation map is a raster map, including an elevation layer and a semantic information layer. The elevation layer stores the elevation information of the terrain, and the semantic information layer stores the type of terrain or obstacles.

[0067] Step 4. Calculate the feasibility of the terrain based on the elevation map and generate a traversability layer in the elevation map;

[0068] The specific process is as follows Figure 5 As shown, including:

[0069] (a) Generate semantic score layer and sensitivity layer based on semantic information layer. The semantic score layer represents whether the terrain type is suitable for tracked vehicles in the form of weight value. The weight value of the i-th grid is expressed as l i, l i ∈w l1 ,w l2 ,...,w ln The sensitivity layer represents the sensitivity of terrain accessibility to the influence of terrain geometric attributes in the form of weight values. The weight value of the i-th grid is expressed as m i , m i ∈w m1 ,w m2 ,...,w mn . Weight value l i and m i It changes with the change of terrain type. For a grid in the elevation map, according to its semantic information layer attributes, the semantic score layer and sensitivity layer weight values ​​are determined by querying the rule table. The rule table is detailed in Table 1.

[0070] Table 1

[0071]

[0072] (b) The terrain geometric properties are analyzed based on the elevation layer to generate the roughness layer and slope layer. The roughness layer is used to represent the undulation of the terrain. The roughness value of the i-th grid is represented by r i , which is calculated as:

[0073]

[0074] Among them, h i represents the elevation value of the i-th grid, h mean Represents the mean elevation of the N neighborhoods around the grid.

[0075] The slope value is used to represent the slope of the terrain surface, expressed as s i , the calculation method is:

[0076]

[0077] in, is the normal vector of the i-th grid terrain surface, Represents the positive direction of the z-axis.

[0078] (c) Generate a vehicle capability layer based on the elevation layer, the vehicle's maximum climbing grade and maximum roll stability angle. Based on a specific vehicle, the vehicle capability layer reflects the difficulty of passing a specific terrain. The accessibility map generated based on this layer will be able to more accurately reflect the difficulty of a vehicle passing through the terrain.

[0079] The method of generating the vehicle traffic layer is as follows: construct four convolution kernels to generate the vehicle traffic capacity layer, and solve the average elevation within the range of the track and the terrain through convolution operation. The vehicle body pitch angle pitch and roll angle roll are predicted based on the average elevation and the position relationship between the tracks. The pitch angle, roll angle and the vehicle maximum climbing grade α are used to calculate the vehicle traffic capacity layer. max and the maximum roll stability angle β max Perform normalization calculations to generate the vehicle traffic layer.

[0080] Specific:

[0081] The convolution kernels are denoted as k 1 , k 2 , k 3 and k 4 , they have the same size, and they approximately represent the projection of the tracked vehicle shape on the ground. The convolution kernel size h×w is calculated as:

[0082]

[0083] Among them, ceil() is the upward rounding function, min(A,B) means taking the minimum value of A and B, B is the center distance of the vehicle track, b is the width of the vehicle track, L is the length of the vehicle track grounding section, and p is the grid map resolution.

[0084] The weights in the convolution kernel are set manually. In a single convolution kernel, the projection area of ​​a single track is set to the same weight, and the rest is 0. A total of 4 different relative positions between the track and the vehicle are used to design the convolution kernel, so 4 convolution kernels are generated. The schematic diagram of the position relationship between the non-zero weight part and the vehicle is shown in the figure. Using this convolution kernel to perform convolution operations in the elevation layer will be able to obtain the average elevation within the range of action of the track and the ground.

[0085] like Figure 6 As shown, when the vehicle is moving along the positive direction of the x-axis of the geodetic coordinate system, the convolution kernel k is used. 1 Solve for the mean elevation of the terrain where the left track of the vehicle contacts. 1 The left m columns have weights The weights of the remaining columns are 0. Similarly, the convolution kernel k 2 It is used to solve the mean elevation of the terrain where the right track of the vehicle contacts. The non-zero weights are distributed in the rightmost m columns. Similarly, when the vehicle is traveling along the positive direction of the y-axis of the geodetic coordinate system (the direction orthogonal to the x-axis, and the x-axis and y-axis form a horizontal plane), the convolution kernels k are used respectively. 3 , k 4 The mean elevation of the terrain contacted by the left and right tracks of the vehicle is solved, and the non-zero weights are distributed in the m rows above and m rows below the convolution kernel, respectively.

[0086] Using the convolution kernel and the elevation layer, the pitch angle of the vehicle at the i-th grid when driving along the x-axis of the geodetic coordinate system is Pitch angle when driving along the y-axis of the geodetic coordinate system where h i (δ)*k represents the convolution operation at this grid. Similarly, the rolling angle of the vehicle along the x-axis of the geodetic coordinate system at the i-th grid is Pitch angle when driving along the y-axis of the geodetic coordinate system

[0087] The vehicle capacity value c of the i-th grid in the elevation map i The calculation method is:

[0088]

[0089] Among them, α max is the maximum climbing grade of the vehicle, β max is the maximum roll stability angle.

[0090] (d) Based on the above results, a cost function is constructed to generate a passability layer. The higher the cost, the easier it is for vehicles to pass. The cost function is:

[0091]

[0092] Among them, t is the passability layer, r is the roughness layer, s is the slope layer, l is the semantic score layer, m is the sensitivity layer, and c is the vehicle capacity layer; λ, μ and γ are weights, and their sum is 1; r th and th are the thresholds of the roughness layer and slope layer scores respectively; ⊙ represents the dot product operation of the matrix.

[0093] The cost function includes the geometric properties of the terrain, such as roughness and slope scores obtained through normalized calculations; it also includes a vehicle capacity layer to make the cost more consistent with vehicle parameters; it also establishes a coupling relationship between geometric features and semantic information through dot product calculations, making the passability assessment results in complex environments more accurate.

[0094] To further optimize the technical solution, this embodiment also provides a terrain passability assessment system for tracked vehicles, such as Figure 7 As shown, it includes: data acquisition module, data processing module, elevation map building module, and passability assessment module;

[0095] A data acquisition module, used to acquire point cloud information and image information around the tracked vehicle;

[0096] A data processing module is used to process and fuse the point cloud information and image information to obtain fused information;

[0097] An elevation map building module is used to build an elevation map based on fused information;

[0098] The passability assessment module is used to evaluate the terrain passability of tracked vehicles based on the elevation map combined with the cost function, wherein the cost function is used to comprehensively evaluate the terrain geometric attributes, image semantic information, the coupling relationship between the terrain geometric attributes and image semantic information, and the vehicle's passability.

[0099] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for evaluating the passability of terrain for tracked vehicles, characterized in that: include: Obtain point cloud information and image information around the tracked vehicle; Processing and fusing the point cloud information and the image information to obtain fused information; constructing an elevation map based on the fused information; The terrain passability of the tracked vehicle is evaluated according to the elevation map in combination with a cost function, wherein the cost function is used to comprehensively evaluate the passability of the terrain geometric attributes, image semantic information, the coupling relationship between the terrain geometric attributes and the image semantic information, and the vehicle's passability; The maximum climbing grade and the maximum roll stability angle of the tracked vehicle are obtained, and the vehicle traffic capacity layer is generated in combination with the elevation layer, including: Solving the average elevation within the range of action of the track and terrain of the tracked vehicle by convolution operation; Predicting based on the average elevation and the positional relationship between the tracks, obtaining a pitch angle and a roll angle of the tracked vehicle; Calculating using the pitch angle, roll angle, maximum climbing grade, and maximum roll stability angle to obtain the vehicle traffic capacity layer; The prediction method of the pitch angle and roll angle of the tracked vehicle is: Among them, pitch x is the pitch angle of the tracked vehicle at the i-th grid when it travels along the x-axis of the geodetic coordinate system, y is the pitch angle of the tracked vehicle at the ith grid when traveling along the y-axis of the geodetic coordinate system, h i (δ)*k is the convolution operation at the i-th grid, B is the center distance of the vehicle track, b is the width of the vehicle track, roll x is the rolling angle of the tracked vehicle at the i-th grid when it travels along the x-axis of the geodetic coordinate system, roll y is the rolling angle of the tracked vehicle at the i-th grid when it travels along the y-axis of the geodetic coordinate system, and L is the length of the ground contact section of the vehicle track; The method for calculating the pitch angle, roll angle, maximum climbing grade, and maximum roll stability angle in the vehicle traffic layer is: Among them, c i is the vehicle capacity value of the i-th grid, α max is the maximum climbing grade of the tracked vehicle, β max is the maximum roll stability angle of the tracked vehicle.

2. The method for evaluating terrain passability for tracked vehicles according to claim 1, characterized in that: Processing and fusing the point cloud information and the image information includes: Performing splicing processing on the point cloud information to obtain spliced ​​point cloud information; Extracting and processing the image information to obtain semantic information of the image; The spliced ​​point cloud information and image semantic information are fused to obtain the fused information, which is a point cloud containing semantic attributes and spatial position information.

3. The method for evaluating terrain passability for tracked vehicles according to claim 2, characterized in that: Constructing an elevation map based on the fused information includes: The elevation map is constructed according to a point cloud containing semantic attributes and spatial position information. The elevation map includes an elevation layer and a semantic information layer. The elevation layer is used to store elevation information of the terrain, and the semantic information layer is used to store terrain types and obstacle types.

4. The method for evaluating terrain passability for tracked vehicles according to claim 3, characterized in that: According to the elevation map combined with the cost function, evaluating the terrain passability of the tracked vehicle includes: Generate a semantic score layer and a sensitivity layer based on the semantic information layer, wherein the semantic score layer is used to represent whether the terrain type is suitable for tracked vehicles to pass through with a weight value, and the sensitivity layer is used to represent the sensitivity of the terrain passability to the influence of the terrain geometric attributes with a weight value; Generate a roughness layer and a slope layer based on the elevation layer, wherein the roughness layer is used to represent the undulation of the terrain, and the slope layer is used to represent the slope of the terrain surface; Acquire the maximum climbing grade and the maximum roll stability angle of the tracked vehicle, and generate a vehicle trafficability layer in combination with the elevation layer, wherein the vehicle trafficability layer is used to indicate the difficulty of passing through the terrain; A cost function is constructed based on the semantic score layer, the sensitivity layer, the roughness layer, the slope layer and the vehicle trafficability layer, and the terrain passability of the tracked vehicle is evaluated based on the cost function.

5. The method for evaluating terrain passability for tracked vehicles according to claim 1, characterized in that: The cost function is: Among them, t is the passability layer, r is the roughness layer, s is the slope layer, l is the semantic score layer, m is the sensitivity layer, c is the vehicle capacity layer, λ, μ and γ are the weights of the roughness layer, slope layer and vehicle capacity layer respectively, r th and th are the thresholds of the roughness layer and slope layer scores respectively, and ⊙ is the dot product operation of the matrix.

6. A terrain passability assessment system for tracked vehicles, characterized in that: The system is used to implement the terrain accessibility assessment method for tracked vehicles as described in any one of claims 1 to 5, the system comprising: a data acquisition module, a data processing module, an elevation map establishment module, and a accessibility assessment module; The data acquisition module is used to acquire point cloud information and image information around the tracked vehicle; The data processing module is used to process and fuse the point cloud information and the image information to obtain fused information; The elevation map building module is used to build an elevation map based on the fusion information; The passability assessment module is used to assess the terrain passability of the tracked vehicle based on the elevation map in combination with a cost function, wherein the cost function is used to comprehensively assess the passability by taking into account terrain geometric attributes, image semantic information, the coupling relationship between the terrain geometric attributes and image semantic information, and the vehicle's passability.

Images